Problems with LVS-NAT and direct routing to network behind LVS.....

Julian Anastasov ja at ssi.bg
Thu Aug 30 03:34:27 BST 2001


	Hello,

On Thu, 30 Aug 2001, Pawel Kisiel wrote:

> > the real servers, you need forward_shared-2.4.5-1.diff from
> > http://www.linuxvirtualserver.org/~julian/
> 	so You mean that to use iptables to DNAT to the internal machines?
> If it is so.... I had problems with DNAT on iptables.....it didn't
> forwarded to the internal machines..like should do......
> I was doing sth like that:
> iptables -t nat -A PREROUTING -p tcp -d world_ip --dport 80 -j \
> DNAT --to 5.6.7.8:8080 ?!?! is it good?  I think it should be good
> but it didn't worked :((

	No, don't use DNAT. Think for the LVS box in DR mode as your
uplink router, the world clients reach your real servers without DNAT,
for example:

director:

# real servers on eth1:
ip add add 10.10.1.1/24 brd + dev eth1

# LVS-DR:
ipvsadm -A -t VIP:80 -s wlc
ipvsadm -a -t VIP:80 -r 10.10.1.X

# Allow the LVS box to play as gateway for the LVS-DR real servers
echo 1 > all/forward_shared
echo 1 > eth1/forward_shared

real server:

echo 1 > all/hidden
echo 1 > lo/hidden
# The HOWTO has info how to solve the ARP problem on W2K

# Add VIP
ip add add VIP dev lo

# reach the other network
ip route add 10.10.0.0/24 via 10.10.1.1 dev eth0

# your default route
ip route add default via 10.10.1.1 dev eth0

> 	Does this patch solve problems with arps in LVS-DR mode ?
> My machines that are hidden are linux boxes and win2k...so will it work??
> 	I know that with LVS-NAT id doesn't matter what kind of OS'es I'm
>  using but in the other ones it does!......

	Yes, you will need to solve the ARP problem. The web server
config should be changed too (we serve traffic to VIPs, not to private
addresses).

> > This patch allows the director to forward packets with local source (VIP)
> > when they come from the real servers (the input device).
>
> 	My real server will still have private range of ip addresess?
> Am I right ?

	Yes, the only public addresses will be the VIPs on lo (Linux).
The ARP probes will use the 10.10.1.0 network.

> > Then you will have identical setup with LVS-NAT. I assume you filter the
> > spoofed packets before the LVS box. If the LVS box is your firewall
> > you have to use two switched hubs, to split the internal from the external
> > networks. BTW, this is true even without using this patch. This patch
> > does not require higher level of security compared to the normal DR
> > or NAT setups.
>
> The only change to the configuration will be to change the -m to -g ??

	In LVS - yes, something like the above settings.

> > 	There are so many ways to break the things. Really, I don't
> > know what you have done. You have first -j MASQ for -s 10.10.1.0/24 port 80
> > for example and then -j ACCEPT for -s 10.10.1.0/24? How you control
> > when to SNAT and when not to SNAT between these private networks. The
> > LVS-NAT methos you are using requires the packets to be NAT-ed in both
> > directions. If you want to check everything you can start with
> > tcpdump outputs on all hosts:
> for FORWARDING
> iptables -A FORWARD -s 10.10.0.0/24 -d 10.10.1.0/24 -j ACCEPT
> iptables -A FORWARD -s 10.10.1.0/24 -d 10.10.0.0/24 -j ACCEPT
>
>
> for SNATING
> iptables -t nat -A POSTROUTING -s 10.10.1.0/24 -d ! 10.10.0.0/24 -j SNAT
> --to world_ips
>
> 	Correct me if I'm wrong with these roules.....?

	You don't distinguish traffic from the NAT-ed ports from the
other ports. OTOH, with LVS in 2.4 if you skip your NAT iptables rules
the setup can work, may be. LVS will detect its traffic from the real
servers on the FORWARD chain and will NAT only these replies. The
other traffic will be passed without SNAT-ing. So, you can try your
current setup without the iptables SNAT rules. If I'm not missing
something, this will solve all your problems. Hm, in fact, I'm not
sure what is going on, the setup should work even with this SNAT rule.
If the both networks are on same device (in the director) may be you
have to set all/send_redirects and eth*/send_redirects to 0. May be
you have to try these variants and if the problem is not solved may
be better to provide a tcpdump output from the director, on all
devices.

> Pawel Kisiel


Regards

--
Julian Anastasov <ja at ssi.bg>





More information about the lvs-users mailing list