LVS-NAT (without MASQ) and contacting real server directly?

Paul Wouters paul at xtdnet.nl
Fri Nov 30 14:52:32 GMT 2001


Hi there,

I have the following situation:

                                    LVS-Firewall
                         +--------------------------+
client a.b.c.d -GW--PtP-+ 1.2.3.253          1.2.3.1+--- 1.2.3.0/24
                         +--------------------------+
                                     VIP 1.2.3.200
                                                         RIP 1.2.3.199
                                                         RIP 1.2.3.198
                                                         RIP 1.2.3.197 ...

So my firewall does LVS-NAT for VIP onto RIP's.
I have no MASQ rules on the LVS machine, since all IP's involved are
routable IP from the same single /24. There is only one gateway to the
outside world, which is the LVS machine.

Although I read many things on the list about not doing it, it works fine :)
It's nice to be able to reach all RIP's seperately. 
ldirectd is used with wlc scheduling.

My problem is, I'd like to move this situation into place for other RIP's,
but I don't want to break the current server. So connections to RIP
directly should be possible. I've read similar things on the list, but I
don't understand why this wouldn't work. Connecting from client to a RIP
should just completely bypass all the lvs code, but it seems that the lvs
code is confused, and things a RIP->client answer *should* be part of its
NAT structure.

This is from tcpdump running on internal interface of the LVS machine:

First, a packet is received from client, and sent to RIP:
(The high port on a.b.c.d is prob due to the fact that my client is also
on a NAT on the other side of the city, cable modem :)

14:21:15.564326 > a.b.c.d.64866 > 1.2.3.198.www: S 2488432980:2488432980(0) win 5840 <mss 1460,sackOK,timestamp 6381088 0,nop,wscale 0> (DF) [tos 0x10] 

The RIP answers (which never reaches the client, the LVS is dropping it):
14:21:15.564516 < 1.2.3.198.www > a.b.c.d.64866: S 2269852048:2269852048(0) ack 2488432981 win 5792 <mss 1460,sackOK,timestamp 330600057 6381088,nop,wscale 0> (DF)

Maybe it tries to lookup the IP in its NAT table, can't find it, and assumes
something went wrong (instead of just leaving the packet alone) and drops
it.  The LVS then sends out a port unreachable:

14:21:15.564637 > 1.2.3.1 > 1.2.3.198: icmp: a.b.c.d tcp port 64866 unreachable [tos 0xc0] 

Seen from the client's point of view:

14:28:26.706229 6.7.8.9.3499 > 1.2.3.198.80: S 2738025006:2738025006(0) win 5840 <mss 1460,sackOK,timestamp 6404178 0,nop,wscale 0> (DF) [tos 0x10] 
14:28:29.697076 6.7.8.9.3499 > 1.2.3.198.80: S 2738025006:2738025006(0) win 5840 <mss 1460,sackOK,timestamp 6404478 0,nop,wscale 0> (DF) [tos 0x10] 


Playing with rp_filter didn't solve it. Logentries I see on the LVS are:

IPVS: mess proto_doff for proto=6, size =204
IPVS: I-pkt invalid packet data size
IPVS: mess proto_doff for proto=6, size =331
IPVS: I-pkt invalid packet data size


I'm still using LVS 0.9.3. I see some NAT changes were made, but at least
the changelog didn't seem to contain relevant updates. I'll try it with
0.9.7 but that one wouldn't compile ipvsadm out of the box.

Paul





More information about the lvs-users mailing list