[lvs-users] ipvsadm and packets leaving a gre tunnel

Joseph Mack NA3T jmack at wm7d.net
Tue Jul 22 19:19:52 BST 2008


On Tue, 22 Jul 2008, Marco Lorig wrote:

>> How do the clients know which datacenter to route to?
>
> The clients` (location1) default route points to the 
> director1, loc1. In some cases, not all (we have about 40 
> pairs of master/slave ipvsadm directors interconnected by 
> GRE tunnel ), packets are translated by iptables to reach 
> the correct destination (DNAT) "on the other" side 
> (location).

quite a setup

If director1 is the default route, then packets get to 
director2 via NAT and then if the realservers there are 
down, the packets come back by gre to director1?

>> can you set mss on this interface?
>>
> I can check this out in my testing area but the new 
> production systems are blades which are limited to four 
> gbit nics, which are bonded (linux bonding) to two pairs 
> (one bonding interface to client, one to server). We use 
> dot1q trunking to connect to different vlans. The GRE 
> Interface is set to 1476 mtu and nopmtudisc (both sides)

this is a little out of my area. Does nopmtudisc mean that 
the source side of the gre tunnel doesn't hear about the 
results of pmtu?

> I´m going to test it, if it makes any difference when I 
> set all physical interfaces to a lower mtu.

if that works, next try settting the mtu for the route 
(Ratz's method).

>> I didn't get this. You have a route from the client to
>> director1, through the gre tunnel, to director2 (with no
>> ipvsadm rules) to the realserver? (the realserver has a
>> public IP?)
>
> All networks are private address range.

so the client can route to the realserver then. I'm OK with 
that.

> just think about two linux routers, connected by gre. One 
> is directly connected to the clients network, the other is 
> directly connected to the realservers network.
>
> Now i start a file transfer. A client connects to the 
> realserver through router1 and router2. The realserver 
> serves the request well.
>
> Let´s start ipvsadm on the second router to balance to n 
> scp-servers. If you connect directly from client to a 
> specific server,

before running ipvsadm I take it

> it works, which means something happens 
> to the mtu behaviour. It seems like linux "discovers" the 
> mtu right or at least both sides can communicate. It also 
> seems like linux remembers mtu issues once discoverd for 
> about 10minutes. Connecting now and within the 10minutes 
> to the ipvsadm instance

to the VIP on director2 (rather than the realserver on
director2) I assume

> everything works fine.
>
> When the timeout is reached, a connection via ipvsadm 
> isn´t possible anymore.

new connection or the current connection as well?

> IMHO it looks like, a connection 
> through ipvsadm doesn´t return any or not the right 
> size/information for mtu. If the size/information was 
> discoverd before (e.g. by connection directly) it works 
> fine.
>
> I´m wondering, why ipvsadm can use discovered information 
> but can´t discover it by itself? Haven´t found any 
> information about detaild mtu processing in 2.6. at all... 
> Any hints welcome ;)

Julian's comments in the MTU section of the HOWTO say that 
Linux doesn't do PMTU on ipip tunnels in 2.6 (what it does 
in 2.4 I don't know)

Julian,

 	Marco's setup has the VIP on a interface which 
receives its LVS packets via a gre tunnel. It worked for 
his 2.4 setup, but stopped working (presumably due to pmtu 
problems) when he switched to 2.6.

 	Any ideas?

Thanks Joe

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!


More information about the lvs-users mailing list