[lvs-users] LVS + Xen issue

Matthias Saou thias at spam.spam.spam.spam.spam.spam.spam.egg.and.spam.freshrpms.net
Wed Aug 8 13:18:02 BST 2007

Joseph Mack NA3T wrote :

> On Wed, 8 Aug 2007, Matthias Saou wrote:
> >> what happens if you have the director(s) on a separate host,
> >> ie not the Xen host?
> >
> > Then it works.
> My guess is that it's related to the problem that you need 
> the director to be a separate box. However there is 
> localnode. Horms figured out how to have several realservers 
> working in localnode mode.
> http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.localnode.html#two_node_localnode
> want to see if that works?

I can't set up localnode, as the director and the real server are two
separate "logical" servers, no loopback possible. I doubt my problem has
anything in common with this, unfortunately.

I'm still convinced it has something to do with connection tracking and
bridges, but I still haven't been able to debug it.

Basically packets go like this when the issue is seen :
- dom0 peth0 ->
- dom0 xenbr0 ->
- dom0 vif7.0 ->
- domUa eth0 -> This is where LVS is running
- domUa eth1 ->
- dom0 vif7.1 ->
- dom0 xenbr1 ->
- dom0 vif10.1 ->
- domUb eth1 -> This is where the web server answers
- dom0 vif10.1 ->
- dom0 xenbr1 ->
- dom0 vif7.1 ->
- domUa eth1 -> This is where SNAT/MASQUERADE occurs
- domUa eth0 ->
- dom0 vif7.0 ->
- dom0 xenbr0 ->
- dom0 peth0 -> Back to the Internet

dom0 : Xen Host
domUa : Xen guest running LVS+NAT using dom0's vif7.0 and vif7.1
domUb : Xen guest running a web server using dom0's vif10.1 only

There is nothing "fancy" in my setup, meaning that I've only configured
the minimum possible iptables rules to get things working, and it
actually works but only sends back partial files to the client. With a
test php script doing a phpinfo() I always got around 12kB, but I since
tried with a simple static file from which I always get exactly 16384
Bytes, while the file itself is a few hundred Bytes long. I'm pretty
sure that value of 16384 Bytes isn't a coincidence...

When domUa queries a real server on a different physical machine, the
main difference is that instead of going through xenbr1, from vif7.1 to
vif10.1, it goes to peth1 and off to the other Xen Host's NIC. But it
actually "stays inside xenbr1" too, which is why I'm confused.

Tips to try and debug this setup would be very welcome too ;-)


Clean custom Red Hat Linux rpm packages : http://freshrpms.net/
Fedora release 7 (Moonshine) - Linux kernel
Load : 0.26 0.34 0.35

More information about the lvs-users mailing list