[lvs-users] LVS Direct Routing Virtualized

Andrew Lau andrew at andrewklau.com
Sun Sep 22 02:16:07 BST 2013

On Sun, Sep 22, 2013 at 12:09 AM, Ferenc Wagner <wferi at niif.hu> wrote:

> Andrew Lau <andrew at andrewklau.com> writes:
> > On Sat, Sep 21, 2013 at 9:57 PM, Ferenc Wagner <wferi at niif.hu> wrote:
> >
> >> Andrew Lau <andrew at andrewklau.com> writes:
> >>
> >>> I have my LVS DR routing setup on a KVM nodes, from a single node it
> >>> works great.
> >>>
> >>> Client->VIP->router dnat->loadbalancer->real server->router
> >>> snat->VIP->client
> >>>
> >>> However when I spread the load across multiple KVM nodes, the
> >>> connections still hit the real server however nothing seems to be
> >>> going out
> >>>
> >>> Client->VIP->router dnat->LVS (kvm node 1)->real server (kvm node 2)
> >>>
> >>> I can see the traffic hitting the real server through the access
> >>> logs, and a tcpdump shows it's trying to send the response
> >>> out. being my virtual IP. But the client doesn't seem to
> >>> get the traffic.
> >>
> >> The client certainly gets the traffic, these seem like normal TCP
> >> session startups to me, with both ends actively involved.
> >
> > That's what it definitely looked like, the router packet scan even showed
> > the packets flowing from the VM however the client is not getting the end
> > result. Just timing out.
> I wonder who does the three way handshake and the data excange then,
> all with correct sequence numbers...  Make a packet trace on the client.
> > Could it be because I'm running NAT between the router->loadbalancer?
> is the DNAT target address for your real VIP, right?  So the
> packets go:
>                from       to             via
> ---------------------------------------------------
> until DNAT:    client     VIP      external gateway
> after DNAT:    client  internal gateway
> after DR:      client      realserver
> reply:   client    internal gateway
> after SNAT:     VIP      client    external gateway
> I don't see any problem with this in principle, especially not anything
> depending on the number of real servers.

Yup, that's the scenario.

My suspicions were correct though, when the LVS and realserver are on the
same KVM node the transmissions go through without an issue. But on
separate KVM nodes (same network) the real servers are receiving the
request but the client isn't getting anything and the tcpdump looked like
it was just the client sending re-transmissions to the real server, but the
real server packets aren't making it to the client.

If the client is within the same network though, it would seem to work so
it's pointing towards possibly a router issue. But I'm really stuck at
seeing why this is happening.


> --
> Regards,
> Feri.
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
> LinuxVirtualServer.org mailing list - lvs-users at LinuxVirtualServer.org
> Send requests to lvs-users-request at LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

More information about the lvs-users mailing list