[lvs-users] UDP load balancing (Virtual server)

Simon Bernard contact at simonbernard.eu
Tue Nov 20 17:28:53 GMT 2018

I tried to experiment this use case and faced this behavior...

Let's say I have a client C1 with IP address : C1IP.
A loadbalancer server LB with virtual IP address : VIP.
2 cluster-nodes (R1 and R2) with IP address : R1IP and R2IP.

All the UDP traffic is done using port 5685.
To simulate traffic I run netcat(nc) on C1, R1 and R2.

---- Configuration on LB : -----------------------
$ ipvsadm -Ln

UDP  VIP:5685 rr persistent 1200
   -> R1IP             Masq    1      0          0
   -> R2IP             Masq    1      0          0

$ iptables -L -t nat -n
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

target     prot opt source               destination
MASQUERADE  all  --  R1-R2subnet/16

Now I send an udp packet from C1 to LB. I get message on R2

Using  "sudo ipvsadm -Lnc" on LB, I see

pro expire state       source             virtual destination
UDP 04:54  UDP         C1IP1:5686   VIP:5685   R2IP:5685
UDP 19:54  UDP         C1IP1:0         VIP:5685   R2IP:0

I reply from R2, all is ok (checked with tcpdump)

I check with conntrack -Ln | grep R2IP and I see nothing (ok probably 
handled by LVS)

Now I send a packet from R1 to C1 (server initiated use case), I 
successfully received packet on C1.
I can see a new entry in conntrack/netfilter

udp      17 24 src=R1IP dst=C1IP sport=5685 dport=5685 [UNREPLIED] 
src=C1IP dst=VIP sport=5685 dport=5685 mark=0 use=1

(this entry has a 30s lifetime)

If I reply in time from C1 to LB. Answer go to R1. and in conntrack I 
can still see the entry (not in UNREPLIED state anymore)
After several exchange the state of the entry go to ASSURED and the life 
time is now around 180s.

While I do that I checked the LVS entries using sudo ipvsadm -Lnc and C1 
is still associated to R2 although packet go to R1 ...

Once the entry in conntrack timeout,  if I send again an UDP packet from 
C1 to LB, R2 will receive it. (as long as the entry in LVS is alive)

(I also tested by removing entry in netfilter/conntrack using conntrack 
-F to flush all entries instead of waiting for lifetime expiration)

I was hopping that when R1 talked to C1 (server initiated use case), the 
entry in LVS would have been modified and would have not been handled in 
I mean is there a way to do ? does it make sense to you ?

(I hope this is clearer)


Le 19/11/2018 à 14:59, Simon Bernard a écrit :
> Hi,
> I would like to understand if it's possible to hide a cluster of servers
> behind 1 public IP address.
> Communication between my clients and my server is done over UDP.
> Communication can be initiated by clients or by one cluster-node.
> For client initiated use-case, if a client C1 with C1IP ip address send
> an udp packet to the public VIP address. The loadbalancer receives the
> packet, examines it and searches if there is a "rule" for it. If there
> is one, it rewrites destination IP choosing a server (let's say R1 with
> R1IP address) depending of loadbalancing algorithm and it adds in kind
> of association table that C1IP is attached to R1IP. In UDP, this
> association has a configurable lifetime.
> Does it works for Server initiated use-case ?
> If I send an udp packet from server-node R2 (R2IP) to C2 (there is no
> association entry for this IP), does it create a new entry (C2IP<=>R2IP)
> in the association table.
> What happened when I send an udp packet from R3 (R3IP) to C1 although
> C1IP is already associated to  R1IP ? does it replace the previous
> association.
> Simon
> _______________________________________________
> Please read the documentation before posting - it's available at:
> http://www.linuxvirtualserver.org/
> LinuxVirtualServer.org mailing list - lvs-users at LinuxVirtualServer.org
> Send requests to lvs-users-request at LinuxVirtualServer.org
> or go to http://lists.graemef.net/mailman/listinfo/lvs-users

More information about the lvs-users mailing list