[lvs-users] Understanding granularity, timeouts and unexpected balance of traffic on reals

Abhijeet Rastogi abhijeet.1989 at gmail.com
Thu Aug 15 21:11:00 BST 2019


Hi everyone,

I'm investigating a typical configuration for an L4 TCP load balancer using
ipvs+keepalived. Settings:-

persistence_timeout: 120 seconds.  (# LVS persistence timeout, sec)
/sbin/ipvsadm --set 1800 120 300 (30 min timeout for TCP)
persistence_granularity: "48" for ipv6.
lb_algo: rr (round-robin)

My expectation is, all the IPs from the same /48 v6 subnet should always
reach the same real_server because of setting granularity. (at least the
connections created in last 120 seconds)

However, I can see that established connections from the same /48 v6 subnet
are spread across multiple reals, even for recently established
connections.

# Same /48 going to different reals, (very recent connections)
1. Grepping with only first 3 quibble to see how a /48 is being
distributed.
2. " | grep ESTAB | grep 29: | head -n 100" to only see first 100
established connections created in last 60 seconds as my timeout is set to
30:00 (1800 seconds)
4. 6th column is the real IP.
I see that the same /48 is getting distributed across multiple different
reals. (should be same real because of persistence_granularity set to 48).
$ sudo ipvsadm -lnc | grep "xxxx:xxxx:xxxx" | grep ESTAB | grep 29: | head
-n 100 | awk '{print $6}' | sort | uniq -c
      2 [V6IP_REDACTED:9222]:443
      9 [V6IP_REDACTED:9223]:443
      7 [V6IP_REDACTED:9224]:443
     13 [V6IP_REDACTED:9225]:443
      1 [V6IP_REDACTED:9226]:443
      ............
      ............ output redacted


   - Why are recent connections going to different reals?
   - For recent connections, shouldn't they always end up on same real?
   - For older connections, I guess, persistence_timeout causes the traffic
   to balance to other reals via round robin.


Thanks in advance!
--
Cheers,
Abhijeet (https://abhi.host)


More information about the lvs-users mailing list