[lvs-users] Sloppy TCP/SCTP, SH fallback and L4 hashing

Alexander Frolkin avf at eldamar.org.uk
Fri Jun 21 10:06:57 BST 2013


If case anyone is interested, I have backported two recent patches:


to 3.2 (e.g., for Ubuntu Precise).  You can find the (combined) patch
for 3.2 and the corresponding patch for ipvsadm at:


The patches add sloppy TCP and SCTP mode (net.ipv4.vs.sloppy_tcp/sctp)
which let IPVS create a connection on any TCP (SCTP) packet, not just a
SYN (INIT).  This allows connections to fail over between directors.

They also add layer 4 mode to SH, which just means that the source port
(TCP/UDP/SCTP), as well as the source address, is added to the hash.
This essentially gives you round-robin load balancing, but in a
predictable way (e.g., identically-configured directors will send the
same packet the same way, with no need for state sync).

Also, there is another flag which makes SH reselect a server if the
server selected on the first try is unavailable (e.g., weight 0).  This
lets you pause a server on a service using SH by setting its weight to 0
(the default behaviour is to reject new connections to that server).

The SH options are enabled through ipvsadm (with the patch applied)
using the -b (or --sched-flags) option to -A (or -E).  The option takes
a comma-separated list of flags, and the flags corresponding to the
above features are sh-port and sh-fallback.  So, for example, to create
a service with both enabled, you would do:

  ipvsadm -A -t -s sh -b sh-port,sh-fallback

We use these patches to run a cluster of directors in active-active
mode, with no state sync, with traffic spread across them using ECMP
routing.  If a director has to be removed, this can be done seamlessly
(without dropping connections) with the above features: sloppy TCP mode
will let the connection continue on another server and SH (optionally
with sh-port) will make sure that the new server sends the traffic to
the same realserver as the original director.  SH fallback allows
realservers to be paused for maintenance in such a way that connections
in progress are allowed to finish, and new connections are not rejected.

I'd be curious to know if anyone else finds this useful. :-)

Thanks to Julian Anastasov for his assistance with these patches!


More information about the lvs-users mailing list