[lvs-users] ipvs does not sync DNATted or fwmarked connection state

Patrick Schaaf netdev at bof.de
Mon Dec 20 20:43:35 GMT 2010

Is the following known / does a solution exist?

I'm setting up two machines with kernel as master/backup ipvs
directors, with keepalived checking real servers and implementing vrrp

Virtual service is for HTTP connections, using NAT method towards the
real servers.

The basic setup has been working fine, with an exemplary set of three
virtual IPs balancing to some real servers, replicating connection state
(ipvsadm -ln counters increasing on the backup, -lc state visible

However, for the production setup, I have to implement roughly 200
different virtual IP addresses, all running onto the same (rather small)
set of real servers.

As is well known, doing that with the corresponding number of different
ipvs virtual services presents problems, as the real server state
(connection count) is kept for each individual virtual service,
resulting in suboptimal balancing.

As a solution to that, I have been testing two different approaches:

1) using fwmark, with --set-mark in the mangle table to mark the
incoming packets for the different virtual IPs, and an fwmark virtual
service set up as usual.
	iptables -t mangle -A PREROUTING -m ... -j MARK --set-mark 80
        ipvsadm -A -f 80 ...

and alternatively

2) using iptables DNAT in PREROUTING to rewrite the various virtual IPs
to specific (few) virtual IPs set up as ipvs services.
        iptables -t nat -A PREROUTING -m ... -j DNAT --to-dest
        ipvsadm -A -t ...

Both approaches work fine WRT balancing, reaching the real servers, and

BUT: no connection state is synchronized, in either of the approaches.
The backup server does not show -ln counter increase, nor -lc
connections, when I test it.

I have even set up the fully working (normal) approach at the same time
as as 1) and/or 2), for different addresses, and the sync-to-backup is
working OK for the normal addresses, but not sending connection state
for  stuff covered by approaches 1) or 2).

Any suggestions as to why this happens? Patches to apply? Good chance
2.6.37-rcX could work? More info needed?

best regards

More information about the lvs-users mailing list