[lvs-users] LVS-TUN BUG? - Intermittent incorrect source IP in TUN header for non-local realsevers (PBR no help)

Julian Anastasov ja at ssi.bg
Thu Jun 18 21:05:33 BST 2015


On Thu, 18 Jun 2015, Michael Vallaly wrote:

> If my understanding of this is correct this means that the IPVS TUN
> code just binds its socket to all interfaces, which leverages the
> kernel routing code to select the "best" interface?

	We simply get source from routing but ensure
with second lookup that IP rules are considered.

> I was under the impression that the routing cache was removed from the
> kernel after 3.6? I see there is some sort of i4flow caching that
> seems to be done now in the fib_trie, but I am not very familiar with
> it. Do you know of a way to dump/monitor the route nexthop information?

	Hm, no.

> I had attempted previous to my email to use "ip monitor
> route/neigh/link" and I don't see any netlink events around/during the
> incorrect packet emission timeframes.

	Yes, we still have caching but it is bound.

> Additionally strange/convenient seems to be the observed use of SRC_IP
> of interfaces currently being used by other IPVS clusters.

	OK, route is there, only saddr is from other service.
I see a bug that can explain it: ip_vs_dest_dst_alloc uses
kmalloc, not kzalloc, so saddr is not always 0 (
If random memory is provided we get -EINVAL and then we
get valid address. What if some memory freed by another
dest_dst (containing saddr used by other dest IP) is allocated?
Routing agrees because this saddr is valid, so we get
valid local address, only that it is not suitable for
our target IP. So, you are probably right about this...
we get saddr from other dests/services.

	You can fix it and test it by changing
ip_vs_dest_dst_alloc to use kzalloc. Let me know if
you prefer to apply patch from me or you can change it
directly for the test. If that works we can prepare it
as patch for mainline because I plan to provide
the old saddr as done before the wrong change.


Julian Anastasov <ja at ssi.bg>

More information about the lvs-users mailing list