From derrickltl at gmail.com Wed Jul 8 12:29:11 2020 From: derrickltl at gmail.com (Derrick Lim) Date: Wed, 08 Jul 2020 11:29:11 -0000 Subject: [lvs-users] packet occasionally gets forwarded to the wrong server Message-ID: Hi all, I've a number of IPVS directors using the source hashing scheduler (sh, sh-fallback) and TUN method to load balance to a number of real servers with the same weight. All the IPVS directors are configured with the same settings and BGP is used to advertise the VIP address on to the network. All IPVS directors run independently and do not sync connections. It is my understanding that sh is a consistent hashing algorithm that, regardless of which IPVS director traffic from the same connection (5-tuple) ends up on, the packet will be forwarded to the same real server. Is this understanding correct? Should I be moving to maglev instead? I'm on ubuntu 18.04, which is why I am using sh instead of maglev. However, very occasionally, a small number of packets from the same connection gets forwarded to a different real server when they end up on a different IPVS director. This causes the real server to send a RST since it has no idea about the connection. Why does this happen? ipvsadm -L -n looks like this: IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP a.b.c.d:443 sh (sh-fallback) -> x.y.z.1:443 Tunnel 10 0 0 -> x.y.z.2:443 Tunnel 10 0 0 -> x.y.z.3:443 Tunnel 10 0 0 -> x.y.z.4:443 Tunnel 10 0 0 I believe this issue is similar to what Philip Moore asked about in 2019-11, which led me to discover that I needed to have the real servers in the same sequence on each director. However, that didn't solve my issue, which is why I am asking for help here. Thanks in advance for any help. Regeards, Derrick From tangyang at zhihu.com Thu Jul 9 04:57:09 2020 From: tangyang at zhihu.com (=?UTF-8?B?5ZSQ6ZizLeaKgOacr+S4reWPsA==?=) Date: Thu, 09 Jul 2020 03:57:09 -0000 Subject: [lvs-users] estimation_timer take too much cpu usage. Message-ID: On my kubernetes node , there are some tcp packets randomly delayed between two containers on one node. I notice that estimation_timer softirq execute about 30ms+.(traced by this tool: https://github.com/bytedance/trace-noschedule) there are about 3000+ local address shown by 'ipvsadm -l'. Hyper threading is enabled and there are 72 logical cores on my machine. Does anyone know how to reduce the execution time of estimation_timer softirq? Thank you.