[lvs-users] TCP timeout and established connections in DR mode

Abhijeet Rastogi abhijeet.1989 at gmail.com
Tue May 5 22:35:45 BST 2020

Hi Julian,

First of all, thanks a lot, really appreciate the response.

>Note that these timeouts are for inactivity, not a time from
connection creation.

This was the golden information that was missing for me. Thanks for helping
out with all the other answers, everything makes sense.

>IPVS also has sysctl vars that can release IPVS structures on memory

Are you referring to drop_entry? Doc says that it is only for SYN-RECV/SYNACK
state. What about the TCP connections that have completed the "fin
handshake"?  The reason I ask is, a default timeout like 15min seems a
little too high for HTTP and I suspect that there'll be a lot of stale
entries in the connection table.


On Tue, May 5, 2020 at 11:17 AM Julian Anastasov <ja at ssi.bg> wrote:

>         Hello,
> On Fri, 1 May 2020, Abhijeet Rastogi wrote:
> > Hi everyone,
> >
> > Considering that IPVS is in DR mode with persistence disabled completely
> > and the client and real servers are configured to handle long-lived HTTP
> > connections (>15min). I understand that the default TCP timeout is 15min
> > but t I'm confused about the impact of this timeout on already active
> > established connections even when the timer value hits.
>         Note that these timeouts are for inactivity, not a time from
> connection creation.
> > For eg, with default value 15min, will the existing connection be simply
> > dropped or do we keep the connection table for that 5-tuple intact?
>         The IPVS structure (IPVS connection) that holds the tuple is
> freed 15mins after the last packet is transferred, no packets are sent to
> abort the TCP connection after this period. But following packets may get
> RST due to the missing IPVS connection. ipvsadm has option to tune this
> period.
> >    - If the connection is simply dropped, are there any signals to look
> for
> >    in terms of finding out how widespread it is?
>         No, we do not have stat/counter for expiration in established
> state.
> >    - If we keep the connection table entry, what's the new policy on this
> >    existing connection? (Note: persistence is disabled, as I'm aware that
> >    there's a 60s timer which reactivates the connection template)
> >       - If this is true, should we keep TCP timeouts on production
> servers
> >       lesser than 15min to ensure we're protected in terms of some
> > sort of abuse?
>         DR can be abused, while for NAT we can change the state based on
> packets from real server. But it depends on the balanced application.
> HTTP usually is not idle for 15mins, ssh apps can use TCP-keepalive to
> protect from firewalls dropping the NAT connections. So, it is up to
> you to decide what idle period you can tolerate for your application.
> IPVS also has sysctl vars that can release IPVS structures on memory
> pressure.
>         Another option is to use source-based hashing (eg. MH, SH
> schedulers) and sloppy mode: echo 1 > sloppy_tcp
> It allows creating IPVS connections from non-SYN packets. So, even if
> IPVS connection is expired, it can be re-created to the same real
> server.
> Regards
> --
> Julian Anastasov <ja at ssi.bg>

Abhijeet (https://abhi.host)

More information about the lvs-users mailing list