LVS/NAT and SYN/ACK issue

Wed Feb 21 13:29:22 GMT 2007

On Wed, 21 Feb 2007, dmitri at nominet.org.uk wrote:

> Hi,
>
> I should probably say in the beginning that the issue I'm 
> going to describe is not directly related to the problem 
> discussed on this list a while ago (http syn/ack not 
> translated when ftp loadbalancing also enabled). We have 
> several LVS/NAT installations which are managed by 
> Keepalived. All of them are pretty much identical and 
> exhibit the same issue. The setup is looking like this (a 
> backup load balancer and a backup router are omitted) and 
> is LVS/NAT standard:
>
>
>        !-----------------!
>        !                 !
>        !     Internet    !
>        !                 !
>        !-----------------!
>                 !
>                 !
>        !-----------------!
>        !                 !
>        !     Router      !
>        !                 !
>        !-----------------!
>                 !
>                 !
>        !-----------------!
>        !      eth0       !
>        !                 !
>        !  LoadBalancer   !
>        !                 !
>        !      eth1       !
>        !-----------------!
>                 !
>                 !192.168.1.0/24
>    ------------------------
>    !       !       !      !
>  !---!                  !---!
>  !RS1!     .........    !RSN!
>  !---!                  !---!
>
> This setup is working fine most of the time except when a 
> client sends a TCP SYN packet and then forgets about this 
> connection. In this case a RealServer starts to send 
> SYN/ACK packets until this connection on the server times 
> out and it sends RST/ACK. The issue is that two last 
> packets don't get translated because ipvs on the 
> LoadBalancer already timed out this connection.

Julian,
 	Do you remember anything about this?

 	I remember something like this in the last 6months 
or so and I don't remember how it was resolved. I don't 
remember writing it up for the HOWTO at least

Below is a 
> tcpdump on LoadBalancer/eth0:
>
> 10:58:20.655059 IP 213.248.204.8.2113 > 213.248.224.116.43: S
> 1402601529:1402601529(0) win 512
> 10:58:20.655335 IP 213.248.224.116.43 > 213.248.204.8.2113: S
> 443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
> 10:58:24.031708 IP 213.248.224.116.43 > 213.248.204.8.2113: S
> 443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
> 10:58:30.792336 IP 213.248.224.116.43 > 213.248.204.8.2113: S
> 443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
> 10:58:44.303557 IP 213.248.224.116.43 > 213.248.204.8.2113: S
> 443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
> 10:59:11.316010 IP 213.248.224.116.43 > 213.248.204.8.2113: S
> 443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
> 11:00:05.330972 IP 213.248.224.116.43 > 213.248.204.8.2113: S
> 443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
> 11:01:05.346329 IP 192.168.1.32.43 > 213.248.204.8.2113: S
> 443218720:443218720(0) ack 1402601530 win 49312 <mss 1460>
> 11:02:05.362233 IP 192.168.1.32.43 > 213.248.204.8.2113: R 1:1(0) ack 1
> win 49312
>
> In this example I simulated the situation with sending SYN 
> packet from my PC to the server and dropping all further 
> packets. While the SYN/ACK packets were still being 
> translated, ipvsadm -lnc was showing this:
>
> TCP 28:12  NONE        213.248.204.8:0    213.248.224.116:43
> 192.168.1.32:43
> TCP 00:57  SYN_RECV    213.248.204.8:2113 213.248.224.116:43
> 192.168.1.32:43
>
> But once I see only this:
>
> TCP 27:02  NONE        213.248.204.8:0    213.248.224.116:43
> 192.168.1.32:43
>
> packets from RealServer belonging to this connection (from 
> RealServer point of view) stop getting translated.
>
> This is not a real problem but rather a nuisance for me. I 
> just don't want packets with private IP's leaving 
> LoadBalancer. I can't block this packets with iptables 
> since I believe ipvs does SNATing somewhere in POSTROUTING 
> chain and there is no way to put any other rules beyond 
> this chain. I also can't modify SYN_RECV timeout since 
> there is no tcp_timeout_syn_recv entry in 
> /proc/sys/net/ipv4/vs/ (this is a stock CentOS 4.3 kernel)

I assume you've looked at the idle timeouts (not sure 
they're going to help here).

http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.services.general.html#tcpip_idle_timeout

Joe

> My question is: Is it possible to block not translated 
> packets from leaving the LoadBalancer without touching 
> RealServers and the Router?
>
> If it can help, here is additional info:
>
> # uname -a
> Linux lb1 2.6.9-34.ELsmp #1 SMP Thu Mar 9 06:23:23 GMT 2006 x86_64 x86_64
> x86_64 GNU/Linux
>
> # ipvsadm --help
> ipvsadm v1.24 2003/06/07 (compiled with getopt_long and IPVS v1.2.0)
>
>
> Thank you
> Dmitri Skachkov
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users at LinuxVirtualServer.org
> Send requests to lvs-users-request at LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>

-- 
Joseph Mack NA3T EME(B,D), FM05lw North Carolina
jmack (at) wm7d (dot) net - azimuthal equidistant map
generator at http://www.wm7d.net/azproj.shtml
Homepage http://www.austintek.com/ It's GNU/Linux!