[lvs-users] Packets on a wrong way?

Gerd Pickel gerd.pickel at it-intuition.de
Sun Jul 3 19:00:12 BST 2011


Hello evereyone,

i have discovered a problem on our newly installed loadbalancer 
enviroment. Here is a short description of the setup:

On a machine with Debian 6.0 I set up the Xen-Hypervisor to run four 
virtual machines:

1. two virtual machines (dmzlb01&dmzlb02) build the loadbalancer 
(ldirectord Version: 1:1.0.3-3.1; heartbeat Version: 1:3.0.3-2

2. the other two virtual machines (dmzproxy01&dmzproxy02) are set up as 
real servers with apache2 (2.2.16-6+squeeze1) and exim4 (4.72-6+squeeze2)

I planed this setup should work for exim4:

public ip:25  --> firewall --> NAT to dmzvirtual00:25 --> firewall --> 
NAT to internal exchange server

internal exchange server --> firewall --> NAT to dmzvirtual00:25 --> 
firewall --> NAT to public ip:25
(IP's like above)

Exim is configured to change the IP from which it initiats the 
connection to other servers based on the domain part of the sender of an 
email.
Like:

domain part            IP from which exim make the connection

hullahu.de:               192.168.180.120
buhuu.de                    192.168.180.121
and so on


When I try to connect to an external host I get a SYN_SENT (and nothing 
else) in the output of netstat and the following output of a tcpdump on 
dmzlb01 and dmzproxy01.


Output of tcpdump when I make a connection from dmzproxy01 with source 
ip 192.168.180.120 to port 25 of mx.schlund.de (external):

root at dmzproxy01:~# telnet -b 192.168.180.120 mx.schlund.de 25

root at dmzproxy01:~# tcpdump host 192.168.180.120 -vvvv
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 
65535 bytes
16:08:30.780149 IP (tos 0x10, ttl 64, id 58852, offset 0, flags [DF], 
proto TCP (6), length 60)
     dmzvirtual00.41762 > mx-b.kundenserver.de.smtp: Flags [S], cksum 
0x24fc (correct), seq 522870993, win 5840, options [mss 1460,sackOK,TS 
val 171869493 ecr 0,nop,wscale 6], length 0
16:08:33.779349 IP (tos 0x10, ttl 64, id 58853, offset 0, flags [DF], 
proto TCP (6), length 60)
     dmzvirtual00.41762 > mx-b.kundenserver.de.smtp: Flags [S], cksum 
0x220e (correct), seq 522870993, win 5840, options [mss 1460,sackOK,TS 
val 171870243 ecr 0,nop,wscale 6], length 0
16:08:39.779355 IP (tos 0x10, ttl 64, id 58854, offset 0, flags [DF], 
proto TCP (6), length 60)
     dmzvirtual00.41762 > mx-b.kundenserver.de.smtp: Flags [S], cksum 
0x1c32 (correct), seq 522870993, win 5840, options [mss 1460,sackOK,TS 
val 171871743 ecr 0,nop,wscale 6], length 0
16:08:51.779439 IP (tos 0x10, ttl 64, id 63843, offset 0, flags [DF], 
proto TCP (6), length 60)
     dmzvirtual00.56587 > mx-b.kundenserver.de.smtp: Flags [S], cksum 
0xd67d (correct), seq 522870996, win 5840, options [mss 1460,sackOK,TS 
val 171874743 ecr 0,nop,wscale 6], length 0


root at dmzlb01:~# tcpdump host 192.168.180.120 -vvvv
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 
65535 bytes
16:08:30.802139 IP (tos 0x0, ttl 54, id 0, offset 0, flags [DF], proto 
TCP (6), length 44)
     mx-b.kundenserver.de.smtp > dmzvirtual00.41762: Flags [S.], cksum 
0x2979 (correct), seq 1859939374, ack 522870994, win 5840, options [mss 
1460], length 0
16:08:30.802172 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
TCP (6), length 40)
     dmzvirtual00.41762 > mx-b.kundenserver.de.smtp: Flags [R], cksum 
0x331e (correct), seq 522870994, win 0, length 0
16:08:33.799782 IP (tos 0x0, ttl 54, id 0, offset 0, flags [DF], proto 
TCP (6), length 44)
     mx-b.kundenserver.de.smtp > dmzvirtual00.41762: Flags [S.], cksum 
0x2979 (correct), seq 1859939374, ack 522870994, win 5840, options [mss 
1460], length 0
16:08:33.799797 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
TCP (6), length 40)
     dmzvirtual00.41762 > mx-b.kundenserver.de.smtp: Flags [R], cksum 
0x331e (correct), seq 522870994, win 0, length 0
16:08:39.799548 IP (tos 0x0, ttl 54, id 0, offset 0, flags [DF], proto 
TCP (6), length 44)
     mx-b.kundenserver.de.smtp > dmzvirtual00.41762: Flags [S.], cksum 
0xf1a6 (correct), seq 1875616529, ack 522870994, win 5840, options [mss 
1460], length 0
16:08:39.799567 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
TCP (6), length 40)
     dmzvirtual00.41762 > mx-b.kundenserver.de.smtp: Flags [R], cksum 
0x331e (correct), seq 522870994, win 0, length 0
16:08:51.799573 IP (tos 0x0, ttl 55, id 0, offset 0, flags [DF], proto 
TCP (6), length 44)
     mx-b.kundenserver.de.smtp > dmzvirtual00.56587: Flags [S.], cksum 
0xecd3 (correct), seq 659928670, ack 522870997, win 5840, options [mss 
1460], length 0
16:08:51.799604 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
TCP (6), length 40)
     dmzvirtual00.56587 > mx-b.kundenserver.de.smtp: Flags [R], cksum 
0xf921 (correct), seq 522870997, win 0, length 0
16:08:54.798667 IP (tos 0x0, ttl 55, id 0, offset 0, flags [DF], proto 
TCP (6), length 44)
     mx-b.kundenserver.de.smtp > dmzvirtual00.56587: Flags [S.], cksum 
0xecd3 (correct), seq 659928670, ack 522870997, win 5840, options [mss 
1460], length 0
16:08:54.798684 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto 
TCP (6), length 40)
     dmzvirtual00.56587 > mx-b.kundenserver.de.smtp: Flags [R], cksum 
0xf921 (correct), seq 522870997, win 0, length 0

What I can see is:
1. the connection attempt on dmzproxy01 to mx.schlund.de
2. the anser of mx.schlund.de on dmzlb01 (?)

But I don't see a connection of mx.schlund.de to the dmzproxy01 or 
dmzvitual00.At last the email can not be send to the external host.
It seems that the answers of the external server never reaches 
dmzproxy01. But why?
Another question is: what would be the correct way of the packets to travel?
I think it should be this way:

mx.schlund.de --> dmzvirtual00

Curiously emails can be received from external servers without a problem.

Here is an excerpt from the /etc/ha-d/ldirectord on dmzlb01

virtual=dmzvirtual00:25
         real=dmzproxy01:25      gate    1000
         real=dmzproxy02:25      gate    3
         service=smtp
         scheduler=wrr
         protocol=tcp
         emailalert=root
         emailalertfreq=3600
         receive="220"


Pointing me in the right direction for a solution would be great.

Thanks!

Gerd






More information about the lvs-users mailing list