[lvs-users] LVS-NAT problems for SMTP cluster

[infernix]

Joseph Mack NA3T wrote:
> On Sat, 3 May 2008, Jason Ledford wrote:
> 
>> I am doing a smtp cluster and after I got the lvs part figured out, smtp works just fine.  In your telnet session, how do you get as far as the mail from and rcpt to if you aren't getting a response?  If you telnet to the server does it respond with a banner?
> I think there's the beginnings of the exchange in the HOWTO

I dug through it again, but couldn't find a more detailed description of 
the packet flow than I already laid out.

>> I've given LVS-DR a spin and that worked fine, but the issue with that
>> is that outgoing SMTP messages (e.g. sendmail queue) would be sent from
>> the various real server public source IPs.
> 
> what public IPs? For security, the only public IP on the 
> realserver should be the VIP

The realservers had an unique public IP + the VIP on lo:1. But i do 
suppose we could change the LVS-DR setup with iproute2 rules so that I 
can get the outgoing SMTP traffic NATed over the VIP on the director, I 
saw some patches about that. Anyhow, that's irrelevant for the LVS-NAT 
issue I'm having.

>> When I've done this, I am getting traffic all the way: client >
>> (internet) > gateway 80.1.1.1 >  VIP on director > RIP on real server >
>> back to director (ipvs?) > gateway 80.1.1.1 > (internet) > client.
> 
> so the LVS is working

No, not really. Look at the twake or tcpdump logs; the realserver keeps 
retransmitting the TCP packets but the client doesn't seem to tie them 
to the active TCP session (regardless of what client is used, whether a 
local address in the public range or any random public IP on the 
internet). So it seems that something is going wrong with the TCP 
session. Perhaps redundant, but I replied this in another post:

mx-node01:~# nc -p 25 -l
client:~# nc 80.1.1.10 25

When I type text in the client nc session, this text shows up on the
realserver netcat session.

When I type text in the *realserver* netcat session, it *doesn't* show
up at the client netcat session.

It is as if the data in the tcp session is one-way, e.g. data only goes
from client -> server. My TCP knowledge isn't good enough to figure out 
why this happens.


> I don't really know what's going on. Since LVS is working, 
> there must be something at the application layer. Is 
> localhost a valid address in this context?

All nodes have just 127.0.0.1 configured on lo, if that's what you mean.