LVS-NAT with multiple RIP to VIP associations

[David M]

Joe:

Thank you for your reply.

The problem is with outbound sendmail connections.  The MTA initiates a
connection and sends out an email without there being a response from an
incoming connection.  So, outgoing connections from RIPs just get routed out
the default gateway for LVS-NAT.  So, in order for certain RIPs (real nat
IPs) to be rounted out the correct VIP (or public IP), we had to do some
packet rewriting with iptables.  For email, we have to send out the email
over a particular VIP (or public IP).  Therefore, we have to create RIP
(real nat IP) to VIP associations.

For HTTP connections, iptables is not needed, since the server is always
responding to an incoming connection.

Since we cannot be sure that the firewall is going to behave like a normal
firewall with the *ipvs_nfct patch*, we have decided to do a "FW --> LVS-NAT
--> Real servers" configuration.

David Mitchell


On 12/19/06, Joseph Mack NA3T <jmack at wm7d.net> wrote:
>
> On Fri, 15 Dec 2006, David M wrote:
>
> > I guess that I did not explain my setup well enough.
>
> I've been busy so haven't got back to you. Seems like you're
> figuring it out with other people though.
>
> > In this example, we have three real sendmail servers.  We have about 30
> > private IPs on each real sendmail server (172.16.0.0/24) with 30
> instances
> > of sendmail on each server.  So, with three real servers, that's 90
> private
> > IPs total.  On the Director, we have 30 public IPs.  And, the Director
> load
> > balances the SMTP connections to the three real servers.  Below is a
> visual
> > representation:
> > On the Director:
> > $VIP_M_01 --> $RIP_M1_01
>
> I assume you're running LVS-NAT.
>
> I assume $RIP_M1_01 is the RIP on realserver M1 that
> handles VIP1
>
> > $VIP_M_01 --> $RIP_M2_01
> > $VIP_M_01 --> $RIP_M3_01
> > $VIP_M_02 --> $RIP_M1_02
> > $VIP_M_02 --> $RIP_M2_02
> > $VIP_M_02 --> $RIP_M3_02
>
> OK
>
> > Each of our public IPs (VIPs) have a domain name associated with them (
> e.g.,
> > mail.domain_name.com).  Also, the sendmail configurations on our private
> IPs
> > have a domain name associated with them.  So, each domain name has one
> > public IP (on the Director, which is a VIP) and three private IPs (one
> for
> > each real server, which are RIPs).
>
> OK
>
>
> > The problem is that the real servers have have to send out their emails
> over
> > the correct public IP, otherwise the email will not be RFC compliant
> (RFC
> > 2822).
>
> OK
>
> Let's assume we only have one director and look at
> $RIP_M1_01. This RIP should only appear in the ipvsadm
> entries for VIP_M_01. Packets returning from RIP_M1_01
> should be intercepted by rules for VIP_M_01 and ignored by
> the rules for all other VIPs. By induction, all RIPs are
> handled and there is no need for any iptables entries.
>
> But what you're saying is that it doesn't work that way?
> What happens if there are absolutely no iptables rules
> or network fiddling anywhere? I assume from what you say
> that it doesn't work.
>
> > Our setup is currently working.
>
> I'm impressed.
>
> > I was just wondering if there is a better way to do this.
>
> I would have thought that you wouldn't need to do this. At
> least you used not to have to do this.
>
> > I have read that the ipvs_nfct patch has some short comings.  Julian
> said
> > that the IPVS packets do not use the same path through the network stack
> as
> > other non-IPVS packets and that all out->in traffic passes INPUT (not
> > FORWARD as in netfilter).  So, we are not certain whether or not we can
> > create a locked-down, bastion host with iptables on the Director, using
> > ipvs_nfct patch.
>
> it's not perfect, but you should be able to do this.
>
> > Are most running an "FW --> LVS-NAT --> Real Server" configuration in
> their
> > production environments?  Or is it more popular to attempt to have
> > stateful-packet filtering on the LVS Director?
>
> We'd hope that you could do the whole thing in one box but
> as you've probably read in the LVS-NAT writeup LVS-NAT for
> 2.6 still has problems. Still the firewall part should work.
>
> Joe
> --
> Joseph Mack NA3T EME(B,D), FM05lw North Carolina
> jmack (at) wm7d (dot) net - azimuthal equidistant map
> generator at http://www.wm7d.net/azproj.shtml
> Homepage http://www.austintek.com/ It's GNU/Linux!
> _______________________________________________
> LinuxVirtualServer.org mailing list - lvs-users at LinuxVirtualServer.org
> Send requests to lvs-users-request at LinuxVirtualServer.org
> or go to http://www.in-addr.de/mailman/listinfo/lvs-users
>