Realservers as client

[Leon Botes]

Hi all.

I have been struggling with a setup for a good while now.
Requirement:
Clients need to connect to a variety of services from external ip addresses.
However real servers need to connect to a vip balanced between each 
other for mysql connections.

We are using LVS-DR for the external connections and alls well. As 
expected the problem is when the realservers need to be clients. To make 
things possibly worse those real server are freebsd so i can't apply any 
of the realserver patches for linux.

What i tried to work around the problem is to add two nics to each 
realserver and the load balancer. Link all ext nics on one switch and 
all int nics on another switch.
I thought of running the services required by external clients in LVS-DR 
and the mysql needed by the real servers in LVS-NAT.
Diagram to help

router
   |
   |  ext if's        int if's
    --------- director--------
   |                          |
   |                          |
    --------- real server 1---
   |                          |
   |                          |
    --------- real server n---

On the ext side i add the VIP1 which is created on the director ext_if 
to the localhost of each server as an alias. Setup LVS-DR and works fine.
Now I can't connect to a balanced service on VIP1 from a real server 
since it always looks at itself - right.
Si i created a second vip VIP2 on the director ext_if in another subnet 
separate from the ext and int nets. I then set the director to nat all 
addresses from the ext side, set routes on the realservers to reach the 
VIP2 and balanced the director to forward the requestes to the 
realservers int_if ip's.
That doesn't work and my knowledge of ip is not that good for an 
explanation. I assumed that since the director is natting all packets 
leaving its int_if that the real servers would reply to the directors 
int_if.

Can anyone suggest a solution or at least show me the flaws in this sort 
of setup. I desperately need to balance the mysql services with the real 
servers as clients.

Leon