[lvs-users] Question about director (LVS-DR) & realservers on different VLANs

Net Ground - Frederique Rijsdijk frederique at netground.nl
Wed Oct 17 15:08:58 BST 2012


Hi,


We have a LVS-DR setup (currently figure 1), where the keepalived hosts use tagged vlan interfaces to 'be' in the same broadcast lans as the realservers. This all works fine for the clients. The problem is that realserver1/2 can not reach VIP2, and realserver 3/4 can not reach VIP1. But even 'other' hosts, like "other server1" and "other server2", that are in the same VLANs as respectivly realserver1/2 and realserver3/4, can not reach neither VIPs. I can see the SYN packets reaching the VIP on the director, but keepalived seems to do nothing with them.  I'm not sure if there is a solution for this. Is there? Would figure 2 make any difference? (VIP on eth0, and VLAN interfaces on eth1). I've tested the whole setup without any firewall anywhere, but that doesn't make any difference.

Director: Ubuntu 12.04 64-bit + Keepalived 1:1.2.2-3ubuntu1
Realservers: FreeBSD (VIP/32 on lo0 with or without -arp)
See below for configuration of network and keepalived


Figure 1
                                           ________
                                          |        |
                                          | client |
                                          |________|
                                              |
                                              |
                                           ________
                                          |        |
                                          |internet|
                                          |________|
                                              |
                                __________    |
                               |          |   |  eth0     RIP=XX.YYY.AAA.5/27
                               | director |---   eth0.608 RIP=XX.YYY.BBB.18/26
                               |__________|   |  eth0.621 RIP=XX.YYY.CCC.71/27
                                              | 
                                              |
                                              |  VIP1=XX.YYY.AAA.7:80  (realserver1/2 - VLAN 621)
                                              |  VIP2=XX.YYY.AAA.9:443 (realserver3/4 - VLAN 608)
                                              |
                       ----------------------------------------------
                      |                                              |
                      |                                              |
              ----------------                           -----------------------
             |                |                         |                       |
             |                |                         |                       |
RIP=XX.YYY.CCC.68/27    RIP=XX.YYY.CCC.70/27     RIP=XX.YYY.BBB.52/26    RIP=XX.YYY.BBB.12/26
VIP=XX.YYY.AAA.7/32     VIP=XX.YYY.AAA.7/32      VIP=XX.YYY.AAA.9/32     VIP=XX.YYY.AAA.9/32
 _____________           _____________            _____________           _____________
|             |         |             |          |             |         |             |
|             |         |             |          |             |         |             |
|  VLAN 621   |         |  VLAN 621   |          |  VLAN 608   |         |  VLAN 608   |
| realserver1 |         | realserver2 |          | realserver3 |         | realserver4 |
|_____________|         |_____________|          |_____________|         |_____________|



            RIP=XX.YYY.CCC.69/27                             RIP=XX.YYY.BBB.9/26
             _____________                                    _____________
            |             |                                  |             |
            |             |                                  |             |
            |  VLAN 621   |                                  |  VLAN 608   |
            |other server1|                                  |other server2|
            |_____________|                                  |_____________|



Figure 2
                                           ________
                                          |        |
                                          | client |
                                          |________|
                                              |
                                              |
                                           ________
                                          |        |
                                          |internet|
                                          |________|
                                              |
                                __________    |
                               |          |---   eth0     RIP=XX.YYY.AAA.5/27
                               | director |      eth1.608 RIP=XX.YYY.BBB.18/26
                               |__________|---   eth1.621 RIP=XX.YYY.CCC.71/27
                                              | 
                                              |
 
                                            the rest



Network configuration on director:

auto eth0
iface eth0 inet static
        address XX.YYY.AAA.5
        netmask 255.255.255.224
        network XX.YYY.AAA.0
        broadcast XX.YYY.AAA.31
        gateway XX.YYY.AAA.1

auto eth0.608
iface eth0.608 inet static
        address XX.YYY.BBB.18
        netmask 255.255.255.192

auto eth0.621
iface eth0.621 inet static
        address XX.YYY.CCC.71
        netmask 255.255.255.224

Non-default sysctls on the director:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv4.ip_nonlocal_bind=1



keepalived.conf:

vrrp_sync_group LBGT0 {
        group {
                LBGT0
        }
}
global_defs {
        lvs_id lb01
}

vrrp_instance LB1 {
        state MASTER
        priority 101
        interface eth0
        virtual_router_id x
        advert_int 1
        authentication {
           auth_type PASS
           auth_pass xxxxxxx
}
virtual_ipaddress {
        XX.YYY.AAA.7
        XX.YYY.AAA.9
}

virtual_server XX.YYY.AAA.9 443 {
        delay_loop 6
        lb_algo wrr
        lb_kind DR
        protocol TCP

        # realserver3
        real_server XX.YYY.BBB.52 443 {
                weight 0

                HTTP_GET {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_timeout 60
                        connect_port 80
                }
        }

        # realserver4
        real_server XX.YYY.144.12 443 {
                weight 100

                HTTP_GET {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_timeout 60
                        connect_port 80
                }
        }
}

virtual_server XX.YYY.AAA.7 80 {
        delay_loop 6
        lb_algo wrr
        lb_kind DR
        protocol TCP

        # realserver1
        real_server XX.YYY.CCC.68 80 {
                weight 60
               
                HTTP_GET
                {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_port 80
                        connect_timeout 60
                }
        }

        # realserver2
        real_server XX.YYY.CCC.70 80 {
                weight 60

                HTTP_GET {
                        url {
                                path /ping.php
                                digest e0aa021e21dddbd6d8cecec71e9cf564
                        }
                        nb_get_retry 3
                        delay_before_retry 3
                        connect_timeout 60
                        connect_port 80
                }
        }
}


More information about the lvs-users mailing list