keepalived handling 2 VIPs [solved -> maybe parse error?]
Dominik Klein
dk at in-telegence.net
Tue Apr 25 09:32:52 BST 2006
> so you have two directors, each with a VIP (say VIP1, VIP2) and
> forwarding its virtual service(s). On failover, what happens? One
> machine winds up with both VIPs? and then when the machine comes up
> again, the VIP migrates back so that each machine has a single VIP again?
>
> If so can you post the working config file and an explanation for how it
> works and I'll put it in the HOWTO
Hi Joe,
I'll explain what I do and post the config file below. This might become
a somewhat long mail :)
Read carefully as there are a lot of numbers, this is a fairly complex
and special setup and I may not have typed everything correctly. I'll
re-read it to correct errors, but you never know.
########
# Goal #
########
My goal is a HA MySQL database. As the MySQL cluster storage engine
lacks several important features (like foreign keys e.g.), I cannot use
a MySQL cluster. So now I use MySQL replication in a
master-to-master-setup. As my clients are able to re-connect after a
connection loss, but cannot connect to a different IP on connection
loss, a VIP setup is the goal. So my clients only know the VIP(s), not
the real IPs of the MySQL Servers.
#########
# Setup #
#########
I have two machines. Each machine runs keepalived and MySQL. Each
machine has 2 NICs. eth0 going to the switch, eth1 connecting SRV1 and SRV2.
My setup looks like this:
Intranet
|
|
##SWITCH##
| |
| |
| |
SRV1---SRV2
Clients connect through the switch, replication is done over the direct
gigabit connection between SRV1 and SRV2.
SRV1 IPs:
eth0 10.6.10.20
eth1 10.250.250.20
SRV2 IPs:
eth0 10.6.10.21
eth1 10.250.250.21
####################
# Virtual Services #
####################
I need two VIPs, as I want write-queries to go to SRV1, and read-queries
to go to SRV2 - just as in a normal replication-setup, for
loadbalancing-purposes. Note that it is not keepalived or LVS that does
the loadbalancing here, as each virtual service only has one realserver
and one sorry-server!
"Loadbalancing" is just writing-to-the-database-software connecting to
one server, reading-from-the-database-software connecting to another server.
10.6.10.24:3306
SRV1 (MASTER state for this VIP)
Realserver: 127.0.0.1:3306
Sorryserver: 10.250.250.21:3306
SRV2 (BACKUP state for this VIP)
Realserver 10.250.250.20:3306
Sorryserver: 127.0.0.1:3306
10.6.10.240:3306
SRV1 (BACKUP state for this VIP)
Realserver 10.250.250.21:3306
Sorryserver: 127.0.0.1:3306
SRV2: (MASTER state for this VIP)
Realserver: 127.0.0.1:3306
Sorryserver: 10.250.250.20:3306
So this is basically the "localhost"-feature, plus one sorryserver per
virtual service.
############
# Failover #
############
If one of the eth0 network connections fail, the VIP moves to the other
director, but connections still get directed to the same MySQL server.
So the MySQL-loadbalancing still works.
If MySQL fails on one machine, connections are redirected to the other
server's eth1-IP (10.250.250.2[01]). In order to be able to route that
back over the director it came from, there are ip-rules on each server:
------------------------------
- SVR1 ip rules and routing: -
------------------------------
cat /etc/iproute2/rt_tables
2 mysqlrouting
...
ip rule show
...
32765: from 10.250.250.20 lookup mysqlrouting
...
ip route show table mysqlrouting
default via 10.250.250.21 dev eth1
Setup-steps for this:
echo "2 mysqlrouting" > /tmp/rt_tables
cat /etc/iproute2/rt_tables >> /tmp/rt_tables
ip rule add from 10.250.250.20 table mysqlrouting
ip route add default via 10.250.250.21 dev eth1 table mysqlrouting
------------------------------
- SVR2 ip rules and routing: -
------------------------------
cat /etc/iproute2/rt_tables
2 mysqlrouting
...
ip rule show
...
32765: from 10.250.250.20 lookup mysqlrouting
...
ip route show table mysqlrouting
default via 10.250.250.20 dev eth1
Setup-steps for this:
echo "2 mysqlrouting" > /tmp/rt_tables
cat /etc/iproute2/rt_tables >> /tmp/rt_tables
ip rule add from 10.250.250.21 table mysqlrouting
ip route add default via 10.250.250.20 dev eth1 table mysqlrouting
#######################
# Configuration files #
#######################
------------------------------------
- keepalived configuration on SRV1 -
------------------------------------
! Configuration File for keepalived
global_defs {
notification_email { foo at mydomain.com }
notification_email_from keepalived at mydomain.com
smtp_server 10.2.20.6
smtp_connect_timeout 30
lvs_id TEST-MYSQL-1
}
vrrp_sync_group test_mysql_one {
group {
vip_mysql_one
}
}
vrrp_sync_group test_mysql_two {
group {
vip_mysql_two
}
}
vrrp_instance vip_mysql_one {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 12345
}
virtual_ipaddress {
10.6.10.24/24 brd 10.6.10.255 dev eth0
}
}
vrrp_instance vip_mysql_two {
state BACKUP
interface eth0
virtual_router_id 52
priority 10
advert_int 1
authentication {
auth_type PASS
auth_pass 12345
}
virtual_ipaddress {
10.6.10.240/24 brd 10.6.10.255 dev eth0
}
}
virtual_server 10.6.10.24 3306 {
delay_loop 6
# lb_algo is actually not important, as we have only one real_server
lb_algo wlc
lb_kind NAT
nat_mask 255.255.255.0
protocol TCP
real_server 127.0.0.1 3306 {
TCP_CHECK {
connect_port 3306
connect_timeout 30
} #TCP_CHECK
}
sorry_server 10.250.250.21 3306
}
virtual_server 10.6.10.240 3306 {
delay_loop 6
# lb_algo is actually not important, as we have only one real_server
lb_algo wlc
lb_kind NAT
nat_mask 255.255.255.0
protocol TCP
real_server 10.250.250.21 3306 {
TCP_CHECK {
connect_port 3306
connect_timeout 30
} #TCP_CHECK
}
sorry_server 127.0.0.1 3306
}
------------------------------------
- keepalived configuration on SRV2 -
------------------------------------
! Configuration File for keepalived
global_defs {
notification_email { foo at mydomain.com }
notification_email_from keepalived at mydomain.com
smtp_server 10.2.20.6
smtp_connect_timeout 30
lvs_id TEST-MYSQL-2
}
vrrp_sync_group ACDDB_mysql_one {
group {
vip_mysql_one
}
}
vrrp_sync_group ACDDB_mysql_two {
group {
vip_mysql_two
}
}
vrrp_instance vip_mysql_one {
state BACKUP
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 12345
}
virtual_ipaddress {
10.6.10.24/24 brd 10.6.10.255 dev eth0
}
}
vrrp_instance vip_mysql_one {
state MASTER
interface eth0
virtual_router_id 52
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 12345
}
virtual_ipaddress {
10.6.10.240/24 brd 10.6.10.255 dev eth0
}
}
virtual_server 10.6.10.24 3306 {
delay_loop 6
# lb_algo is actually not important, as we have only one real_server
lb_algo wlc
lb_kind NAT
nat_mask 255.255.255.0
protocol TCP
real_server 10.250.250.20 3306 {
TCP_CHECK {
connect_port 3306
connect_timeout 30
} #TCP_CHECK
}
sorry_server 127.0.0.1 3306
}
virtual_server 10.6.10.240 3306 {
delay_loop 6
# lb_algo is actually not important, as we have only one real_server
lb_algo wlc
lb_kind NAT
nat_mask 255.255.255.0
protocol TCP
real_server 127.0.0.1 3306 {
TCP_CHECK {
connect_port 3306
connect_timeout 30
} #TCP_CHECK
}
sorry_server 10.250.250.20 3306
}
As MySQL requires some specific configuration, I will briefly post the
relevant parts, but not go into detail here, because it is actually OT
for this list. Read the MySQL-Documentation for further detail, if you
do not understand the configuration parts below:
http://dev.mysql.com/doc/refman/5.0/en/replication.html
-------------------------------
- MySQL configuration on SRV1 -
-------------------------------
log-bin=mysql-bin
log-slave-updates
server-id = 5000
auto_increment_increment=2
auto_increment_offset=1
master-host = 10.250.250.21
master-user = replication
master-password = replication
master-port = 3306
-------------------------------
- MySQL configuration on SRV1 -
-------------------------------
log-bin=mysql-bin
log-slave-updates
server-id = 5001
auto_increment_increment=2
auto_increment_offset=2
master-host = 10.250.250.20
master-user = replication
master-password = replication
master-port = 3306
########
# Note #
########
On failover, there is no connection-sync, so every client has to
re-connect. Connection-sync is imho not possible in this setup, as
real-servers are different on SRV1 and SRV2.
Comment if you like, feel free to ask questions if something is not clear.
Feel free to add it to the HOWTO, if you think this might be of help for
someone. Please use "klein.dominik at web.de" as author-mail-address for
that, as you never know how long one works for one company :)
Best regards,
Dominik
More information about the lvs-users
mailing list