[lvs-users] LVS delays

Anders Kvist anders at kvistmail.dk
Tue Nov 15 13:10:48 GMT 2011


Hey

I have a LVS loadbalancer (keepalived) that causes some problems...

First, the network - same physical network:

firewall bond0 10.0.101.1 (forwards port 80 to 10.0.101.100)
loadbalancer bond0 10.0.101.100 (corosync/pacemaker cluster real IPs are
10.0.101.101 and 10.0.101.102)
loadbalancer bond0 10.0.102.1 (webserver network)

webservers bond0 10.0.102.121-128 (only 121 is active during the
following tests).

To avoid redirects, the loadbalancer has send_redirects and
accept_redirects disabled.

Connections through the loadbalancer to a webserver are connected fine,
but as soon as the http request is sent it causes problem when the
response is large (in this case 10K is enough to make it fail).



Examples with 1K and 10K files through loadbalancer:

$ ab -c 10 -n 100 http://10.0.101.100/1K
Document Path:          /1K
Document Length:        1024 bytes
Time taken for tests:   0.069 seconds
Requests per second:    1456.39 [#/sec] (mean)
Time per request:       6.866 [ms] (mean)
Time per request:       0.687 [ms] (mean, across all concurrent requests)
Transfer rate:          1825.56 [Kbytes/sec] received

$ ab -c 10 -n 100 http://10.0.101.100/10K
Document Path:          /10K
Document Length:        10240 bytes
Time taken for tests:   2.373 seconds
Requests per second:    42.14 [#/sec] (mean)
Time per request:       237.317 [ms] (mean)
Time per request:       23.732 [ms] (mean, across all concurrent requests)
Transfer rate:          432.03 [Kbytes/sec] received


Examples with 1K and 10K files directly to webserver:

$ ab -c 10 -n 100 http://10.0.102.121/1K
Document Path:          /1K
Document Length:        1024 bytes
Time taken for tests:   0.069 seconds
Requests per second:    1459.04 [#/sec] (mean)
Time per request:       6.854 [ms] (mean)
Time per request:       0.685 [ms] (mean, across all concurrent requests)
Transfer rate:          1825.23 [Kbytes/sec] received

$ ab -c 10 -n 100 http://10.0.102.121/10K
Document Path:          /10K
Document Length:        10240 bytes
Time taken for tests:   0.101 seconds
Requests per second:    988.28 [#/sec] (mean)
Time per request:       10.119 [ms] (mean)
Time per request:       1.012 [ms] (mean, across all concurrent requests)
Transfer rate:          10200.69 [Kbytes/sec] received



This shows that if the filesize is small enough it works fine, but when
the size grows, something goes wrong...

I have been looking at tcpdumps, but haven't figured out what excatly
happens - but it looks like a ACK from the client doesn't get through to
the webserver and it therefore waits and retransmits.

/Anders




More information about the lvs-users mailing list