[lvs-users] DTLS Load balancing

Julian Anastasov ja at ssi.bg
Sat Oct 21 15:13:50 BST 2017


On Thu, 19 Oct 2017, Simon Bernard wrote:

> Hi,
>    There is a draft[1] at the IETF about connection ID for DTLS . This 
> is a way to identify a "DTLS connection" by an ID instead of the 
> classical Ip address/port tuple. The objective is to reduce the need of 
> DTLS full handshake when client address/port change.
>     I would like to know if it make sense to make load balancing based 
> on this connection ID.
>     Here is the use case:
>     You have a cluster of servers behind a unique IP address.
>     You do load balancing using IP address.
>     You use UDP/DTLS.
>     Some clients are behind NAT and so theirs IP/port can change.
>     DTLS connection states are store in each server and so are not shared.
>     So if clients use same address/port, there is no issue as traffic 
> will be redirect always on the same server. Server has already a 
> connection for this peer, no need to full-handshake.
>     If address/port change, 2 possibilities:
>       - by chance load balancer, send traffic to the same server and 
> thanks to CID the server can reuse its connection, no-need to full-handshake
>       - bad luck, traffic is redirect on server which does not know this 
> peer, so a new full-handshake is needed.
>     It seems to me that doing load balancing on this connection ID could 
> solve the problem. [2]
>     Does it make sense to you ? Is it a way to extends LVS to support 
> this kind of behavior ? I don't think so as LVS seems to be a 4-layers 
> load balancer, but I'm still interesting to know your opinions as your 
> seems to know better than me in load-balancing.

	There is a similar example in IPVS: the SIP Persistence
Engine (PE in short), see ip_vs_pe_sip.c. The SIP PE details:

- every SIP message contains Call-ID that can be used
to schedule different packets in same UDP transport connection
to different real servers. Or many UDP connections with same
Call-ID to same real server.

- As IPVS first lookups for existing connection (by address/port)
and then if not found lookups for persistence template (which 
forwards separate connections to specific real server by some
criteria defined by the PE) with the goal to create new
connection, the SIP PE does not keep the normal UDP connections
when OPS mode is selected (One-packet scheduling, the --ops option).
By this way, for every packet, the "ct_match" SIP PE method can
inspect the UDP payload and to maintain a context (persistence template 
with pe_data) where the Call-ID is saved together with the real server
to be used for all packets with such Call-ID.

- if OPS mode is not set, same connection can not be used for
different Call-IDs.

	If you want to go ahead with implementing ip_vs_pe_dtls.c
I can give you some hints to consider:

- if some DTLS protocol version does not contain the CID in
every packet, DTLS PE will have to fallback to address/port
identification. This should be decided for the first packet in
UDP connection because the dialog should be with the same
real server. I'm refering to this text in your [1] draft:

"In DTLS 1.2, connection ids are exchanged at the beginning of the
"DTLS session only."

	So, we have 2 choices when first packet in UDP
connection is received:

1. Create and keep normal connection (without IP_VS_CONN_F_ONE_PACKET)
when CID can not be used or CID is present only early.
This means we can not inspect further this UDP connection for
CID until it is expired. The first lookup by addr/port will
find the existing connection for every following packet. Why we
need to keep the connection? Because we can use the initially
provided CID to get the needed real server but when next
packets without CID come, we should know where to forward them.
Or may be this can be handled by the "ct_match" PE method
and we can create only IP_VS_CONN_F_ONE_PACKET conns.

2. Create connection for the packet (IP_VS_CONN_F_ONE_PACKET)
but not based on svc->flags & IP_VS_SVC_F_ONEPACKET, this
time we should provide it from the "ct_match" PE method.
Connection is dropped after the packet is forwarded and
the next packets from same client port will come again to
the "ct_match" method. We will use this when CID will be
present in all packets from this client port.

	As result, the PE should be able to maintain
persistence templates with or without pe_data (CID),
i.e. one per CID and additional without CID, all for the

- According to RFC 6347 4.1.1. Transport Layer Mapping,
"Multiple DTLS records may be placed in a single datagram".
So, if multiple records contain different CIDs, this should
be considered. I can think for some fatal cases where
two records in one packet can be mapped to different
real servers.


Julian Anastasov <ja at ssi.bg>

More information about the lvs-users mailing list