Websites need a traffic rate limiting feature to keep their backend safe from abusive or malfunctioning clients. This requires the ability to track user sessions of a particular type and/or from a given IP address. HAProxy is a HTTP proxy which can be configured as a reverse proxy to protect a website. It uses an in-memory table to store state related to incoming HTTP connections, indexed by a key such as the client IP address. This table is called a stick-table and is enabled using the ‘stick-table’ directive in the haproxy config file.

The stick-table directive allows specifying the key, the size of the table, the duration an entry (key) is kept in seconds and various counts such as currently active connections, connection rate, http request rate, http error rate etc.

Stick tables are very useful for rate-limiting traffic and tagging traffic that meets certain criteria such as a high connection or error rate with a header which can be used by the backend to log the traffic. They can be used to provide detection of and protection against DDoS attacks.

HAProxy receives client requests in its frontend and sends those requests to servers in its backend.  The config file has corresponding frontend and backend sections.

The origin of this rate-limiting feature request along with an example is at https://blog.serverfault.com/2010/08/26/1016491873/ . Serverfault is a high traffic website so it is a good indication if the feature works for them.

Here’s an example.

frontend http
    bind *:2550

stick-table type ip size 200k expire 10m store gpc0

# check the source before tracking counters, that will allow it to
# expire the entry even if there is still activity.
acl whitelist src 192.168.1.154
acl source_is_abuser src_get_gpc0(http) gt 0
use_backend ease-up-y0 if source_is_abuser
tcp-request connection track-sc1 src if ! source_is_abuser

acl is_test1 hdr_sub(host) -i test1.com
acl is_test2 hdr_sub(host) -i test2.com

use_backend test1  if is_test1
use_backend test2  if is_test2

backend test1 
stick-table type ip size 200k expire 30s store conn_rate(100s),bytes_out_rate(60s) 
acl whitelist src 192.168.1.154

# values below are specific to the backend
tcp-request content  track-sc2 src
acl conn_rate_abuse  sc2_conn_rate gt 3
acl data_rate_abuse  sc2_bytes_out_rate  gt 20000000

# abuse is marked in the frontend so that it's shared between all sites
acl mark_as_abuser   sc1_inc_gpc0 gt 0
tcp-request content  reject if conn_rate_abuse !whitelist mark_as_abuser
tcp-request content  reject if data_rate_abuse mark_as_abuser

server local_apache localhost:80

Note that the frontend and backend sections have their own stick-table sections.

A general strategy would be to allow enough buffer for legitimate traffic to pass in, drop abnormally high traffic and flag intermediate risk traffic to the backend so it can either drop it or log the request for appropriate action, including potentially adding the IP to an abusers list for correlation, reverse lookup and other analysis. These objectives are achievable with stick-tables.

An overview of the HAProxy config file with the sections global, defaults, frontend, backend is here.

Stick tables use elastic binary trees-

https://github.com/haproxy/haproxy/blob/master/include/types/stick_table.h

https://github.com/haproxy/haproxy/blob/master/src/stick_table.c

https://wtarreau.blogspot.com/2011/12/elastic-binary-trees-ebtree.html

Related, for analysis of packet captures in DDoS context, a useful tool is python dpkt – https://mmishou.wordpress.com/2010/04/13/passive-dns-mining-from-pcap-with-dpkt-python .