6 ms·

How we fine-tuned HAProxy to achieve 2M concurrent SSL connections (2017)

82 points by piyushgupta27 8y ago

billconan 8y ago
I have a stupid question, there are at most 65536 ports and some are not usable. There is also a file descriptor limitation. How to have that many concurrent connections? https://unix.stackexchange.com/questions/84227/limits-on-the-number-of-file-descriptors https://unix.stackexchange.com/questions/84227/limits-on-the...
- chmod775 8y ago
  There are at most 65536 ports per remote IP. TCP connections are basically identified with an (ip, port) tuple. Also you can set the file descriptor limits to whatever you want.
  - LinuxBender 8y ago
    To further clarify, even if you only had 1 host behind a proxy, you can create multiple listeners for that application on different ports or IP's. I do this for a few of my servers to avoid fin-wait / time-wait assassination attacks. i.e. have apache listen on 127.0.0.* and set up ifcfg-lo-range with IPADDR_START=127.0.0.10 IPADDR_END=127.0.0.99 PREFIX=8 CLONENUM_START=10 Or have the server behind haproxy listen on a few hundred ports.
  - Florin_Andrei 8y ago
    > TCP connections are basically identified with an (ip, port) tuple. I believe the full identifier is a quad: (source ip, source port, dest ip, dest port).
    - mrkstu 8y ago
      Also note that DNS will load balance the IP presented. For instance 'dig'ing yahoo.com gives me the following: ;; ANSWER SECTION: yahoo.com. 92 IN A 72.30.35.9 yahoo.com. 92 IN A 98.138.219.231 yahoo.com. 92 IN A 72.30.35.10 yahoo.com. 92 IN A 98.137.246.7 yahoo.com. 92 IN A 98.138.219.232 yahoo.com. 92 IN A 98.137.246.8 So you can now multiple the quad by the number of public IPs you load up into your DNS record.
  - zokier 8y ago
    Aren't TCP connections more accurately identified by (srcip, srcport, dstip, dstport) tuple? This is somewhat significant as you can easily have tons of IPs in a single box.
    - toredash 8y ago
      Correct
    - guiambros 8y ago
      Yes, but after the initial syn/ack, the daemon will allocate an outgoing port number for the connection. So if you have a single IP address and a burst with hundred of thousands of requests, you will run into problems..
    - chmod775 8y ago
      Yes, but local port and IP are usually fixed when running a HTTP server (unless you're binding to multiple IPs on one machine).
  - dorfsmay 8y ago
    > Also you can set the file descriptor limits to whatever you want. Isn't limited by the amount of memory?
muststopmyths 8y ago
One quibble I would make with the article is that you are not actually limited to ~65k connections from a client. It is only ~65k per IP address on the client (given that they're all talking to a single remote port) You can add NICs or virtual IPs and bind your client instances to specific IP addresses instead of INADDR_ANY.
- ldng 8y ago
  It is addressed in the previous article of the serie.
andyidsinga 8y ago
hot damn! I really appreciate the walk through of the apache bench (ab) results and learning process even though it didn't get them to their objective - I've been thinking about using ab myself, and these are great things to know. I took down a few notes while reading the article: - mentions use of apache bench ( ab ) command for load testing - mentions use of ganglia tool - mentions configuring HAProxy for multi-core using nbproc setting - mentions the 'parallel' tool for running commands in parallel - simulate long run requests by having the server delay a little vs client (work around for ab deficiencies ) - have the server also send different response lengths back simulating varying load - pdsh tool to remote parallel shell (ssh) sessions - vegeta tool - that got them to their scalability / tipping point objective - nodejs (used for their backends) had a default request timeout of 2 mins - used dmesg to learn that haproxy was running out of mem (at around 1.2mm conns) - pdsh to run vegeta tool on multiple machines (acting as clients) - script included in article - mentions haproxy maxconn setting - and verification by checking proc fs limits for the haproxy pid

82 points by piyushgupta27 8y ago

I have a stupid question, there are at most 65536 ports and some are not usable. There is also a file descriptor limitation. How to have that many concurrent connections? https://unix.stackexchange.com/questions/84227/limits-on-the-number-of-file-descriptors https://unix.stackexchange.com/questions/84227/limits-on-the...

chmod775 8y ago
There are at most 65536 ports per remote IP. TCP connections are basically identified with an (ip, port) tuple. Also you can set the file descriptor limits to whatever you want.
- LinuxBender 8y ago
  To further clarify, even if you only had 1 host behind a proxy, you can create multiple listeners for that application on different ports or IP's. I do this for a few of my servers to avoid fin-wait / time-wait assassination attacks. i.e. have apache listen on 127.0.0.* and set up ifcfg-lo-range with IPADDR_START=127.0.0.10 IPADDR_END=127.0.0.99 PREFIX=8 CLONENUM_START=10 Or have the server behind haproxy listen on a few hundred ports.
- Florin_Andrei 8y ago
  > TCP connections are basically identified with an (ip, port) tuple. I believe the full identifier is a quad: (source ip, source port, dest ip, dest port).
  - mrkstu 8y ago
    Also note that DNS will load balance the IP presented. For instance 'dig'ing yahoo.com gives me the following: ;; ANSWER SECTION: yahoo.com. 92 IN A 72.30.35.9 yahoo.com. 92 IN A 98.138.219.231 yahoo.com. 92 IN A 72.30.35.10 yahoo.com. 92 IN A 98.137.246.7 yahoo.com. 92 IN A 98.138.219.232 yahoo.com. 92 IN A 98.137.246.8 So you can now multiple the quad by the number of public IPs you load up into your DNS record.
- zokier 8y ago
  Aren't TCP connections more accurately identified by (srcip, srcport, dstip, dstport) tuple? This is somewhat significant as you can easily have tons of IPs in a single box.
  - toredash 8y ago
    Correct
  - guiambros 8y ago
    Yes, but after the initial syn/ack, the daemon will allocate an outgoing port number for the connection. So if you have a single IP address and a burst with hundred of thousands of requests, you will run into problems..
  - chmod775 8y ago
    Yes, but local port and IP are usually fixed when running a HTTP server (unless you're binding to multiple IPs on one machine).
- dorfsmay 8y ago
  > Also you can set the file descriptor limits to whatever you want. Isn't limited by the amount of memory?

muststopmyths 8y ago

One quibble I would make with the article is that you are not actually limited to ~65k connections from a client. It is only ~65k per IP address on the client (given that they're all talking to a single remote port) You can add NICs or virtual IPs and bind your client instances to specific IP addresses instead of INADDR_ANY.

ldng 8y ago
It is addressed in the previous article of the serie.

andyidsinga 8y ago

hot damn! I really appreciate the walk through of the apache bench (ab) results and learning process even though it didn't get them to their objective - I've been thinking about using ab myself, and these are great things to know. I took down a few notes while reading the article: - mentions use of apache bench ( ab ) command for load testing - mentions use of ganglia tool - mentions configuring HAProxy for multi-core using nbproc setting - mentions the 'parallel' tool for running commands in parallel - simulate long run requests by having the server delay a little vs client (work around for ab deficiencies ) - have the server also send different response lengths back simulating varying load - pdsh tool to remote parallel shell (ssh) sessions - vegeta tool - that got them to their scalability / tipping point objective - nodejs (used for their backends) had a default request timeout of 2 mins - used dmesg to learn that haproxy was running out of mem (at around 1.2mm conns) - pdsh to run vegeta tool on multiple machines (acting as clients) - script included in article - mentions haproxy maxconn setting - and verification by checking proc fs limits for the haproxy pid