8 ms·

How can the arguably best piece of open source tech keep get better? The only thing I could ask for is a REST admin api to assist with my deploys.

Scarbutt 9y ago
How about adding support for serving static files?
- jimktrains2 9y ago
  It's not a web server.
- osrec 9y ago
  Maybe, but you can always put something more specialised behind it to do that. The fact that haproxy is light-weight makes it all the more brilliant in my opinion!
- rkeene2 9y ago
  If you want something in the spirit of HAProxy for serving static content, filed ( http://filed.rkeene.org/ http://filed.rkeene.org/ ) is the fastest static HTTP/1.1 server in the world. Putting it behind HAProxy 1.8 for TLS and HTTP/2 would be trivial until those features are directly available.
  - snvzz 9y ago
    >is the fastest static HTTP/1.1 server in the world As you're the author and you're promoting your own work, may I ask what sort of testing this claim is supported on?
    - rkeene2 9y ago
      This claim is based on two parts: benchmarks users have performed and the fact that nobody has called me on the claim :-) Seriously though if you have the time to benchmark it and compare to your favorite alternative please post the results !
      - wtarreau 9y ago
        Well, I didn't know it so I tried it on my laptop but sadly that doesn't reflect my observations. Injecting here with 2000 concurrent connections results in a very fluctuating load oscillating between 2000 and 31000 requests/s (column ^h/s) for a small object like mime.types : hits ^hits hits/s ^h/s bytes kB/s last errs tout htime sdht ptime 32871 32871 31515 31515 11997915 11503 11503 0 0 19.1 92.2 16.5 41915 9044 20957 9450 15298975 7649 3449 0 0 23.4 128.0 30.8 43968 2053 14651 2050 16048320 5347 748 0 0 8.3 59.1 9.8 65994 22026 16494 22026 24087810 6020 8039 0 0 60.4 393.2 66.5 69176 3182 13832 3182 25249240 5048 1161 0 0 70.1 393.2 109.8 69176 0 11527 0 25249240 4207 0 0 0 0.0 0.0 0.0 69176 0 9880 0 25249240 3606 0 0 0 0.0 0.0 0.0 69746 570 8717 570 25457290 3181 208 0 0 941.4 1847.7 835.2 69746 0 7748 0 25457290 2828 0 0 0 0.0 0.0 0.0 69746 0 6973 0 25457290 2545 0 0 0 0.0 0.0 0.0 And after that it totally stops responding until I restart it. On 404 it's more around 37000. This is using 2 threads. I looked at the code and saw a select() in use so that limits the number of concurrent connections to ~512 on recent libcs (1 fd for the socket, 1 fd for the file, 1024 total). Reducing the number of concurrent connections seemed to help a bit (it delayed the occurrence a bit). Also it doesn't seem to support keep-alive so we can't get more performance on the client side by reducing the kernel-side work in the TCP stack. I'm getting the same level of performance out of a single process on thttpd. Doing the same stuff with haproxy gives me 72000 requests/s in close mode as well delivering a small object with the errorfile trick, at only 80% CPU (my load generator reached its limit), so I guess there's still room for improvement since it's possible to achieve twice the performance using 80% of a single thread on the same machine, and it reaches 88k using the cache. I don't know how other servers compare though, but I definitely think that some might get better results.
        rkeene2 9y ago
        select() should definitely never get used -- it's only even compiled in to the executable if you enable the non-default option of "FILED_NONBLOCK_HTTP" which you almost certainly don't want. Did you compile filed in this way ? It sounds like you grep'd the source for select() but didn't actually see that it was #ifdef'd out. The number of concurrent connections is limited by your resource limits and the fact that file descriptors are cached. You can tune the cache, but you should update your resource limits if so, this is documented here in the man page available here: http://filed.rkeene.org/fossil/home?name=Manual http://filed.rkeene.org/fossil/home?name=Manual It does support keep-alive and I have no idea why you think it doesn't ( http://filed.rkeene.org/fossil/artifact?ln=1130-1132%201240-1250&name=8f507e131591c7e7 http://filed.rkeene.org/fossil/artifact?ln=1130-1132%201240-... ) Using only 2 threads (way less than the default) is also very sub-optimal since most of those threads will be waiting for the kernel to deliver the file. On average filed does about 2 system calls before asking the kernel to send the file to the client -- One read() of the HTTP request, one write() of the HTTP response header, and then the sendfile() of the contents. There's no reason not to use more threads than 2 other than to limit I/O (since they do not increase CPU utilization significantly).
  - reacharavindh 9y ago
    I like the idea of doing one thing well, and being great at it. Out of curiosity, why depend on haproxy or something else to provide http/2 support? Wouldn’t it make filed even faster (by letting the client pull all assets in parallel) if it supported natively for serving static files? I’m imagining static web pages with a few images to load. As always, appreciate your open source work! Thanks!
    - rkeene2 9y ago
      HTTP/2 has been on the To Do list for a while and is a great idea for any HTTP server that wishes to remain relevant. I just suggested HAProxy to be topical and address HTTP/2 right now.
- bedis9 9y ago
  Let me say it again: no disks I/O in HAProxy at run time. Simply put an nginx locally and use HAProxy's cache (or nginx one).
osrec 9y ago
Have you tried hatop? Command line only, but very useful.
mschuster91 9y ago
> The only thing I could ask for is a REST admin api to assist with my deploys. I run haproxy in docker - create a shellscript that allows your jenkins (or whatever you use) to send a SIGHUP to the haproxy container, and it will reload the config.
- m11r 9y ago
  Out of curiosity, do you use host networking for higher performance (latency, conn/s, etc.) on those containers, or use the default docker-proxy forwarding?
  - mschuster91 9y ago
    I use --net=host for the haproxy container because I dont want to restart the container every time I need to expose a new frontend port. In fact this is my biggest issue with docker - that properties like bound ports or volumes cant be changed on the fly and without deleting the container.
- djsumdog 9y ago
  I'm currently working on getting this setup myself. I eventually decided to exposing /var/run/docker.sock to my Certbot container and have a posthook that sends -HUP to the HAProxy container, which has a wrapper script that forwards HUP to haproxy's PID. It would be nice if I could just do this via the HAProxy admin socket .. or not even reloading the whole config but just tell it to rescan the SSL certs.
  - mschuster91 9y ago
    > I eventually decided to exposing /var/run/docker.sock to my Certbot container Don't do this, please. It is a security risk as access to the docker socket can be used to take over the entire system as root.
djsumdog 9y ago
> How can the arguably best piece of open source tech keep get better? Letsencrypt support? I realize that's a big undertaking though. There is a Lua plugin, but it does depend on certbot being installed and running, which is a bit difficult if you use the docker image (there are some docker containers on github that achieve this by running supervisord in the docker container for cron/certbot and haproxy). This is probably outside the scope of HAProxy really though, and could probably be implemented entirely as a lua plugin that handles ACME, maybe? Kudos to the HAProxy devs and contributors. I've always been a fan and great to see them pushing forward with these big improvements.
bedis9 9y ago
> The only thing I could ask for is a REST admin api to assist with my deploys. For now, there is one in HAProxy Enterprise, and it manages HAProxy's configuration file and triggers reload. We're working on opening it, as soon as we have improved it: make it use both HAProxy Runtime API (stats socket) and configuration file to trigger reloads only when required. Stay tuned as we say :)
- sieabah 9y ago
  Oh is there a plan to move it towards community? Here I thought the one big feature is behind a paywall. Competing with https://traefik.io/ https://traefik.io/ in that regard.

Scarbutt 9y ago

How about adding support for serving static files?

jimktrains2 9y ago
It's not a web server.
osrec 9y ago
Maybe, but you can always put something more specialised behind it to do that. The fact that haproxy is light-weight makes it all the more brilliant in my opinion!
rkeene2 9y ago
If you want something in the spirit of HAProxy for serving static content, filed ( http://filed.rkeene.org/ http://filed.rkeene.org/ ) is the fastest static HTTP/1.1 server in the world. Putting it behind HAProxy 1.8 for TLS and HTTP/2 would be trivial until those features are directly available.
- snvzz 9y ago
  >is the fastest static HTTP/1.1 server in the world As you're the author and you're promoting your own work, may I ask what sort of testing this claim is supported on?
  - rkeene2 9y ago
    This claim is based on two parts: benchmarks users have performed and the fact that nobody has called me on the claim :-) Seriously though if you have the time to benchmark it and compare to your favorite alternative please post the results !
    - wtarreau 9y ago
      Well, I didn't know it so I tried it on my laptop but sadly that doesn't reflect my observations. Injecting here with 2000 concurrent connections results in a very fluctuating load oscillating between 2000 and 31000 requests/s (column ^h/s) for a small object like mime.types : hits ^hits hits/s ^h/s bytes kB/s last errs tout htime sdht ptime 32871 32871 31515 31515 11997915 11503 11503 0 0 19.1 92.2 16.5 41915 9044 20957 9450 15298975 7649 3449 0 0 23.4 128.0 30.8 43968 2053 14651 2050 16048320 5347 748 0 0 8.3 59.1 9.8 65994 22026 16494 22026 24087810 6020 8039 0 0 60.4 393.2 66.5 69176 3182 13832 3182 25249240 5048 1161 0 0 70.1 393.2 109.8 69176 0 11527 0 25249240 4207 0 0 0 0.0 0.0 0.0 69176 0 9880 0 25249240 3606 0 0 0 0.0 0.0 0.0 69746 570 8717 570 25457290 3181 208 0 0 941.4 1847.7 835.2 69746 0 7748 0 25457290 2828 0 0 0 0.0 0.0 0.0 69746 0 6973 0 25457290 2545 0 0 0 0.0 0.0 0.0 And after that it totally stops responding until I restart it. On 404 it's more around 37000. This is using 2 threads. I looked at the code and saw a select() in use so that limits the number of concurrent connections to ~512 on recent libcs (1 fd for the socket, 1 fd for the file, 1024 total). Reducing the number of concurrent connections seemed to help a bit (it delayed the occurrence a bit). Also it doesn't seem to support keep-alive so we can't get more performance on the client side by reducing the kernel-side work in the TCP stack. I'm getting the same level of performance out of a single process on thttpd. Doing the same stuff with haproxy gives me 72000 requests/s in close mode as well delivering a small object with the errorfile trick, at only 80% CPU (my load generator reached its limit), so I guess there's still room for improvement since it's possible to achieve twice the performance using 80% of a single thread on the same machine, and it reaches 88k using the cache. I don't know how other servers compare though, but I definitely think that some might get better results.
      - rkeene2 9y ago
        select() should definitely never get used -- it's only even compiled in to the executable if you enable the non-default option of "FILED_NONBLOCK_HTTP" which you almost certainly don't want. Did you compile filed in this way ? It sounds like you grep'd the source for select() but didn't actually see that it was #ifdef'd out. The number of concurrent connections is limited by your resource limits and the fact that file descriptors are cached. You can tune the cache, but you should update your resource limits if so, this is documented here in the man page available here: http://filed.rkeene.org/fossil/home?name=Manual http://filed.rkeene.org/fossil/home?name=Manual It does support keep-alive and I have no idea why you think it doesn't ( http://filed.rkeene.org/fossil/artifact?ln=1130-1132%201240-1250&name=8f507e131591c7e7 http://filed.rkeene.org/fossil/artifact?ln=1130-1132%201240-... ) Using only 2 threads (way less than the default) is also very sub-optimal since most of those threads will be waiting for the kernel to deliver the file. On average filed does about 2 system calls before asking the kernel to send the file to the client -- One read() of the HTTP request, one write() of the HTTP response header, and then the sendfile() of the contents. There's no reason not to use more threads than 2 other than to limit I/O (since they do not increase CPU utilization significantly).
- reacharavindh 9y ago
  I like the idea of doing one thing well, and being great at it. Out of curiosity, why depend on haproxy or something else to provide http/2 support? Wouldn’t it make filed even faster (by letting the client pull all assets in parallel) if it supported natively for serving static files? I’m imagining static web pages with a few images to load. As always, appreciate your open source work! Thanks!
  - rkeene2 9y ago
    HTTP/2 has been on the To Do list for a while and is a great idea for any HTTP server that wishes to remain relevant. I just suggested HAProxy to be topical and address HTTP/2 right now.
bedis9 9y ago
Let me say it again: no disks I/O in HAProxy at run time. Simply put an nginx locally and use HAProxy's cache (or nginx one).

osrec 9y ago

Have you tried hatop? Command line only, but very useful.

mschuster91 9y ago

> The only thing I could ask for is a REST admin api to assist with my deploys. I run haproxy in docker - create a shellscript that allows your jenkins (or whatever you use) to send a SIGHUP to the haproxy container, and it will reload the config.

m11r 9y ago
Out of curiosity, do you use host networking for higher performance (latency, conn/s, etc.) on those containers, or use the default docker-proxy forwarding?
- mschuster91 9y ago
  I use --net=host for the haproxy container because I dont want to restart the container every time I need to expose a new frontend port. In fact this is my biggest issue with docker - that properties like bound ports or volumes cant be changed on the fly and without deleting the container.
djsumdog 9y ago
I'm currently working on getting this setup myself. I eventually decided to exposing /var/run/docker.sock to my Certbot container and have a posthook that sends -HUP to the HAProxy container, which has a wrapper script that forwards HUP to haproxy's PID. It would be nice if I could just do this via the HAProxy admin socket .. or not even reloading the whole config but just tell it to rescan the SSL certs.
- mschuster91 9y ago
  > I eventually decided to exposing /var/run/docker.sock to my Certbot container Don't do this, please. It is a security risk as access to the docker socket can be used to take over the entire system as root.

djsumdog 9y ago

> How can the arguably best piece of open source tech keep get better? Letsencrypt support? I realize that's a big undertaking though. There is a Lua plugin, but it does depend on certbot being installed and running, which is a bit difficult if you use the docker image (there are some docker containers on github that achieve this by running supervisord in the docker container for cron/certbot and haproxy). This is probably outside the scope of HAProxy really though, and could probably be implemented entirely as a lua plugin that handles ACME, maybe? Kudos to the HAProxy devs and contributors. I've always been a fan and great to see them pushing forward with these big improvements.

bedis9 9y ago

> The only thing I could ask for is a REST admin api to assist with my deploys. For now, there is one in HAProxy Enterprise, and it manages HAProxy's configuration file and triggers reload. We're working on opening it, as soon as we have improved it: make it use both HAProxy Runtime API (stats socket) and configuration file to trigger reloads only when required. Stay tuned as we say :)

sieabah 9y ago
Oh is there a plan to move it towards community? Here I thought the one big feature is behind a paywall. Competing with https://traefik.io/ https://traefik.io/ in that regard.