6 ms·

When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the so

When the Unicorn master starts, it loads our app into memory. As soon as it’s ready to serve requests it forks 16 workers. Those workers then select() on the socket, only serving requests they’re capable of handling. In this way the kernel handles the load balancing for us. Wasn't there a heated debate here just the other day about the prefork model? Guess it's at least back en vogue @github.

seiji 17y ago
It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.
- aaronblohowiak 17y ago
  Right, but it is being handled by processes and not threads, which the argument was about the other day.
  - antonovka 17y ago
    The argument encompassed two aspects of the model: 1) "Threads are out -- processes are better than threads." 2) Process-per-connection architectures. The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github's issues with mongrel "thread-killing". More broadly used, look at Servlets and the Servlet 3.0 support for async comet-style event-based request handling. Each request is handled on a thread as necessary. Inter-thread communication (where necessary) is cheap, and this scales just fine. If you use the fork() model and a long-running request blocks the entire process, comet is basically a non-starter. This is why people are interested (and implement) lightweight threads, coroutines, and restartable request implementations. For a conceptual challenge, consider how you would implement a live web chat system that can scale up to a considerable number of clients, with extremely low resource usage, and instant message distribution (no polling). Locally, we implemented this with async servlet support and M:N scheduled scala actors -- a blocking HTTP request doesn't hold a thread or process hostage in the web server or the application, and we can scale up to enormous number of live clients on one machine. The second was merely a lack of understanding of the model. In implementations where fork() is used as an alternative to threads and multiple connections are not handled per sub-process, you quickly run into scaling issues with subprocess memory utilization. In cases where subprocesses handle multiple connections via an event mechanism, you're just using fork() instead of threads. I'm wondering how many servers github is using to keep up with load -- If each 8 core 16 gigabyte (!!!) server can actually only handle 16 concurrent requests via a pool of 16 workers, that's an incredibly poor (and expensive) scaling model.
    - mrshoe 17y ago
      Where were all you people in this thread the other day when I seemed to be the only one who understood the other side of this debate?!? :-P I'm also very curious about the 16 workers per box model. Frankly, that github would ditch haproxy in favor of a pre-fork Ruby web server is shocking. I'm just glad we have CherryPy in the Python world.
      - mojombo 17y ago
        Why is that shocking? Can you elaborate?
        mrshoe 17y ago
        Sure, now that HN has decided that enough time has elapsed to allow me to reply. :-) Actually, antonovka did an excellent job explaining it above. The most important aspect is that threads are "lighter weight" than processes. They use less memory and are quicker to context switch (usually). The result of this lighter weight is that you can spawn more threads than you could processes, on the same hardware. And when you're using one thread/process per connection, that means more concurrent connections on the same hardware. So, if Unicorn used its exact same architecture, but replaced the worker processes with worker threads, you could scale much better. Secondly, haproxy is written in C, which generally means it's going to perform much better and use a lot less memory than a Ruby webserver. This translates, once again, to more output from the same amount of hardware. That's why I was pretty surprised to see that github would choose both Ruby and pre-fork over C and threads (or in the case of haproxy, async, which scales even better).
        joevandyk 17y ago
        The limiting factor of concurrency in a Rails application is the slowness of Ruby, not processes vs threads. If the bottleneck of Ruby were fixed, then the next bottleneck would be the database.
        grandalf 17y ago
        I think you missed the point. The point was not that github stopped using haproxy, but that it no longer needs to use it. Assuming haproxy can work with unix sockets, github could configure nginx/unicorn to use it, but with unicorn its current work load does not require additional load balancing acrobatics beyond what the unix socket does. Similarly, many rails sites use thin/nginx or mongrel/nginx and do not even have a workload that necessitates haproxy. Plus, haproxy adds another layer of complexity, configuration, and management, which is nice to avoid if you can.
- tptacek 17y ago
  Has any mainstream web server ever been pure demand-forked? Apache has been connection pooled since the '90s; the second edition of Unix Network Programming used it as a case study.
  - mrshoe 17y ago
    Exactly. Hence the "pre" in prefork. A "forking" web server (one fork per request) would be incredibly inefficient, whereas prefork is only... slightly inefficient. :-)
    - tlrobinson 17y ago
      How's this for incredibly inefficient http://github.com/tlrobinson/wwwoosh/blob/master/wwwoosh.sh http://github.com/tlrobinson/wwwoosh/blob/master/wwwoosh.sh ;)
  - gaius 17y ago
    Ho ho ho. Back in the day, people used to run httpd off of inetd. I was real smug back then when I replaced that at our site with NCSA httpd, which did indeed fork on demand. Think this was '93 or thenabouts.
    - tptacek 17y ago
      NCSA HTTP was released at the end of '93. Apache was the first mainstream server to prefork a connection pool; it's not demand-forked. There wasn't much advantage to running NCSA standalone vs. out of inetd. I believe Apache was '95; it came out when I was working at an ISP, and I graduated from high school in '94.
- davidw 17y ago
  Except that Apache (and Passenger) don't just start up N worker processes - they can dynamically allocate them as the need arises. Perhaps this is less important for a site like github, which can plan exactly how many to use, but if you have more than one site on the same server, it's really convenient, and, IMO, makes Passenger the best choice for deploying Rails for most people. So it appears as if this setup replicates a small portion of what Apache does without all the bells and whistles.

seiji 17y ago

It's connection pooling, not a fork-per-accept web server. Each worker is select/epoll/kqueue'ing for individual requests.

aaronblohowiak 17y ago
Right, but it is being handled by processes and not threads, which the argument was about the other day.
- antonovka 17y ago
  The argument encompassed two aspects of the model: 1) "Threads are out -- processes are better than threads." 2) Process-per-connection architectures. The first is demonstrably false -- for instance, look at Erlang, which maps lightweight erlang processes to operating system threads, providing SMP scalability at a low cost without running into Github's issues with mongrel "thread-killing". More broadly used, look at Servlets and the Servlet 3.0 support for async comet-style event-based request handling. Each request is handled on a thread as necessary. Inter-thread communication (where necessary) is cheap, and this scales just fine. If you use the fork() model and a long-running request blocks the entire process, comet is basically a non-starter. This is why people are interested (and implement) lightweight threads, coroutines, and restartable request implementations. For a conceptual challenge, consider how you would implement a live web chat system that can scale up to a considerable number of clients, with extremely low resource usage, and instant message distribution (no polling). Locally, we implemented this with async servlet support and M:N scheduled scala actors -- a blocking HTTP request doesn't hold a thread or process hostage in the web server or the application, and we can scale up to enormous number of live clients on one machine. The second was merely a lack of understanding of the model. In implementations where fork() is used as an alternative to threads and multiple connections are not handled per sub-process, you quickly run into scaling issues with subprocess memory utilization. In cases where subprocesses handle multiple connections via an event mechanism, you're just using fork() instead of threads. I'm wondering how many servers github is using to keep up with load -- If each 8 core 16 gigabyte (!!!) server can actually only handle 16 concurrent requests via a pool of 16 workers, that's an incredibly poor (and expensive) scaling model.
  - mrshoe 17y ago
    Where were all you people in this thread the other day when I seemed to be the only one who understood the other side of this debate?!? :-P I'm also very curious about the 16 workers per box model. Frankly, that github would ditch haproxy in favor of a pre-fork Ruby web server is shocking. I'm just glad we have CherryPy in the Python world.
    - mojombo 17y ago
      Why is that shocking? Can you elaborate?
      - mrshoe 17y ago
        Sure, now that HN has decided that enough time has elapsed to allow me to reply. :-) Actually, antonovka did an excellent job explaining it above. The most important aspect is that threads are "lighter weight" than processes. They use less memory and are quicker to context switch (usually). The result of this lighter weight is that you can spawn more threads than you could processes, on the same hardware. And when you're using one thread/process per connection, that means more concurrent connections on the same hardware. So, if Unicorn used its exact same architecture, but replaced the worker processes with worker threads, you could scale much better. Secondly, haproxy is written in C, which generally means it's going to perform much better and use a lot less memory than a Ruby webserver. This translates, once again, to more output from the same amount of hardware. That's why I was pretty surprised to see that github would choose both Ruby and pre-fork over C and threads (or in the case of haproxy, async, which scales even better).
        joevandyk 17y ago
        The limiting factor of concurrency in a Rails application is the slowness of Ruby, not processes vs threads. If the bottleneck of Ruby were fixed, then the next bottleneck would be the database.
        grandalf 17y ago
        I think you missed the point. The point was not that github stopped using haproxy, but that it no longer needs to use it. Assuming haproxy can work with unix sockets, github could configure nginx/unicorn to use it, but with unicorn its current work load does not require additional load balancing acrobatics beyond what the unix socket does. Similarly, many rails sites use thin/nginx or mongrel/nginx and do not even have a workload that necessitates haproxy. Plus, haproxy adds another layer of complexity, configuration, and management, which is nice to avoid if you can.
tptacek 17y ago
Has any mainstream web server ever been pure demand-forked? Apache has been connection pooled since the '90s; the second edition of Unix Network Programming used it as a case study.
- mrshoe 17y ago
  Exactly. Hence the "pre" in prefork. A "forking" web server (one fork per request) would be incredibly inefficient, whereas prefork is only... slightly inefficient. :-)
  - tlrobinson 17y ago
    How's this for incredibly inefficient http://github.com/tlrobinson/wwwoosh/blob/master/wwwoosh.sh http://github.com/tlrobinson/wwwoosh/blob/master/wwwoosh.sh ;)
- gaius 17y ago
  Ho ho ho. Back in the day, people used to run httpd off of inetd. I was real smug back then when I replaced that at our site with NCSA httpd, which did indeed fork on demand. Think this was '93 or thenabouts.
  - tptacek 17y ago
    NCSA HTTP was released at the end of '93. Apache was the first mainstream server to prefork a connection pool; it's not demand-forked. There wasn't much advantage to running NCSA standalone vs. out of inetd. I believe Apache was '95; it came out when I was working at an ISP, and I graduated from high school in '94.
davidw 17y ago
Except that Apache (and Passenger) don't just start up N worker processes - they can dynamically allocate them as the need arises. Perhaps this is less important for a site like github, which can plan exactly how many to use, but if you have more than one site on the same server, it's really convenient, and, IMO, makes Passenger the best choice for deploying Rails for most people. So it appears as if this setup replicates a small portion of what Apache does without all the bells and whistles.