6 ms·

8ms per fork means you can only accept 125 connections per second. (per core) That means it's only viable for connections where the connection is very long liv

by inglor 8y ago

8ms per fork means you can only accept 125 connections per second. (per core) That means it's only viable for connections where the connection is very long lived and messages are very sparse (because context switches).

jondubois 8y ago
I was going to mention CPU context switching as well, it's very expensive even when you don't account for the initial fork overhead but I guess it's definitely a better idea than CGI for HTTP given the long lived connection. In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.
- marktangotango 8y ago
  Does this concern hold for containers per machine?
  - jondubois 8y ago
    Yes I would think so because containers run on top of the OS. If you can run multiple containerized apps side-by-side on the same machine at the same time on a single CPU core, then you can be sure that there is some kind of context switching happening. Modern Operating Systems are good at minimizing the amount of context switching. If you run 4 CPU-intensive processes at the same time on a machine which has 4 CPU cores, then the OS will typically assign each process to a different CPU core (with minimal context switching). Then if you launch a 5th CPU-intensive process, then the OS will have no choice but to start doing context switching since it doesn't have any idle cores left. On Linux, based on tests I did a couple of years ago with a multi-process WebSocket server, I can confirm that the penalty of context switching is proportional to the CPU usage of each process. So for example, if you have a very CPU-intensive process sharing a CPU core with a non-intensive process, then the penalty will be small, but if you have two intensive processes sharing the same core, the penalty will be high.
nabla9 8y ago
If you use websocket for short lived connections, you are doing something wrong. -- Meta observation: Maybe this is the bane of smartphone era and small screens, but the context of the discussion seems to disappear instantly. Subject: websockets > forking processes > .. aaand the websocket context is lost and we are talking generally about forks in web applications with growing thread.
- angusp 8y ago
  > If you use websocket for short lived connections, you are doing something wrong. Pretty-much true, but I remember a funny story from Dropbox where their websocket service couldn't come back up after a crash because their normal users trying to re-open super long lived connections all at once was well-beyond the capacity of the system
geocar 8y ago
Cores are cheap: I've got 200, and a lot of that cost is page faults. It also means you should use posix_spawn instead of fork+exec since you can control when the page faults occur better.
TickleSteve 8y ago
I find it ironic that people will complain about milliseconds for a fork and talk about the time for context switches... and then serve their pages using an interpreted language that is an order of magnitude slower than it could be in a compiled language...
- rowanG077 8y ago
  Who did that? I saw no one talking about interpreted languages here.
  - marcosdumay 8y ago
    The comment immediately below the GP that claims that performance is relative literally says that you don't have to care about forking in Node.js.
    - ibotty 8y ago
      Not defending node.js, but it is not interpreted but run with jit compilation.
rakoo 8y ago
I think the real issue now is not how fast you can accept and process a new connection, but how many you can have in parallel. If you're going to have a native os process for each, you'll soon run out of resources
zorga 8y ago
125 connections per second is vastly more than most business applications will ever see in their entire lifetimes. That means this approach is viable most of the time for most of the things people write. Everyone likes to think they're the next big unicorn and code for it; you're wasting your time, you're not.

by inglor 8y ago

jondubois 8y ago

I was going to mention CPU context switching as well, it's very expensive even when you don't account for the initial fork overhead but I guess it's definitely a better idea than CGI for HTTP given the long lived connection. In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.

marktangotango 8y ago
Does this concern hold for containers per machine?
- jondubois 8y ago
  Yes I would think so because containers run on top of the OS. If you can run multiple containerized apps side-by-side on the same machine at the same time on a single CPU core, then you can be sure that there is some kind of context switching happening. Modern Operating Systems are good at minimizing the amount of context switching. If you run 4 CPU-intensive processes at the same time on a machine which has 4 CPU cores, then the OS will typically assign each process to a different CPU core (with minimal context switching). Then if you launch a 5th CPU-intensive process, then the OS will have no choice but to start doing context switching since it doesn't have any idle cores left. On Linux, based on tests I did a couple of years ago with a multi-process WebSocket server, I can confirm that the penalty of context switching is proportional to the CPU usage of each process. So for example, if you have a very CPU-intensive process sharing a CPU core with a non-intensive process, then the penalty will be small, but if you have two intensive processes sharing the same core, the penalty will be high.

nabla9 8y ago

If you use websocket for short lived connections, you are doing something wrong. -- Meta observation: Maybe this is the bane of smartphone era and small screens, but the context of the discussion seems to disappear instantly. Subject: websockets > forking processes > .. aaand the websocket context is lost and we are talking generally about forks in web applications with growing thread.

angusp 8y ago
> If you use websocket for short lived connections, you are doing something wrong. Pretty-much true, but I remember a funny story from Dropbox where their websocket service couldn't come back up after a crash because their normal users trying to re-open super long lived connections all at once was well-beyond the capacity of the system

geocar 8y ago

Cores are cheap: I've got 200, and a lot of that cost is page faults. It also means you should use posix_spawn instead of fork+exec since you can control when the page faults occur better.

TickleSteve 8y ago

I find it ironic that people will complain about milliseconds for a fork and talk about the time for context switches... and then serve their pages using an interpreted language that is an order of magnitude slower than it could be in a compiled language...

rowanG077 8y ago
Who did that? I saw no one talking about interpreted languages here.
- marcosdumay 8y ago
  The comment immediately below the GP that claims that performance is relative literally says that you don't have to care about forking in Node.js.
  - ibotty 8y ago
    Not defending node.js, but it is not interpreted but run with jit compilation.

rakoo 8y ago

I think the real issue now is not how fast you can accept and process a new connection, but how many you can have in parallel. If you're going to have a native os process for each, you'll soon run out of resources

zorga 8y ago

125 connections per second is vastly more than most business applications will ever see in their entire lifetimes. That means this approach is viable most of the time for most of the things people write. Everyone likes to think they're the next big unicorn and code for it; you're wasting your time, you're not.