7 ms·

The myth that fork is expensive is pervasive, and speaking to the ways that it is true†, well: performance is relative. fork() only takes around 8ms on my Linu

by geocar 8y ago

The myth that fork is expensive is pervasive, and speaking to the ways that it is true†, well: performance is relative. fork() only takes around 8ms on my Linux machine and I can get 100,000 posix_spawn() per second there with 100MB RSS. That's "fast enough" for a large number of applications. †: fork() is a lot slower (over 20x) on Windows

iforgotpassword 8y ago
Related: when I worked on a server (on linux) that would spawn threads for new connections that were mostly short lived, I tried to use a thread pool instead. Hand crafted, with push and pop both O(1). It was still mostly slower than just spawning a new thread every time.
- kevingadd 8y ago
  It's generally considered the right move to just spawn your own thread for things like connections instead of pooling threads - people tend to think they should throw everything into a threadpool but it's not actually encouraged to do that for stuff like web server connections, compiles, etc unless the task is short-lived. I wish more people knew that going in :) Some threadpool APIs actually ask you whether the job is going to take a while and if it is, they function more as a job limiter - not 500 active threads all competing for CPU - than a thread reuser.
inglor 8y ago
8ms per fork means you can only accept 125 connections per second. (per core) That means it's only viable for connections where the connection is very long lived and messages are very sparse (because context switches).
- jondubois 8y ago
  I was going to mention CPU context switching as well, it's very expensive even when you don't account for the initial fork overhead but I guess it's definitely a better idea than CGI for HTTP given the long lived connection. In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.
  - marktangotango 8y ago
    Does this concern hold for containers per machine?
    - jondubois 8y ago
      Yes I would think so because containers run on top of the OS. If you can run multiple containerized apps side-by-side on the same machine at the same time on a single CPU core, then you can be sure that there is some kind of context switching happening. Modern Operating Systems are good at minimizing the amount of context switching. If you run 4 CPU-intensive processes at the same time on a machine which has 4 CPU cores, then the OS will typically assign each process to a different CPU core (with minimal context switching). Then if you launch a 5th CPU-intensive process, then the OS will have no choice but to start doing context switching since it doesn't have any idle cores left. On Linux, based on tests I did a couple of years ago with a multi-process WebSocket server, I can confirm that the penalty of context switching is proportional to the CPU usage of each process. So for example, if you have a very CPU-intensive process sharing a CPU core with a non-intensive process, then the penalty will be small, but if you have two intensive processes sharing the same core, the penalty will be high.
- nabla9 8y ago
  If you use websocket for short lived connections, you are doing something wrong. -- Meta observation: Maybe this is the bane of smartphone era and small screens, but the context of the discussion seems to disappear instantly. Subject: websockets > forking processes > .. aaand the websocket context is lost and we are talking generally about forks in web applications with growing thread.
  - angusp 8y ago
    > If you use websocket for short lived connections, you are doing something wrong. Pretty-much true, but I remember a funny story from Dropbox where their websocket service couldn't come back up after a crash because their normal users trying to re-open super long lived connections all at once was well-beyond the capacity of the system
- geocar 8y ago
  Cores are cheap: I've got 200, and a lot of that cost is page faults. It also means you should use posix_spawn instead of fork+exec since you can control when the page faults occur better.
- TickleSteve 8y ago
  I find it ironic that people will complain about milliseconds for a fork and talk about the time for context switches... and then serve their pages using an interpreted language that is an order of magnitude slower than it could be in a compiled language...
  - rowanG077 8y ago
    Who did that? I saw no one talking about interpreted languages here.
    - marcosdumay 8y ago
      The comment immediately below the GP that claims that performance is relative literally says that you don't have to care about forking in Node.js.
      - ibotty 8y ago
        Not defending node.js, but it is not interpreted but run with jit compilation.
- rakoo 8y ago
  I think the real issue now is not how fast you can accept and process a new connection, but how many you can have in parallel. If you're going to have a native os process for each, you'll soon run out of resources
- zorga 8y ago
  125 connections per second is vastly more than most business applications will ever see in their entire lifetimes. That means this approach is viable most of the time for most of the things people write. Everyone likes to think they're the next big unicorn and code for it; you're wasting your time, you're not.
mig_ 8y ago
The initial fork is fast because of the copy on write memory semantic. You'll pay a price later.
walterstucco 8y ago
8ms Is eternity
NathanKP 8y ago
I see this argument all the time but its not the fork() that is the most expensive anymore, its the actual program initialization after the fork(). If your app is non trivial (lets say a websocket based chat server that needs to persist messages to a DB and use pubsub to sync them to other processes) it probably needs a connection to a database, a connection to a cache server like Redis or Memcached, etc, or perhaps a connection to some other backend service. Reinitializing these dependency connections from scratch in a new forked process for every single incoming websocket connection is expensive and slow. On the other hand a Node.js or Go program can have a preestablished pool of keep alive connections to the backends already ready to go and reuse that connection pool for many hundreds or even thousands of concurrent websocket connections. You can approximate something like this with the fork() model by having a local daemon process that manages the connection pool and have your forked process talk to that local helper daemon when it needs a connection, but you are still going to pay a penalty for that compared with having a fully preestablished connection ready to use right there in the process already
- zorga 8y ago
  > If your app is non trivial (lets say a websocket based chat server that needs to persist messages to a DB and use pubsub to sync them to other processes) it probably needs a connection to a database, a connection to a cache server like Redis or Memcached, etc All things that are already generally connection pooled so this point is moot. > On the other hand a Node.js or Go program can have a preestablished pool of keep alive connections to the backends already ready to go Which has nothing remotely to do with why they scale. Also so can the forking process, it can quite easily even have a pool of preforked workers ready to go and handle requests to avoid the cost of the fork. None of the things you've said have anything to do with why Node and Go are more scalable than a forking server. The issue is ram, not the price of setting up connections; processes are much heavier on ram and limit your number of concurrent clients, Node and Go are event driven single process apps that scale better because they consume less resources, not because of all that other stuff you said. I don't think you really understand what you're saying or you're comparing apples and oranges and don't know it.
- geocar 8y ago
  Then consider posix_spawn(), since it pays that cost, but then we have to pay IPC as well: fasthttp(go) can do around 80k/sec on my laptop; dash(C) can do almost 88k/sec on my laptop†. As soon as dash does IPC, it drops to around 51k/sec, and as soon as it needs a reply, we're down to 25k/sec††. I see no reason to believe fasthttp would do any faster. That means I'm spending around 70% of my time in IPC -- something posix_spawn() would let me avoid (if my application were designed to do so). My same laptop will do 100k/sec posix_spawn() so I'd find this difficult to believe (1-4 msec per call) fork() or exec() is the bottleneck for any application with this architecture. Do you think posix_spawn() represents 70% of your costs? If our goal is to beat 51k/sec requests, sure, but NodeJS on my laptop (btw) gets 10k/sec, so if it's a contender, I'd say posix_spawn() is as well. † https://github.com/geocar/dash/blob/master/README.techempower.md https://github.com/geocar/dash/blob/master/README.techempowe... †† https://github.com/geocar/dash/blob/master/README.md https://github.com/geocar/dash/blob/master/README.md
porpoisely 8y ago
Only 8 ms? That's slower than a ping. You were right about performance being relative. And of course the trade off between ease of development, use and performance. At the end of the day, practical considerations are going to determine what is "expensive".
- geocar 8y ago
  > Only 8 ms? That's slower than a ping. Also depends what you're pinging. I'm in the UK, so everything in America is 30-80msec away anyway.

by geocar 8y ago

iforgotpassword 8y ago

Related: when I worked on a server (on linux) that would spawn threads for new connections that were mostly short lived, I tried to use a thread pool instead. Hand crafted, with push and pop both O(1). It was still mostly slower than just spawning a new thread every time.

kevingadd 8y ago
It's generally considered the right move to just spawn your own thread for things like connections instead of pooling threads - people tend to think they should throw everything into a threadpool but it's not actually encouraged to do that for stuff like web server connections, compiles, etc unless the task is short-lived. I wish more people knew that going in :) Some threadpool APIs actually ask you whether the job is going to take a while and if it is, they function more as a job limiter - not 500 active threads all competing for CPU - than a thread reuser.

inglor 8y ago

8ms per fork means you can only accept 125 connections per second. (per core) That means it's only viable for connections where the connection is very long lived and messages are very sparse (because context switches).

jondubois 8y ago
I was going to mention CPU context switching as well, it's very expensive even when you don't account for the initial fork overhead but I guess it's definitely a better idea than CGI for HTTP given the long lived connection. In an ideal scenario, you shouldn't have more active processes than you have CPU cores. As soon as that happens, context switching kicks in and performance degrades sharply.
- marktangotango 8y ago
  Does this concern hold for containers per machine?
  - jondubois 8y ago
    Yes I would think so because containers run on top of the OS. If you can run multiple containerized apps side-by-side on the same machine at the same time on a single CPU core, then you can be sure that there is some kind of context switching happening. Modern Operating Systems are good at minimizing the amount of context switching. If you run 4 CPU-intensive processes at the same time on a machine which has 4 CPU cores, then the OS will typically assign each process to a different CPU core (with minimal context switching). Then if you launch a 5th CPU-intensive process, then the OS will have no choice but to start doing context switching since it doesn't have any idle cores left. On Linux, based on tests I did a couple of years ago with a multi-process WebSocket server, I can confirm that the penalty of context switching is proportional to the CPU usage of each process. So for example, if you have a very CPU-intensive process sharing a CPU core with a non-intensive process, then the penalty will be small, but if you have two intensive processes sharing the same core, the penalty will be high.
nabla9 8y ago
If you use websocket for short lived connections, you are doing something wrong. -- Meta observation: Maybe this is the bane of smartphone era and small screens, but the context of the discussion seems to disappear instantly. Subject: websockets > forking processes > .. aaand the websocket context is lost and we are talking generally about forks in web applications with growing thread.
- angusp 8y ago
  > If you use websocket for short lived connections, you are doing something wrong. Pretty-much true, but I remember a funny story from Dropbox where their websocket service couldn't come back up after a crash because their normal users trying to re-open super long lived connections all at once was well-beyond the capacity of the system
geocar 8y ago
Cores are cheap: I've got 200, and a lot of that cost is page faults. It also means you should use posix_spawn instead of fork+exec since you can control when the page faults occur better.
TickleSteve 8y ago
I find it ironic that people will complain about milliseconds for a fork and talk about the time for context switches... and then serve their pages using an interpreted language that is an order of magnitude slower than it could be in a compiled language...
- rowanG077 8y ago
  Who did that? I saw no one talking about interpreted languages here.
  - marcosdumay 8y ago
    The comment immediately below the GP that claims that performance is relative literally says that you don't have to care about forking in Node.js.
    - ibotty 8y ago
      Not defending node.js, but it is not interpreted but run with jit compilation.
rakoo 8y ago
I think the real issue now is not how fast you can accept and process a new connection, but how many you can have in parallel. If you're going to have a native os process for each, you'll soon run out of resources
zorga 8y ago
125 connections per second is vastly more than most business applications will ever see in their entire lifetimes. That means this approach is viable most of the time for most of the things people write. Everyone likes to think they're the next big unicorn and code for it; you're wasting your time, you're not.

mig_ 8y ago

The initial fork is fast because of the copy on write memory semantic. You'll pay a price later.

walterstucco 8y ago

8ms Is eternity

NathanKP 8y ago

I see this argument all the time but its not the fork() that is the most expensive anymore, its the actual program initialization after the fork(). If your app is non trivial (lets say a websocket based chat server that needs to persist messages to a DB and use pubsub to sync them to other processes) it probably needs a connection to a database, a connection to a cache server like Redis or Memcached, etc, or perhaps a connection to some other backend service. Reinitializing these dependency connections from scratch in a new forked process for every single incoming websocket connection is expensive and slow. On the other hand a Node.js or Go program can have a preestablished pool of keep alive connections to the backends already ready to go and reuse that connection pool for many hundreds or even thousands of concurrent websocket connections. You can approximate something like this with the fork() model by having a local daemon process that manages the connection pool and have your forked process talk to that local helper daemon when it needs a connection, but you are still going to pay a penalty for that compared with having a fully preestablished connection ready to use right there in the process already

zorga 8y ago
> If your app is non trivial (lets say a websocket based chat server that needs to persist messages to a DB and use pubsub to sync them to other processes) it probably needs a connection to a database, a connection to a cache server like Redis or Memcached, etc All things that are already generally connection pooled so this point is moot. > On the other hand a Node.js or Go program can have a preestablished pool of keep alive connections to the backends already ready to go Which has nothing remotely to do with why they scale. Also so can the forking process, it can quite easily even have a pool of preforked workers ready to go and handle requests to avoid the cost of the fork. None of the things you've said have anything to do with why Node and Go are more scalable than a forking server. The issue is ram, not the price of setting up connections; processes are much heavier on ram and limit your number of concurrent clients, Node and Go are event driven single process apps that scale better because they consume less resources, not because of all that other stuff you said. I don't think you really understand what you're saying or you're comparing apples and oranges and don't know it.
geocar 8y ago
Then consider posix_spawn(), since it pays that cost, but then we have to pay IPC as well: fasthttp(go) can do around 80k/sec on my laptop; dash(C) can do almost 88k/sec on my laptop†. As soon as dash does IPC, it drops to around 51k/sec, and as soon as it needs a reply, we're down to 25k/sec††. I see no reason to believe fasthttp would do any faster. That means I'm spending around 70% of my time in IPC -- something posix_spawn() would let me avoid (if my application were designed to do so). My same laptop will do 100k/sec posix_spawn() so I'd find this difficult to believe (1-4 msec per call) fork() or exec() is the bottleneck for any application with this architecture. Do you think posix_spawn() represents 70% of your costs? If our goal is to beat 51k/sec requests, sure, but NodeJS on my laptop (btw) gets 10k/sec, so if it's a contender, I'd say posix_spawn() is as well. † https://github.com/geocar/dash/blob/master/README.techempower.md https://github.com/geocar/dash/blob/master/README.techempowe... †† https://github.com/geocar/dash/blob/master/README.md https://github.com/geocar/dash/blob/master/README.md

porpoisely 8y ago

Only 8 ms? That's slower than a ping. You were right about performance being relative. And of course the trade off between ease of development, use and performance. At the end of the day, practical considerations are going to determine what is "expensive".

geocar 8y ago
> Only 8 ms? That's slower than a ping. Also depends what you're pinging. I'm in the UK, so everything in America is 30-80msec away anyway.