4 ms·

Please find one web server being actively developed using one process per request. Handling thousands of concurrent requests is table stakes for a simple web s

Please find one web server being actively developed using one process per request. Handling thousands of concurrent requests is table stakes for a simple web server. Handling thousands of concurrent processes is beyond most OSs. The context switching overhead alone would consume much of the CPU of the system. Even hundreds of processes will mean a good fraction of the CPU being spent solely on context switching - which is a terrible place to be.

nosefrog 1y ago
We did that at Dropbox in Python for a while. Though they switched to async after I left.
monkeyelite 1y ago
> Handling thousands of concurrent processes is beyond most OS It works fine on Linux - the operating system for the internet. Have you tried it? > good fraction of the CPU being spent solely on context switching I was waiting for this one. Threads and processes do the same amount of context switching. The overhead of processes switch is a little higher. The main cost is memory.
- ordu 1y ago
  > Threads and processes do the same amount of context switching. Yes, therefore real webservers use a limited amount of threads/processes (in the same ballpark as a number of CPU cores). Modern approach is to use green threads which are really cheap to switch, it is like store registers, read registers and jmp. > The main cost is memory. The main cost is scheduling, not switching per se. Preemptive multitasking needs to deal with priorities to not waste time, and algorithms that do it are O(N) mostly. All these O(N) calculations needs to be completed multiple times per second, the higher the frequency of switching the more work to do. When you have thousands of processes it is the main cost. If you have tens of thousands it starts to bite hard.
  - monkeyelite 1y ago
    > The main cost is scheduling, not switching per se. Preemptive multitasking needs to deal with priorities to not waste time, and algorithms that do it The person I am having a conversation with is advocating for threads instead of processes. How do you think threads work? > Modern approach is to use green threads which are really cheap to switch, it is like store registers, read registers and jmp. That’s certainly the popular approach. As I said at the beginning this approach is making a mini operating system with more bugs and less security rather than leveraging the capabilities of your operating system. Once again, im waiting to here about your experience of maxing out processes and after that having to switch to green threads.
    - ordu 1y ago
      > The person I am having a conversation with is advocating for threads instead of processes. How do you think threads work? Are they? I looked back and I've found this quote of them: "The overhead of process-per-request, or even thread-per-request, is absurd if you're already using a memory safe language." Doesn't seem as an advocacy for thread-per-request to me. > As I said at the beginning this approach is making a mini operating system with more bugs and less security rather than leveraging the capabilities of your operating system. Lets look at Apache for example. It starts a few processes and/or threads, but then each thread deals with a lot of connections. The threads Apache starts are for spreading work over several CPUs and maybe to overcome some limits of select/poll/epoll. The main approach is to track a state of a connection, and when something happens on a socket, Apache find the state of the connection and deals with events on the socket. Then it stores the new state and moves to deal with other sockets in the same manner. It is like green threads but without green threads. Green threads streamlines all this state keeping by allowing each connection to have it's own stack. And I'd say it is easier to do right than to write a finite automata for HTTP/HTTPS. > Once again, im waiting to here about your experience of maxing out processes and after that having to switch to green threads. Oh, I didn't. A long long time ago I was reading stuff on networking. All of it was in one opinion: 10k kernel tasks maybe a tolerable solution, but 100k is bad. IIRC Apache had a document describing its internal architecture and explaining why it is as it is. So I wouldn't even try to start thousands of threads. I mean I tried to start 1000s of processes when I was young and learned about fork-bombs, and this experience confirmed it for me, that 1000s of processes is not a really good idea. Moreover I completely agree with them: if you use a memory-safe language, then it is strange to pay costs for preemptive multitasking just to have separate virtual address spaces. I mean, it will be better to get a virtual machine with JIT compiler, and run code for different connection on different instances of a virtual machine. O(1) complexity of cooperative switching will beat O(N) complexity of preemptive switching. To my mind hardware memory management is overrated.

nosefrog 1y ago

We did that at Dropbox in Python for a while. Though they switched to async after I left.

monkeyelite 1y ago

> Handling thousands of concurrent processes is beyond most OS It works fine on Linux - the operating system for the internet. Have you tried it? > good fraction of the CPU being spent solely on context switching I was waiting for this one. Threads and processes do the same amount of context switching. The overhead of processes switch is a little higher. The main cost is memory.

ordu 1y ago
> Threads and processes do the same amount of context switching. Yes, therefore real webservers use a limited amount of threads/processes (in the same ballpark as a number of CPU cores). Modern approach is to use green threads which are really cheap to switch, it is like store registers, read registers and jmp. > The main cost is memory. The main cost is scheduling, not switching per se. Preemptive multitasking needs to deal with priorities to not waste time, and algorithms that do it are O(N) mostly. All these O(N) calculations needs to be completed multiple times per second, the higher the frequency of switching the more work to do. When you have thousands of processes it is the main cost. If you have tens of thousands it starts to bite hard.
- monkeyelite 1y ago
  > The main cost is scheduling, not switching per se. Preemptive multitasking needs to deal with priorities to not waste time, and algorithms that do it The person I am having a conversation with is advocating for threads instead of processes. How do you think threads work? > Modern approach is to use green threads which are really cheap to switch, it is like store registers, read registers and jmp. That’s certainly the popular approach. As I said at the beginning this approach is making a mini operating system with more bugs and less security rather than leveraging the capabilities of your operating system. Once again, im waiting to here about your experience of maxing out processes and after that having to switch to green threads.
  - ordu 1y ago
    > The person I am having a conversation with is advocating for threads instead of processes. How do you think threads work? Are they? I looked back and I've found this quote of them: "The overhead of process-per-request, or even thread-per-request, is absurd if you're already using a memory safe language." Doesn't seem as an advocacy for thread-per-request to me. > As I said at the beginning this approach is making a mini operating system with more bugs and less security rather than leveraging the capabilities of your operating system. Lets look at Apache for example. It starts a few processes and/or threads, but then each thread deals with a lot of connections. The threads Apache starts are for spreading work over several CPUs and maybe to overcome some limits of select/poll/epoll. The main approach is to track a state of a connection, and when something happens on a socket, Apache find the state of the connection and deals with events on the socket. Then it stores the new state and moves to deal with other sockets in the same manner. It is like green threads but without green threads. Green threads streamlines all this state keeping by allowing each connection to have it's own stack. And I'd say it is easier to do right than to write a finite automata for HTTP/HTTPS. > Once again, im waiting to here about your experience of maxing out processes and after that having to switch to green threads. Oh, I didn't. A long long time ago I was reading stuff on networking. All of it was in one opinion: 10k kernel tasks maybe a tolerable solution, but 100k is bad. IIRC Apache had a document describing its internal architecture and explaining why it is as it is. So I wouldn't even try to start thousands of threads. I mean I tried to start 1000s of processes when I was young and learned about fork-bombs, and this experience confirmed it for me, that 1000s of processes is not a really good idea. Moreover I completely agree with them: if you use a memory-safe language, then it is strange to pay costs for preemptive multitasking just to have separate virtual address spaces. I mean, it will be better to get a virtual machine with JIT compiler, and run code for different connection on different instances of a virtual machine. O(1) complexity of cooperative switching will beat O(N) complexity of preemptive switching. To my mind hardware memory management is overrated.