4 ms·
No baseline against UNIX processes?
by liontwist 2y ago
No baseline against UNIX processes?
- masklinn 2y agoA million processes?
- liontwist 2y agoI've seen 100k. What happens at a million? How many is unworkable?
- Izkata 2y ago$ cat /proc/sys/kernel/pid_max 4194304 My computer can handle that many processes, after that no new processes can be spawned (see: forkbomb)
- blitzar 2y agoI am left wondering what happens at 4194305
- masklinn 2y agoYou can fork bomb your system and observe.
- liontwist 2y agoThe fork syscall will fail at the hard limit. What’s missing here is that all these async/await, lightweight threads, etc features exist because presumably because OS processes and threads consume too many resources to feasibly serve a similar role. However nobody seems to have any hard numbers about the topic. How bad is a Linux context switch? How much memory does a process use? Suddenly everyone is a micro optimizer seeking to gain a small constant multiple. Since this is not ready at hand I suspect the rewards are much less clear. It’s more likely that the languages benefit from greater cross platform control and want to avoid cross platform inconsistencies.
- Izkata 2y agoThere's two main reasons: * gunicorn doesn't have worker scaling, they're all always running, so async workers were to not waste those resources idling while still allowing lots of simultaneous clients * The limit was 32768 not that long ago, which was at least possible to hit with a ton of clients
- liontwist 2y agoWhy would I be worried about processes using resources while blocked (not using CPU)? Virtual memory is a thing. And if it’s a web server that’s just not being used.
- Izkata 2y agoRAM, not CPU, which goes right back to this post's benchmark. Python code takes up memory too, not just data. The workers consume this memory on startup, not just while handling a request.