3 ms·

In theory there is nothing wrong with spawning tens/hundreds of thousands OS threads, and Linux does a decent job at this. But in practise you'll run into issue

In theory there is nothing wrong with spawning tens/hundreds of thousands OS threads, and Linux does a decent job at this. But in practise you'll run into issues such as: The other big problem on Linux is stack sizes - "hundreds" or "thousands" of threads isn't a problem - the C10K problem which proved the dominance of nginx's evented model vs. Apache thread-per-conncetion model was broken 20 years ago. Where it matters is when we reach 100s of thousands or millions of connections on a single node. The first issue you run into is the thread stack size - Linux has a hard default of 8MB; at 100,000 you start to hit OOM issues first. Go (and BEAM) has a ton of work to make this use case possible within the runtime. What I'm unsure about if Linux even cares about solving this problem - the threading model is pretty general and I'm not sure how important high performance connection handling is. At the end of the day, if you are really I/O bound, an async style, co-operative, event queue will always be more performant than a generic threaded scheduler like the one Linux has; meaning if you care about performance (like Django probably does) they would probably go with an async style scheduler even if Linux could support a million threads.

YorickPeterse 6y ago
> The first issue you run into is the thread stack size - Linux has a hard default of 8MB; at 100,000 you start to hit OOM issues first. Go (and BEAM) has a ton of work to make this use case possible within the runtime. This is not how thread stack sizes work. Thread stack sizes allocate a certain amount of _virtual_ memory. This means that with a stack size of 8MB you don't actually immediately use 8MB of physical memory. Only when you start filling up that memory do you start using physical memory. We can easily demonstrate this using a simple Rust program: use std::thread; use std::time::Duration; fn main() { let mut handles = Vec::with_capacity(10_000); for _ in 0..handles.capacity() { let handle = thread::spawn(|| thread::sleep(Duration::from_secs(1))); handles.push(handle); } for handle in handles { handle.join().unwrap(); } } We when compile this in release mode, and run it as follows: /usr/bin/time -f '%M KB RSS' program-name On my Linux desktop this prints out: 88492 KB RSS This is roughly 86 MB or RSS memory being used. If the thread stacks would use physical memory, this would translate to just under 80 GB; and the program would have been OOM killed because I only have 16 GB of RAM available. The virtual memory limits in turn are not much of an issue. I think on most 64 bits systems the virtual memory limit is 256 TB. A 256 TB limit would allow up to 33 554 432 OS threads with a stack size of 8MB. And since you can configure the limit when spawning threads, you can increase this number by spawning threads with a smaller limit (e.g. 4MB).

YorickPeterse 6y ago

> The first issue you run into is the thread stack size - Linux has a hard default of 8MB; at 100,000 you start to hit OOM issues first. Go (and BEAM) has a ton of work to make this use case possible within the runtime. This is not how thread stack sizes work. Thread stack sizes allocate a certain amount of _virtual_ memory. This means that with a stack size of 8MB you don't actually immediately use 8MB of physical memory. Only when you start filling up that memory do you start using physical memory. We can easily demonstrate this using a simple Rust program: use std::thread; use std::time::Duration; fn main() { let mut handles = Vec::with_capacity(10_000); for _ in 0..handles.capacity() { let handle = thread::spawn(|| thread::sleep(Duration::from_secs(1))); handles.push(handle); } for handle in handles { handle.join().unwrap(); } } We when compile this in release mode, and run it as follows: /usr/bin/time -f '%M KB RSS' program-name On my Linux desktop this prints out: 88492 KB RSS This is roughly 86 MB or RSS memory being used. If the thread stacks would use physical memory, this would translate to just under 80 GB; and the program would have been OOM killed because I only have 16 GB of RAM available. The virtual memory limits in turn are not much of an issue. I think on most 64 bits systems the virtual memory limit is 256 TB. A 256 TB limit would allow up to 33 554 432 OS threads with a stack size of 8MB. And since you can configure the limit when spawning threads, you can increase this number by spawning threads with a smaller limit (e.g. 4MB).