4 ms·

All network I/O is inherently asynchronous of course, so the sync vs async debate is more about whether to use the blocking abstraction the kernel provides vers

All network I/O is inherently asynchronous of course, so the sync vs async debate is more about whether to use the blocking abstraction the kernel provides versus bringing your own or writing async code directly. Using async I/O or a userland abstraction like green threads necessarily means you're moving the I/O scheduling work into userland. Sometimes this might be the right call, but it's effectively a bet that your userland scheduler can do a better job than the kernel's own scheduler. This bet might pay off in highly specialised cases where your userland scheduler is well tuned to your workload, but the majority of userland schedulers (to name a few examples: the Go runtime, node.js's libuv) are also aimed at general purpose workloads and in many cases are far less mature than the kernel's scheduler. There's been a massive amount of engineering work that's gone into the Linux kernel's various scheduling algorithms over the years. Pathological edge cases resulting in starvation or other performance issues have largely been identified and worked out. The various schedulers are also highly tuneable to the specifics of your workload if you need to do that. These days OS threads are a totally viable option for building highly concurrent network services on Linux. Spawning hundreds of thousands of threads works fine and is fast enough for most applications. While threads will have 8MB of virtual address space reserved for their stack by default, this is just 'on paper' memory use - no pages are actually allocated until you use them.

coder543 8y ago
> These days OS threads are a totally viable option for building highly concurrent network services on Linux. Spawning hundreds of thousands of threads works fine and is fast enough for most applications. I don't agree. If you're at the point where you need to handle such large numbers of concurrent threads, my own experience and online benchmarks clearly show it is not fast enough. Why are none of the top performers on the TechEmpower benchmark using a thread per connection? The simplest answer is that that can't or they would, since it would be a waste of time to reimplement so much in userland. More anecdotally, I have been unable to get my Fedora laptop to spawn more than a few tens of thousands of threads before the kernel tells everyone that "resource is temporarily unavailable" to any process that attempts to spawn a new thread or process until the offending process is stopped. I've googled the issue extensively, tweaked countless knobs, and come up empty handed. I'm certain the Linux kernel can spawn more threads than that, but my point still stands that it is nontrivial. In theory, that's a great perspective you presented. However, the async interfaces into the kernel exist for very real reasons. I would also challenge you (if you're bored) to write a C, C++, or Rust program that spawns 1 or 2 million threads that do nothing but sleep for a very long time, and write a similar program in Go. See how the two compare in how long it takes to launch all of the tasks, and see how much memory they use. Ideally, you would also use, say, Rust futures + tokio and spawn a million sleeping futures to compare there too, but futures and tokio aren't the most intuitive or well documented things yet, in my opinion. I've done similar tests in the past, and OS threads are not a lightweight resource.
- vbezhenar 8y ago
  You don't need million threads to be really fast. You need enough threads for your load. If you need to handle 1000 requests per second with each request taking 10 ms in average, you need 100 threads. It's absolutely adequate number of threads for OS to manage. If you would write this code with async style, you won't achieve anything, because bottleneck would be with database or another service or disk I/O. Million threads is very rare case.

coder543 8y ago

> These days OS threads are a totally viable option for building highly concurrent network services on Linux. Spawning hundreds of thousands of threads works fine and is fast enough for most applications. I don't agree. If you're at the point where you need to handle such large numbers of concurrent threads, my own experience and online benchmarks clearly show it is not fast enough. Why are none of the top performers on the TechEmpower benchmark using a thread per connection? The simplest answer is that that can't or they would, since it would be a waste of time to reimplement so much in userland. More anecdotally, I have been unable to get my Fedora laptop to spawn more than a few tens of thousands of threads before the kernel tells everyone that "resource is temporarily unavailable" to any process that attempts to spawn a new thread or process until the offending process is stopped. I've googled the issue extensively, tweaked countless knobs, and come up empty handed. I'm certain the Linux kernel can spawn more threads than that, but my point still stands that it is nontrivial. In theory, that's a great perspective you presented. However, the async interfaces into the kernel exist for very real reasons. I would also challenge you (if you're bored) to write a C, C++, or Rust program that spawns 1 or 2 million threads that do nothing but sleep for a very long time, and write a similar program in Go. See how the two compare in how long it takes to launch all of the tasks, and see how much memory they use. Ideally, you would also use, say, Rust futures + tokio and spawn a million sleeping futures to compare there too, but futures and tokio aren't the most intuitive or well documented things yet, in my opinion. I've done similar tests in the past, and OS threads are not a lightweight resource.

vbezhenar 8y ago
You don't need million threads to be really fast. You need enough threads for your load. If you need to handle 1000 requests per second with each request taking 10 ms in average, you need 100 threads. It's absolutely adequate number of threads for OS to manage. If you would write this code with async style, you won't achieve anything, because bottleneck would be with database or another service or disk I/O. Million threads is very rare case.