3 ms·

> and it's synchronous, so it's really slow. > it's asynchronous (so it's really fast) Whether something is 'synchronous' or 'asynchronous' has absolutely no

> and it's synchronous, so it's really slow. > it's asynchronous (so it's really fast) Whether something is 'synchronous' or 'asynchronous' has absolutely no bearing on performance.

coder543 8y ago
I don't understand the point you're trying to make. In networked stuff, the (a)synchronicity absolutely does make something fast or slow. Suppose you have a synchronous web server with 4 threads, synchronous naturally means that each thread can only handle one request at a time. Each request can take dozens or hundreds of milliseconds to complete, just by being bottlenecked on latency. If it takes 100ms to complete each request, that server can only handle 40 requests per second. If that server were asynchronous, each thread could handle thousands of simultaneous connections, making the performance literally thousands of times better at a minimum. If you suggest spinning up an unlimited number of OS threads, that's quickly going to run into problems because most OSes can only handle a couple of thousand threads before you start running into OS limits that you have to adjust, knobs for which aren't always easy to find, and even then, each thread takes a significant amount of time to start and stop, as well as much larger amounts of RAM. An alternative solution is green threading, which Go calls Goroutines, for example. In such a system, the asynchronous operations are handled by the underlying runtime and your code can treat them as synchronous and just spawn new "threads". If you disagree, I would love to hear a more detailed response, because I can't see how your claim could be true.
- charliesome 8y ago
  All network I/O is inherently asynchronous of course, so the sync vs async debate is more about whether to use the blocking abstraction the kernel provides versus bringing your own or writing async code directly. Using async I/O or a userland abstraction like green threads necessarily means you're moving the I/O scheduling work into userland. Sometimes this might be the right call, but it's effectively a bet that your userland scheduler can do a better job than the kernel's own scheduler. This bet might pay off in highly specialised cases where your userland scheduler is well tuned to your workload, but the majority of userland schedulers (to name a few examples: the Go runtime, node.js's libuv) are also aimed at general purpose workloads and in many cases are far less mature than the kernel's scheduler. There's been a massive amount of engineering work that's gone into the Linux kernel's various scheduling algorithms over the years. Pathological edge cases resulting in starvation or other performance issues have largely been identified and worked out. The various schedulers are also highly tuneable to the specifics of your workload if you need to do that. These days OS threads are a totally viable option for building highly concurrent network services on Linux. Spawning hundreds of thousands of threads works fine and is fast enough for most applications. While threads will have 8MB of virtual address space reserved for their stack by default, this is just 'on paper' memory use - no pages are actually allocated until you use them.
  - coder543 8y ago
    > These days OS threads are a totally viable option for building highly concurrent network services on Linux. Spawning hundreds of thousands of threads works fine and is fast enough for most applications. I don't agree. If you're at the point where you need to handle such large numbers of concurrent threads, my own experience and online benchmarks clearly show it is not fast enough. Why are none of the top performers on the TechEmpower benchmark using a thread per connection? The simplest answer is that that can't or they would, since it would be a waste of time to reimplement so much in userland. More anecdotally, I have been unable to get my Fedora laptop to spawn more than a few tens of thousands of threads before the kernel tells everyone that "resource is temporarily unavailable" to any process that attempts to spawn a new thread or process until the offending process is stopped. I've googled the issue extensively, tweaked countless knobs, and come up empty handed. I'm certain the Linux kernel can spawn more threads than that, but my point still stands that it is nontrivial. In theory, that's a great perspective you presented. However, the async interfaces into the kernel exist for very real reasons. I would also challenge you (if you're bored) to write a C, C++, or Rust program that spawns 1 or 2 million threads that do nothing but sleep for a very long time, and write a similar program in Go. See how the two compare in how long it takes to launch all of the tasks, and see how much memory they use. Ideally, you would also use, say, Rust futures + tokio and spawn a million sleeping futures to compare there too, but futures and tokio aren't the most intuitive or well documented things yet, in my opinion. I've done similar tests in the past, and OS threads are not a lightweight resource.
    - vbezhenar 8y ago
      You don't need million threads to be really fast. You need enough threads for your load. If you need to handle 1000 requests per second with each request taking 10 ms in average, you need 100 threads. It's absolutely adequate number of threads for OS to manage. If you would write this code with async style, you won't achieve anything, because bottleneck would be with database or another service or disk I/O. Million threads is very rare case.

> and it's synchronous, so it's really slow. > it's asynchronous (so it's really fast) Whether something is 'synchronous' or 'asynchronous' has absolutely no bearing on performance.

coder543 8y ago

I don't understand the point you're trying to make. In networked stuff, the (a)synchronicity absolutely does make something fast or slow. Suppose you have a synchronous web server with 4 threads, synchronous naturally means that each thread can only handle one request at a time. Each request can take dozens or hundreds of milliseconds to complete, just by being bottlenecked on latency. If it takes 100ms to complete each request, that server can only handle 40 requests per second. If that server were asynchronous, each thread could handle thousands of simultaneous connections, making the performance literally thousands of times better at a minimum. If you suggest spinning up an unlimited number of OS threads, that's quickly going to run into problems because most OSes can only handle a couple of thousand threads before you start running into OS limits that you have to adjust, knobs for which aren't always easy to find, and even then, each thread takes a significant amount of time to start and stop, as well as much larger amounts of RAM. An alternative solution is green threading, which Go calls Goroutines, for example. In such a system, the asynchronous operations are handled by the underlying runtime and your code can treat them as synchronous and just spawn new "threads". If you disagree, I would love to hear a more detailed response, because I can't see how your claim could be true.

charliesome 8y ago
All network I/O is inherently asynchronous of course, so the sync vs async debate is more about whether to use the blocking abstraction the kernel provides versus bringing your own or writing async code directly. Using async I/O or a userland abstraction like green threads necessarily means you're moving the I/O scheduling work into userland. Sometimes this might be the right call, but it's effectively a bet that your userland scheduler can do a better job than the kernel's own scheduler. This bet might pay off in highly specialised cases where your userland scheduler is well tuned to your workload, but the majority of userland schedulers (to name a few examples: the Go runtime, node.js's libuv) are also aimed at general purpose workloads and in many cases are far less mature than the kernel's scheduler. There's been a massive amount of engineering work that's gone into the Linux kernel's various scheduling algorithms over the years. Pathological edge cases resulting in starvation or other performance issues have largely been identified and worked out. The various schedulers are also highly tuneable to the specifics of your workload if you need to do that. These days OS threads are a totally viable option for building highly concurrent network services on Linux. Spawning hundreds of thousands of threads works fine and is fast enough for most applications. While threads will have 8MB of virtual address space reserved for their stack by default, this is just 'on paper' memory use - no pages are actually allocated until you use them.
- coder543 8y ago
  > These days OS threads are a totally viable option for building highly concurrent network services on Linux. Spawning hundreds of thousands of threads works fine and is fast enough for most applications. I don't agree. If you're at the point where you need to handle such large numbers of concurrent threads, my own experience and online benchmarks clearly show it is not fast enough. Why are none of the top performers on the TechEmpower benchmark using a thread per connection? The simplest answer is that that can't or they would, since it would be a waste of time to reimplement so much in userland. More anecdotally, I have been unable to get my Fedora laptop to spawn more than a few tens of thousands of threads before the kernel tells everyone that "resource is temporarily unavailable" to any process that attempts to spawn a new thread or process until the offending process is stopped. I've googled the issue extensively, tweaked countless knobs, and come up empty handed. I'm certain the Linux kernel can spawn more threads than that, but my point still stands that it is nontrivial. In theory, that's a great perspective you presented. However, the async interfaces into the kernel exist for very real reasons. I would also challenge you (if you're bored) to write a C, C++, or Rust program that spawns 1 or 2 million threads that do nothing but sleep for a very long time, and write a similar program in Go. See how the two compare in how long it takes to launch all of the tasks, and see how much memory they use. Ideally, you would also use, say, Rust futures + tokio and spawn a million sleeping futures to compare there too, but futures and tokio aren't the most intuitive or well documented things yet, in my opinion. I've done similar tests in the past, and OS threads are not a lightweight resource.
  - vbezhenar 8y ago
    You don't need million threads to be really fast. You need enough threads for your load. If you need to handle 1000 requests per second with each request taking 10 ms in average, you need 100 threads. It's absolutely adequate number of threads for OS to manage. If you would write this code with async style, you won't achieve anything, because bottleneck would be with database or another service or disk I/O. Million threads is very rare case.