8 ms·

Async/Await is especially important to use in .NET and .NET Core web servers if high concurrency is desired. .NET uses a managed thread pool that will only spa

Async/Await is especially important to use in .NET and .NET Core web servers if high concurrency is desired. .NET uses a managed thread pool that will only spawn up to something like 50 threads quickly by default. If it sees that more threads are needed, it will slowly add them. This means that a Synchronous web endpoint (think API call that does not use "Async/Await") will hog an entire thread for the life of the request. I graphed the difference between Synchronous and Asynchronous API endpoints in a blog post last year, and Synchronous endpoints were far less efficient: https://caleblloyd.com/software/net-core-mvc-thread-pool-vs-async/ https://caleblloyd.com/software/net-core-mvc-thread-pool-vs-... Note that just because endpoints are declared as "async" does not automatically make them efficient - Asynchronous I/O must be used all the way to the lowest level. This can be an issue when using library code that you don't control. For example, Oracle's official MySQL package provides "async" methods, but it uses Synchronous I/O in these methods. Therefore, "async" calls into their library will really cause synchronous performance. Async/Await is a really cool feature of C#, but I wish that the language had more protection that forced users to do things "the right way", like disallowing Synchronous I/O in async methods. In order to write libraries that do I/O properly that include both Synchronous and Asynchronous methods, maintainers have to write two separate implementations of the same logic, which is a bit of a pain.

dom96 10y ago
> In order to write libraries that do I/O properly that include both Synchronous and Asynchronous methods, maintainers have to write two separate implementations of the same logic, which is a bit of a pain. A solution to this that I am particularly proud of can be found in the Nim programming language. It is a macro that transforms a single procedure into a synchronous and asynchronous procedure[1]. 1 - http://nim-lang.org/news/e027_version_0_15_0.html#multisync-macro http://nim-lang.org/news/e027_version_0_15_0.html#multisync-...
paulddraper 10y ago
> Oracle's official MySQL package provides "async" methods, but it uses Synchronous I/O in these methods. Therefore, "async" calls into their library will really cause synchronous performance. It's all about scale. If you're an Nginx server serving thousands of files concurrently, or a websocket server, you need async. But if you can spare a thread for a connection to MySQL, it's the MySQL server that's going to be in trouble, because the server already uses a thread per connection.
dhd415 10y ago
This is very true and a better explanation of the benefits of async/await than "Efficiency and Comfort" from the article. There are a lot of articles on async/await in .NET that use the example of off-loading a long-running operation from the GUI event thread. While that's a legitimate use, it obscures the true benefits which are most closely tied to throughput scalability. In other words, maintaining a relatively small pool of threads to service I/O completion events allows a machine to minimize the overhead of thread context switching that can become a bottleneck in a thread-per-operation model. Additionally, many people have the misconception that async operations are somehow faster than their sync counterparts. In fact, they're generally slightly slower, but a fully asynchronous server will be able to maintain a much higher level of throughput, usually with similar latencies, than a fully synchronous server. In other words, the principal class of applications that benefit from async/await are high-concurrency server applications. One more place I see these kinds of misconceptions is in naive benchmarks that purport to test async performance without using a concurrent workload. That tests only straight-line latency of individual requests and isn't the point of async. The benchmarks in the blog post above properly use a concurrent workload for testing.
- JamesNK 10y ago
  > In fact, they're generally slightly slower An async method is much slower (40x) compared to a normal sync method. If you're writing a high performance library you need to be careful to not repeatedly call a method decorated with async in a tight loop. Of course 40x times slower of extremely fast is still really fast but I found the overhead can add up really quickly. Good video on async/await and performance: https://channel9.msdn.com/Events/Build/BUILD2011/TOOL-829T https://channel9.msdn.com/Events/Build/BUILD2011/TOOL-829T
  - dhd415 10y ago
    Did that 40x number come from that video? In my experience, any non-trivial async operation (e.g., a 2ms database insert in a fast OLTP system) is going to be on the order of 10-20% slower in straight-line operation latency than a sync operation while handling vastly more throughput.
jsingleton 10y ago
I would argue that async/await is far more important for native applications. I've seen plenty of apps doing significant work on the UI thread and hanging. Making it easy for developers to run things in the background is great for performance. Web browsers automatically add a level of asynchronous interaction. It's also important on a web server but it won't improve performance for a single user in isolation. As you say, it allows more requests to be serviced concurrently. This won't speed up an individual request unless the server is already heavily loaded. Edit: You don't need to build two library APIs. You can just build an Async method as the default implementation then have a blocking call to that as the synchronous method. I believe that all new MS APIs are now Async by default.
- recursive 10y ago
  Async/Await does not imply multiple threads. Often, the async version will use fewer threads than the equivalent synchronous code. http://blog.stephencleary.com/2013/11/there-is-no-thread.html http://blog.stephencleary.com/2013/11/there-is-no-thread.htm...
- dhd415 10y ago
  >>I would argue that async/await is far more important for native applications. I've seen plenty of apps doing significant work on the UI thread and hanging. Making it easy for developers to run things in the background is great for performance. Web browsers automatically add a level of asynchronous interaction.<< You can achieve the same thing in a GUI app by simply off-loading the long-running operation to a separate thread. The same is _not_ true for a highly concurrent, I/O-heavy server workload. That's where async/await is necessary and not just a syntactic convenience.
- caleblloyd 10y ago
  > You don't need to build two library APIs. You can just build an Async method as the default implementation then have a blocking call to that as the synchronous method This can lead to a nasty problem: Thread Pool Starvation. I contribute to MySqlConnector, an MIT Licensed MySql driver for .NET that implements truly Asynchronous I/O. We did this at first, but had to do a large refactor because we hit Thread Pool Starvation in our high concurrency performance tests. Here's the original issue: https://github.com/mysql-net/MySqlConnector/issues/62 https://github.com/mysql-net/MySqlConnector/issues/62 Stephen Toub has a pretty good article explaining why you shouldn't do "Sync over Async": https://blogs.msdn.microsoft.com/pfxteam/2012/04/13/should-i-expose-synchronous-wrappers-for-asynchronous-methods/ https://blogs.msdn.microsoft.com/pfxteam/2012/04/13/should-i...
jameshart 10y ago
It's also worth considering why .NET is conservative about spawning threads. The basic reason is that threads are really big objects. The default stack space allocation for a 64bit thread in .NET is 4MB, so you do not want hundreds of these things getting spawned when your traffic spikes. Since most of your threads handling requests would be spending most of their time waiting for IO operations (network requests to DB servers being the most common wait state) to complete, this is a recipe for having huge blocks of committed memory sitting around, each holding a small amount of stack state for a single request (most of the request data being strings and dictionaries will be on the managed heap anyway, the stack will just contain a few references to those heap objects), waiting for a DB query to complete. A web server using few threads and async/await state machines can transfer those stack states out into managed heap objects too, and handle many times more concurrent requests without running out of RAM.
- balfirevic 10y ago
  Aync/await just seems like really heavy handed solution to this problem. I wish .Net runtime provided lightweight threads (with low memory and switching overhead) so that it would be possible to tune how application runs without needing to modify the code.
  - platz 10y ago
    what language does "lightweight threads" exactly like this?
    - int_19h 10y ago
      Erlang?
    - balfirevic 10y ago
      I think JVM with Quasar gets close. It uses bytecode instrumentation to support suspending and resuming method execution. It modifies methods so that they know how to store and retrieve their stack state on heap when they are suspended and resumed. I think the Quasar API is not exactly what I described in my previous comment but the above building blocks are what is needed to achieve something close to it.

dom96 10y ago

> In order to write libraries that do I/O properly that include both Synchronous and Asynchronous methods, maintainers have to write two separate implementations of the same logic, which is a bit of a pain. A solution to this that I am particularly proud of can be found in the Nim programming language. It is a macro that transforms a single procedure into a synchronous and asynchronous procedure[1]. 1 - http://nim-lang.org/news/e027_version_0_15_0.html#multisync-macro http://nim-lang.org/news/e027_version_0_15_0.html#multisync-...

paulddraper 10y ago

> Oracle's official MySQL package provides "async" methods, but it uses Synchronous I/O in these methods. Therefore, "async" calls into their library will really cause synchronous performance. It's all about scale. If you're an Nginx server serving thousands of files concurrently, or a websocket server, you need async. But if you can spare a thread for a connection to MySQL, it's the MySQL server that's going to be in trouble, because the server already uses a thread per connection.

dhd415 10y ago

This is very true and a better explanation of the benefits of async/await than "Efficiency and Comfort" from the article. There are a lot of articles on async/await in .NET that use the example of off-loading a long-running operation from the GUI event thread. While that's a legitimate use, it obscures the true benefits which are most closely tied to throughput scalability. In other words, maintaining a relatively small pool of threads to service I/O completion events allows a machine to minimize the overhead of thread context switching that can become a bottleneck in a thread-per-operation model. Additionally, many people have the misconception that async operations are somehow faster than their sync counterparts. In fact, they're generally slightly slower, but a fully asynchronous server will be able to maintain a much higher level of throughput, usually with similar latencies, than a fully synchronous server. In other words, the principal class of applications that benefit from async/await are high-concurrency server applications. One more place I see these kinds of misconceptions is in naive benchmarks that purport to test async performance without using a concurrent workload. That tests only straight-line latency of individual requests and isn't the point of async. The benchmarks in the blog post above properly use a concurrent workload for testing.

JamesNK 10y ago
> In fact, they're generally slightly slower An async method is much slower (40x) compared to a normal sync method. If you're writing a high performance library you need to be careful to not repeatedly call a method decorated with async in a tight loop. Of course 40x times slower of extremely fast is still really fast but I found the overhead can add up really quickly. Good video on async/await and performance: https://channel9.msdn.com/Events/Build/BUILD2011/TOOL-829T https://channel9.msdn.com/Events/Build/BUILD2011/TOOL-829T
- dhd415 10y ago
  Did that 40x number come from that video? In my experience, any non-trivial async operation (e.g., a 2ms database insert in a fast OLTP system) is going to be on the order of 10-20% slower in straight-line operation latency than a sync operation while handling vastly more throughput.

jsingleton 10y ago

I would argue that async/await is far more important for native applications. I've seen plenty of apps doing significant work on the UI thread and hanging. Making it easy for developers to run things in the background is great for performance. Web browsers automatically add a level of asynchronous interaction. It's also important on a web server but it won't improve performance for a single user in isolation. As you say, it allows more requests to be serviced concurrently. This won't speed up an individual request unless the server is already heavily loaded. Edit: You don't need to build two library APIs. You can just build an Async method as the default implementation then have a blocking call to that as the synchronous method. I believe that all new MS APIs are now Async by default.

recursive 10y ago
Async/Await does not imply multiple threads. Often, the async version will use fewer threads than the equivalent synchronous code. http://blog.stephencleary.com/2013/11/there-is-no-thread.html http://blog.stephencleary.com/2013/11/there-is-no-thread.htm...
dhd415 10y ago
>>I would argue that async/await is far more important for native applications. I've seen plenty of apps doing significant work on the UI thread and hanging. Making it easy for developers to run things in the background is great for performance. Web browsers automatically add a level of asynchronous interaction.<< You can achieve the same thing in a GUI app by simply off-loading the long-running operation to a separate thread. The same is _not_ true for a highly concurrent, I/O-heavy server workload. That's where async/await is necessary and not just a syntactic convenience.
caleblloyd 10y ago
> You don't need to build two library APIs. You can just build an Async method as the default implementation then have a blocking call to that as the synchronous method This can lead to a nasty problem: Thread Pool Starvation. I contribute to MySqlConnector, an MIT Licensed MySql driver for .NET that implements truly Asynchronous I/O. We did this at first, but had to do a large refactor because we hit Thread Pool Starvation in our high concurrency performance tests. Here's the original issue: https://github.com/mysql-net/MySqlConnector/issues/62 https://github.com/mysql-net/MySqlConnector/issues/62 Stephen Toub has a pretty good article explaining why you shouldn't do "Sync over Async": https://blogs.msdn.microsoft.com/pfxteam/2012/04/13/should-i-expose-synchronous-wrappers-for-asynchronous-methods/ https://blogs.msdn.microsoft.com/pfxteam/2012/04/13/should-i...

jameshart 10y ago

It's also worth considering why .NET is conservative about spawning threads. The basic reason is that threads are really big objects. The default stack space allocation for a 64bit thread in .NET is 4MB, so you do not want hundreds of these things getting spawned when your traffic spikes. Since most of your threads handling requests would be spending most of their time waiting for IO operations (network requests to DB servers being the most common wait state) to complete, this is a recipe for having huge blocks of committed memory sitting around, each holding a small amount of stack state for a single request (most of the request data being strings and dictionaries will be on the managed heap anyway, the stack will just contain a few references to those heap objects), waiting for a DB query to complete. A web server using few threads and async/await state machines can transfer those stack states out into managed heap objects too, and handle many times more concurrent requests without running out of RAM.

balfirevic 10y ago
Aync/await just seems like really heavy handed solution to this problem. I wish .Net runtime provided lightweight threads (with low memory and switching overhead) so that it would be possible to tune how application runs without needing to modify the code.
- platz 10y ago
  what language does "lightweight threads" exactly like this?
  - int_19h 10y ago
    Erlang?
  - balfirevic 10y ago
    I think JVM with Quasar gets close. It uses bytecode instrumentation to support suspending and resuming method execution. It modifies methods so that they know how to store and retrieve their stack state on heap when they are suspended and resumed. I think the Quasar API is not exactly what I described in my previous comment but the above building blocks are what is needed to achieve something close to it.