3 ms·

A lot of the time, the benefit can be from much more controlled scheduling without having to introduce actual threads. A lot of software doesn't want to take th

by slaymaker1907 3y ago

A lot of the time, the benefit can be from much more controlled scheduling without having to introduce actual threads. A lot of software doesn't want to take the overhead of creating platform threads (memory and taking additional CPU time) but does want to do scheduling internally for various CPU bound workloads. I think web pages are often like this. You don't really want to hog tons of CPU, but you also want to have guarantees like not blocking rendering when running a user provided regex since said regex could take a long time to run, but in practice that regex will be fast. Even if we want to eventually offload that to a separate thread, async gives us the option to first try evaluating it in the main thread for some number of iterations before offloading it. GPU/FPGA programming is kind of irrelevant because as expensive as developing high performance code is for CPUs, costs go up by an order of magnitude for those platforms unless there is some existing library you can utilize (mainly applicable for ML/AI). These platforms are also very expensive. It's like saying just rent a helicopter if you need to get somewhere quickly instead of asking what "quickly" actually means and optimizing accordingly.

exfalso 3y ago
Suspendable computations like regex statemachines are actually a good example use case. However I'd still categorize this as extremely marginal. I have seen literally zero Rust crates that handle suspendable computations (ofc outside of the scheduler runtime). Also note that by their nature regex statemachines do not actually necessitate the use of async, it's effectively syntax sugar on top. My point with GPU/FPGA is that if you're at the point where this level of optimization matters then you're actually dealing with situations where it's worth to invest in the big boy tools. Examples are HPC in fintech (low latency trading) and scientific computations, game development, video codecs etc. You know, "actual" computations. Webservices are not in this category. Generally speaking with web services your goal is to "hide in the shadow of IO". If you max out your network and/or database and/or filesystem capacity, further CPU optimizations will have literally no effect. I have yet to encounter a web service where this wasn't the case. What I do see sometimes with webservices is simply unnecessary compute, bad internal structuring, dynamic dispatch, parallelism overcommit, lack of batching, fragmented apis, fragmented data accesses etc etc all of which appear as CPU capacity saturation and also sometimes as kernelspace overhead in profiles. Async does not help solving any of these issues. And once you do solve them, you reached IO boundness and it doesn't matter anymore. Again this is just my experience, and I'm happy to learn about what kind of web service can utilize coroutines with measurable performance benefits over a managed threadpool.

exfalso 3y ago

Suspendable computations like regex statemachines are actually a good example use case. However I'd still categorize this as extremely marginal. I have seen literally zero Rust crates that handle suspendable computations (ofc outside of the scheduler runtime). Also note that by their nature regex statemachines do not actually necessitate the use of async, it's effectively syntax sugar on top. My point with GPU/FPGA is that if you're at the point where this level of optimization matters then you're actually dealing with situations where it's worth to invest in the big boy tools. Examples are HPC in fintech (low latency trading) and scientific computations, game development, video codecs etc. You know, "actual" computations. Webservices are not in this category. Generally speaking with web services your goal is to "hide in the shadow of IO". If you max out your network and/or database and/or filesystem capacity, further CPU optimizations will have literally no effect. I have yet to encounter a web service where this wasn't the case. What I do see sometimes with webservices is simply unnecessary compute, bad internal structuring, dynamic dispatch, parallelism overcommit, lack of batching, fragmented apis, fragmented data accesses etc etc all of which appear as CPU capacity saturation and also sometimes as kernelspace overhead in profiles. Async does not help solving any of these issues. And once you do solve them, you reached IO boundness and it doesn't matter anymore. Again this is just my experience, and I'm happy to learn about what kind of web service can utilize coroutines with measurable performance benefits over a managed threadpool.