3 ms·
> without work stealing where heavy requests slow everything else down Right because threads vs async doesn't make any sense. You use both, async is your defau
by Spivak 3y ago
> without work stealing where heavy requests slow everything else down
Right because threads vs async doesn't make any sense. You use both, async is your default, free yourself up during io, and is basically the developer friendly abstraction on top of epoll, and then use threads for stuff you actually need threads for.
- fiddlerwoaroof 3y agoI don’t understand this: context switching takes microseconds, I/O latencies are typically in the millisecond range. I’d think thread overhead would be negligible in an I/O-bound application (especially if you take steps to reduce the amount of memory per thread)
- jandrewrogers 3y agoThe important distinction is between operation latency and operation rate. Modern I/O devices are highly concurrent and support massive throughput. A device can have millisecond latency while still executing an operation every microsecond. In these cases, the operation latency doesn't matter, your thread has to handle events at the rate the device executes operations. If it is a million operations per second then from the perspective of the thread you have a microsecond to handle each operation. Context switch throughput is much lower by comparison. In these types of systems, you may issue a thousand concurrent I/O operations before the first I/O operation returns a result. Threads don't wait for the first operation to finish, they keep a deep pipeline of I/O operations in flight concurrently so that they always have something to do.
- fiddlerwoaroof 3y ago> Modern I/O devices are highly concurrent and support massive throughput. A device can have millisecond latency while still executing an operation every microsecond. In these cases, the operation latency doesn't matter, your thread has to handle events at the rate the device executes operations. This is true for some applications, like an OLAP database or similar. It’s not true for the typical user-facing app where you want to finish requests as soon as possible because a user is waiting and every millisecond costs you money.
- laurencerowe 3y agoMixing the two is absolutely the right approach in general. But I still find myself wishing it were easier to build a very simple threaded web server in Rust so I could play around with stuff like MSG_ZEROCOPY [1]. Last time I looked there were no non-async http libraries in cargo other than a few toy servers. [1] https://lwn.net/Articles/726917/ https://lwn.net/Articles/726917/