3 ms·

From the author's previous piece: https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/ https://www.scylladb.com/20

From the author's previous piece: https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-will-revolutionize-programming-in-linux/ https://www.scylladb.com/2020/05/05/how-io_uring-and-ebpf-wi... > Our CTO, Avi Kivity, made the case for async at the Core C++ 2019 event. The bottom line is this; in modern multicore, multi-CPU devices, the CPU itself is now basically a network, the intercommunication between all the CPUs is another network, and calls to disk I/O are effectively another. There are good reasons why network programming is done asynchronously, and you should consider that for your own application development too. > > It fundamentally changes the way Linux applications are to be designed: Instead of a flow of code that issues syscalls when needed, that have to think about whether or not a file is ready, they naturally become an event-loop that constantly add things to a shared buffer, deals with the previous entries that completed, rinse, repeat. As someone that's been working on FRP related things for a while now, this feels very vindicating. :) I few like as recently as a few years ago, the systems world was content with it's incremental hacks, but now the gap between the traditional interfaces and hardware realities has become too much, and bigger redesigning is afoot. Excited for what emerges!

mehrdadn 6y ago
> in modern multicore, multi-CPU devices, the CPU itself is now basically a network, the intercommunication between all the CPUs is another network, and calls to disk I/O are effectively another. Interesting take, and NUMA CPUs have felt networked to me when I've used them, but typical multicore UMA CPUs sure haven't... is there a reason to believe this will change (or already has), or did the author mean to only talk about NUMA?
- ddorian43 6y ago
  Both, even multicore. Same ideas are used by Red Panda (open source kafka++ clone).
- jeffbee 6y ago
  I guess it depends on what you consider to be "typical". If you look at a many-core chip from AMD you'll see a gradient of access latency from one core to another. You'll see the same on any Intel Skylake-X descendant, although the slope of that gradient is less. Your software will need to be very highly optimized already before you start to sweat the difference, though.
  - masklinn 6y ago
    Here's what jeffbee is talking about: https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-dive-review-5950x-5900x-5800x-and-5700x-tested/5 https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-di...
    - jeffbee 6y ago
      Yep. If you are sweating a microsecond, 100 nanoseconds is significant chunk of your budget. For this and possibly other reasons, a many-core CPU isn't always a great choice for hosted storage. If your goal is to export NVMe blocks over a network interface, you might be better off with an easier-to-program 4- or 8-core CPU. I don't like seeing 128 cores and a bunch of NVMe devices in the same box because it just causes trouble.
      - magicalhippo 6y ago
        With p2pdma[1], it seems you don't even need a fancy CPU to push GB/s. [1]: https://www.youtube.com/watch?v=LDOlqgUZtHE https://www.youtube.com/watch?v=LDOlqgUZtHE (9:40)
hinkley 6y ago
> that have to think about whether or not a file is ready, they naturally become an event-loop that constantly add things to a shared buffer, deals with the previous entries that completed, rinse, repeat. This is the exception, not the rule, and it bugs me when APIs default to this. Most consumers of data are not looking at the stream of data, and in many cases where streaming is what I want, there are tools and APIs for handling that outside of my application logic. Much of the time I’m dealing with units of data only after the entire unit has arrived. Because if the message is not complete there is no forward progress to be made. My tools should reflect that reality, not what’s quickest for the API writers to create. In fact, if I remember my queuing theory properly, responsiveness is improved if the system prioritizes IO operations that can be finished (eg, EOS, EOF) over processing buffers for one that is still in the middle, which can’t happen with an event stream abstraction.
pjmlp 6y ago
The problem is forcing everyone to program in asynchronous way. WinRT tried to go down that route (only asynchronous APIs), to drive developers into that path, but eventually they had to support synchronous as well due to the received resistence.
- justicezyx 6y ago
  Before Herb Sutter's free lunch is over, not many writes in multi-threading code. Then today, everyone is writing multi-threading code, one way or another; and majority indirectly in newer languages like Go and Rust, or better frameworks like Actors, message passing, coroutines, and some noble souls who are capable enough, in classic pthread and other threading APIs. Of course everyone is going to program in an async way; that's how nature works. But it certainly will not be in a fashinon that is repulsive to you, just give it some time. Maybe 10 years.
  - pjmlp 6y ago
    Agreed, it is also why Microsoft was one of the biggest contributors to the co-routines support in C++, and have frameworks like Orleans and Coyote.

mehrdadn 6y ago

> in modern multicore, multi-CPU devices, the CPU itself is now basically a network, the intercommunication between all the CPUs is another network, and calls to disk I/O are effectively another. Interesting take, and NUMA CPUs have felt networked to me when I've used them, but typical multicore UMA CPUs sure haven't... is there a reason to believe this will change (or already has), or did the author mean to only talk about NUMA?

ddorian43 6y ago
Both, even multicore. Same ideas are used by Red Panda (open source kafka++ clone).
jeffbee 6y ago
I guess it depends on what you consider to be "typical". If you look at a many-core chip from AMD you'll see a gradient of access latency from one core to another. You'll see the same on any Intel Skylake-X descendant, although the slope of that gradient is less. Your software will need to be very highly optimized already before you start to sweat the difference, though.
- masklinn 6y ago
  Here's what jeffbee is talking about: https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-dive-review-5950x-5900x-5800x-and-5700x-tested/5 https://www.anandtech.com/show/16214/amd-zen-3-ryzen-deep-di...
  - jeffbee 6y ago
    Yep. If you are sweating a microsecond, 100 nanoseconds is significant chunk of your budget. For this and possibly other reasons, a many-core CPU isn't always a great choice for hosted storage. If your goal is to export NVMe blocks over a network interface, you might be better off with an easier-to-program 4- or 8-core CPU. I don't like seeing 128 cores and a bunch of NVMe devices in the same box because it just causes trouble.
    - magicalhippo 6y ago
      With p2pdma[1], it seems you don't even need a fancy CPU to push GB/s. [1]: https://www.youtube.com/watch?v=LDOlqgUZtHE https://www.youtube.com/watch?v=LDOlqgUZtHE (9:40)

hinkley 6y ago

> that have to think about whether or not a file is ready, they naturally become an event-loop that constantly add things to a shared buffer, deals with the previous entries that completed, rinse, repeat. This is the exception, not the rule, and it bugs me when APIs default to this. Most consumers of data are not looking at the stream of data, and in many cases where streaming is what I want, there are tools and APIs for handling that outside of my application logic. Much of the time I’m dealing with units of data only after the entire unit has arrived. Because if the message is not complete there is no forward progress to be made. My tools should reflect that reality, not what’s quickest for the API writers to create. In fact, if I remember my queuing theory properly, responsiveness is improved if the system prioritizes IO operations that can be finished (eg, EOS, EOF) over processing buffers for one that is still in the middle, which can’t happen with an event stream abstraction.

pjmlp 6y ago

The problem is forcing everyone to program in asynchronous way. WinRT tried to go down that route (only asynchronous APIs), to drive developers into that path, but eventually they had to support synchronous as well due to the received resistence.

justicezyx 6y ago
Before Herb Sutter's free lunch is over, not many writes in multi-threading code. Then today, everyone is writing multi-threading code, one way or another; and majority indirectly in newer languages like Go and Rust, or better frameworks like Actors, message passing, coroutines, and some noble souls who are capable enough, in classic pthread and other threading APIs. Of course everyone is going to program in an async way; that's how nature works. But it certainly will not be in a fashinon that is repulsive to you, just give it some time. Maybe 10 years.
- pjmlp 6y ago
  Agreed, it is also why Microsoft was one of the biggest contributors to the co-routines support in C++, and have frameworks like Orleans and Coyote.