5 ms·

This is billed as something revolutionary and forward progress but that’s exactly how we used to do async in $lang - including Rust - before language support fo

This is billed as something revolutionary and forward progress but that’s exactly how we used to do async in $lang - including Rust - before language support for async/await landed. The biggest productivity boost to my rust embedded firmware development was when I could stop manually implementing state machines and marshalling all local variables into custom state after custom state between each I/O operation snd let rust do that for me by using async/await syntax! That’s, after all, what async desugars to in rust: an automatic state machine that saves values across I/O (await) points for you.

wh33zle 2y ago
I tried to address this at the end of the post: If what you are implementing is mostly _sequential_ IO operations, then this model becomes a bit painful. That isn't always the case though. In more packet-oriented usecases (QUIC, WebRTC & IP), doing the actual IO bit is easy: send & receive individual packets / datagrams. There isn't really much the compiler can generate for you because you don't end up with many `.await` points. At the same time, the state management across all these futures becomes spaghetti code because many of these aspects should run concurrently and thus need to be in their own future / task.
k_bx 2y ago
Yep. The only things about async that bothers me is the need to write ".await" everywhere. I wish there'd be a way to inverse this, and actually just run ".await" by default, while having a special construct not to.
- mdtusz 2y ago
  You mean `.await`, I assume?
  - k_bx 2y ago
    Thanks, didn't have my coffee yet :)
  - sirdvd 2y ago
    /s https://xkcd.com/2954/ https://xkcd.com/2954/
- vlmutolo 2y ago
  It’s important to be able to see where the async function might pause execution. For example, if you’re holding a mutex lock, you probably want to avoid holding it “across” an await point so that it’s not locked for longer than necessary if the function is paused.
  - Arnavion 2y ago
    I agree that the explicit yield syntax is good. To play devil's advocate though, the case of Mutex specifically has it going for it that MutexGuard is !Send, so it's a compiler error if a MutexGuard is held across an await point in a Send Future. But yes if your Future is also !Send then the compiler will allow it. In that case, your only recourse is that clippy has lints for holding Mutex and RefCell guards across await points, as long as you're running it and paying attention to it of course.
  - k_bx 2y ago
    I disagree. It should be a compiler warning, maybe a "clippy" one, in such cases. Btw the problem of sync code blocking async code is very real and also needs to be resolved, adding explicit `.blocking` to every blocking call is just as bad as explicit .await at every line. Also, I like Haskell approach of being able to introduce syntax extensions at a file-level, so that for code that'd benefit from explicit await – I'd rather let author have it explicit.
  - PenguinCoder 2y ago
    > It’s important to be able to see where the async function might pause execution. Why? Your example does not prove the point.
Aissen 2y ago
It is not billed as revolutionary. From the article: > This pattern isn't something that we invented! The Python world even has a dedicated website about it. And yet it is too common to find protocol libraries doing I/O in the wild :-(
PaulHoule 2y ago
It was how we did I/O in assembly language in the 1980s. How else would you write an interrupt-driven YMODEM implementation?
tel 2y ago
I don't think that's quite true. The lift here is that the state machine does not do any IO on its own. It always delegates that work to the event loop that's hosting it, which allows it to be interpreted in different contexts. That makes it more testable and more composable as it makes fewer assumptions about the runtime environment. Theoretically, you could do the same thing with async/await constructing the state machines for you, although in practice it's pretty painful and most async/await code is impure. There are lots of more experimental languages which exceptional support for this style of programming (Eff, Koka, Frank). Underlying all of Haskell's IO discourse is a very deep investment into several breeds of this kind of technology (free monads and their variants). Lately, Unison has been a really interesting language which explores lots of new concepts but also has at its core an extensible effects system that provides excellent language-level support for this kind of coding.
- sriram_malhar 2y ago
  > I don't think that's quite true. The lift here is that the state machine does not do any IO on its own. Here is a simple counter example. Suppose you have to process a packet that contains many sequences (strings/binary blobs) prefixed by 4 bytes of length. You are not always guaranteed to get the length bytes or the string all in one go. In a sequential system you'd accumulate the string as follows handle_input(...) while not received 4 bytes accumulate in buf len = toInt(buf[0..4]) while not received len bytes accumulate in buf If implemented as a state machine ,these would require two await points to assemble the string. Flattening this out into a state machine manually is a pain.
  - tel 2y ago
    I agree totally, it wasn't my intention to say that there aren't protocols which require non-trivial state machines to implement their behavior. To be more clear, I'm contesting that the only thing being discussed in the article is this convenience around writing state machines. I think whether or not you have to write non-trivial state machines by hand or have them generated by some convenient syntax is orthogonal to the bigger insight of what Sans-IO is going after. I think the most important part here is that you write these state machines such that they perform no impure calculation on their own. In other words, you write state machines that must be driven by an event loop which is responsible for interpreting commands from those state machines and that all IO (and more generally, all impure calculation) is performed exclusively by that event loop. It's much more possible to compose machines like this because they don't make as many assumptions on the runtime. It's not that they're reading from a blocking or non-blocking socket. It's that they process some chunk of bytes and _possibly_ want to send some chunk of bytes back. The event loop, constructed by the user of the state machine, is responsible for deciding how to read/write those bytes.

wh33zle 2y ago

I tried to address this at the end of the post: If what you are implementing is mostly _sequential_ IO operations, then this model becomes a bit painful. That isn't always the case though. In more packet-oriented usecases (QUIC, WebRTC & IP), doing the actual IO bit is easy: send & receive individual packets / datagrams. There isn't really much the compiler can generate for you because you don't end up with many `.await` points. At the same time, the state management across all these futures becomes spaghetti code because many of these aspects should run concurrently and thus need to be in their own future / task.

k_bx 2y ago

Yep. The only things about async that bothers me is the need to write ".await" everywhere. I wish there'd be a way to inverse this, and actually just run ".await" by default, while having a special construct not to.

mdtusz 2y ago
You mean `.await`, I assume?
- k_bx 2y ago
  Thanks, didn't have my coffee yet :)
- sirdvd 2y ago
  /s https://xkcd.com/2954/ https://xkcd.com/2954/
vlmutolo 2y ago
It’s important to be able to see where the async function might pause execution. For example, if you’re holding a mutex lock, you probably want to avoid holding it “across” an await point so that it’s not locked for longer than necessary if the function is paused.
- Arnavion 2y ago
  I agree that the explicit yield syntax is good. To play devil's advocate though, the case of Mutex specifically has it going for it that MutexGuard is !Send, so it's a compiler error if a MutexGuard is held across an await point in a Send Future. But yes if your Future is also !Send then the compiler will allow it. In that case, your only recourse is that clippy has lints for holding Mutex and RefCell guards across await points, as long as you're running it and paying attention to it of course.
- k_bx 2y ago
  I disagree. It should be a compiler warning, maybe a "clippy" one, in such cases. Btw the problem of sync code blocking async code is very real and also needs to be resolved, adding explicit `.blocking` to every blocking call is just as bad as explicit .await at every line. Also, I like Haskell approach of being able to introduce syntax extensions at a file-level, so that for code that'd benefit from explicit await – I'd rather let author have it explicit.
- PenguinCoder 2y ago
  > It’s important to be able to see where the async function might pause execution. Why? Your example does not prove the point.

Aissen 2y ago

It is not billed as revolutionary. From the article: > This pattern isn't something that we invented! The Python world even has a dedicated website about it. And yet it is too common to find protocol libraries doing I/O in the wild :-(

PaulHoule 2y ago

It was how we did I/O in assembly language in the 1980s. How else would you write an interrupt-driven YMODEM implementation?

tel 2y ago

I don't think that's quite true. The lift here is that the state machine does not do any IO on its own. It always delegates that work to the event loop that's hosting it, which allows it to be interpreted in different contexts. That makes it more testable and more composable as it makes fewer assumptions about the runtime environment. Theoretically, you could do the same thing with async/await constructing the state machines for you, although in practice it's pretty painful and most async/await code is impure. There are lots of more experimental languages which exceptional support for this style of programming (Eff, Koka, Frank). Underlying all of Haskell's IO discourse is a very deep investment into several breeds of this kind of technology (free monads and their variants). Lately, Unison has been a really interesting language which explores lots of new concepts but also has at its core an extensible effects system that provides excellent language-level support for this kind of coding.

sriram_malhar 2y ago
> I don't think that's quite true. The lift here is that the state machine does not do any IO on its own. Here is a simple counter example. Suppose you have to process a packet that contains many sequences (strings/binary blobs) prefixed by 4 bytes of length. You are not always guaranteed to get the length bytes or the string all in one go. In a sequential system you'd accumulate the string as follows handle_input(...) while not received 4 bytes accumulate in buf len = toInt(buf[0..4]) while not received len bytes accumulate in buf If implemented as a state machine ,these would require two await points to assemble the string. Flattening this out into a state machine manually is a pain.
- tel 2y ago
  I agree totally, it wasn't my intention to say that there aren't protocols which require non-trivial state machines to implement their behavior. To be more clear, I'm contesting that the only thing being discussed in the article is this convenience around writing state machines. I think whether or not you have to write non-trivial state machines by hand or have them generated by some convenient syntax is orthogonal to the bigger insight of what Sans-IO is going after. I think the most important part here is that you write these state machines such that they perform no impure calculation on their own. In other words, you write state machines that must be driven by an event loop which is responsible for interpreting commands from those state machines and that all IO (and more generally, all impure calculation) is performed exclusively by that event loop. It's much more possible to compose machines like this because they don't make as many assumptions on the runtime. It's not that they're reading from a blocking or non-blocking socket. It's that they process some chunk of bytes and _possibly_ want to send some chunk of bytes back. The event loop, constructed by the user of the state machine, is responsible for deciding how to read/write those bytes.