4 ms·

This is big! Turns out that syntactic support for asynchronous programming in Rust isn't just syntactic: it enables the compiler to reason about the lifetimes i

This is big! Turns out that syntactic support for asynchronous programming in Rust isn't just syntactic: it enables the compiler to reason about the lifetimes in asynchronous code in a way that wasn't possible to implement in libraries. The end result of having async/await syntax is that async code reads just like normal Rust, which definitely wasn't the case before. This is a huge improvement in usability.

takeda 7y ago
Why would it be different for async code than sync code? The goal of Rust's checker is to track lifetime of an object so for example it knows that at the end of a function the object should be freed. Async shouldn't matter here.
- aliceryhl 7y ago
  In Rust, it is normally not possible or at least very difficult to create structs where one field references another, and if you were to create a future that borrows some field, awaits a future, and uses the borrowed field, the resulting future will need to have a field with a reference to another. This is the challenge and async await lets you make this kind of self referential types without unsafe code.
  - antisemiotic 7y ago
    That's just the "Pin" type, which is heavily used in async code behind (and occasionally in front) the scenes, but is by no means restricted to it.
    - Nemo157 7y ago
      It's impossible to use the guarantees provided by the `Pin` type without `unsafe` code, except in `async fn`.
- GolDDranks 7y ago
  The point is that Rust's borrow checker can't reason about lifetimes very well over function boundaries. It can reason about coarse things that are expressable in the type language, but everything more nuanced than that, such as reasoning about how control flow affects the lifetimes is limited to inside function bodies. The difference between synchronous code and async code implemented as libraries is that async code involves jumping in and out of functions a lot, while employing runtime library code in between. A piece of code that is conceptually straightforward, may, in the async case, involve multiple returns and restores. In the sync case it doesn't need to do that, since it just blocks the thread and does the processing in other threads and in kernel land. Rust's async/await support makes it possible to write code that is structurally "straightforward" in a similar way than synchronous code would be. That allows the borrow checker to reason about it in a similar way it would reason about sync code.
  - asdkhadsj 7y ago
    If you're familiar with this, can you describe some of those new concepts in ... slightly more detail? I say slightly, because I'm still seeking high level explanations, but at the same time I'm curious what new features might be making this async lifetime talk possible. To further frame that question: I had assumed Rust was implementing Async within the capabilities of normal Rust. Such that, if lifetimes were being managed across function bounds, I assumed the lifetime would have "simply" been bound to the scope of the executor, polling the future you're actively waiting on. However quickly I can see confusions in that description. Normal lifetimes within a function would need to bubble up and be managed by a parent, since that function's lifetime is, as you put it, being jumped in and out of frequently. So I imagine that is partly where new features come into play? Giving Rust the ability to take a normal lifetime and extend or manage it in new ways?
    - GolDDranks 7y ago
      I'm not going to elaborate super deeply in a HN comment, but here's the gist: inside a function body, you can take a reference to a thing, and store that reference in a variable. Then if that function were "yields" to the executor, we have a situation that we couldn't have with sync code: in sync code, the "yielding" only happens at the function return, and by that point, all the internal references must be gone, as the function stack frame is going to disappear. But with async, as the yielding can happen without the function actually ending, we have to be able to store the stack frame, and the references stay alive. The new feature helps the compiler to reason about this. Before the "yields" were implemented as just returns, so the compiler didn't allow borrowing over yield points.
- epage 7y ago
  The challenge with async code is that the state across yield points is put into a struct and if a reference to a stack variable is used across yield points, then that struct is self-referential which Rust can't reason about (yet?). I honestly do not know who much can be recreated with `unsafe` and `Pin` vs how much is built-in. I'd love for Rust to eventually get a Move trait (something like C=+ move constructors) to resolve this. Besides some complexity in designing it, there is resistance from some corners about having anything execute on a move.
  - stbuehler 7y ago
    That is exactly what `async` blocks are about: they do support self-referential structs, and `Pin` is what allows us to use those in a safe way, as `Pin`ned data can't be moved again (unless the data is `Unpin`, which self-referential structs are not), so the self-references are safe.

takeda 7y ago

Why would it be different for async code than sync code? The goal of Rust's checker is to track lifetime of an object so for example it knows that at the end of a function the object should be freed. Async shouldn't matter here.

aliceryhl 7y ago
In Rust, it is normally not possible or at least very difficult to create structs where one field references another, and if you were to create a future that borrows some field, awaits a future, and uses the borrowed field, the resulting future will need to have a field with a reference to another. This is the challenge and async await lets you make this kind of self referential types without unsafe code.
- antisemiotic 7y ago
  That's just the "Pin" type, which is heavily used in async code behind (and occasionally in front) the scenes, but is by no means restricted to it.
  - Nemo157 7y ago
    It's impossible to use the guarantees provided by the `Pin` type without `unsafe` code, except in `async fn`.
GolDDranks 7y ago
The point is that Rust's borrow checker can't reason about lifetimes very well over function boundaries. It can reason about coarse things that are expressable in the type language, but everything more nuanced than that, such as reasoning about how control flow affects the lifetimes is limited to inside function bodies. The difference between synchronous code and async code implemented as libraries is that async code involves jumping in and out of functions a lot, while employing runtime library code in between. A piece of code that is conceptually straightforward, may, in the async case, involve multiple returns and restores. In the sync case it doesn't need to do that, since it just blocks the thread and does the processing in other threads and in kernel land. Rust's async/await support makes it possible to write code that is structurally "straightforward" in a similar way than synchronous code would be. That allows the borrow checker to reason about it in a similar way it would reason about sync code.
- asdkhadsj 7y ago
  If you're familiar with this, can you describe some of those new concepts in ... slightly more detail? I say slightly, because I'm still seeking high level explanations, but at the same time I'm curious what new features might be making this async lifetime talk possible. To further frame that question: I had assumed Rust was implementing Async within the capabilities of normal Rust. Such that, if lifetimes were being managed across function bounds, I assumed the lifetime would have "simply" been bound to the scope of the executor, polling the future you're actively waiting on. However quickly I can see confusions in that description. Normal lifetimes within a function would need to bubble up and be managed by a parent, since that function's lifetime is, as you put it, being jumped in and out of frequently. So I imagine that is partly where new features come into play? Giving Rust the ability to take a normal lifetime and extend or manage it in new ways?
  - GolDDranks 7y ago
    I'm not going to elaborate super deeply in a HN comment, but here's the gist: inside a function body, you can take a reference to a thing, and store that reference in a variable. Then if that function were "yields" to the executor, we have a situation that we couldn't have with sync code: in sync code, the "yielding" only happens at the function return, and by that point, all the internal references must be gone, as the function stack frame is going to disappear. But with async, as the yielding can happen without the function actually ending, we have to be able to store the stack frame, and the references stay alive. The new feature helps the compiler to reason about this. Before the "yields" were implemented as just returns, so the compiler didn't allow borrowing over yield points.
epage 7y ago
The challenge with async code is that the state across yield points is put into a struct and if a reference to a stack variable is used across yield points, then that struct is self-referential which Rust can't reason about (yet?). I honestly do not know who much can be recreated with `unsafe` and `Pin` vs how much is built-in. I'd love for Rust to eventually get a Move trait (something like C=+ move constructors) to resolve this. Besides some complexity in designing it, there is resistance from some corners about having anything execute on a move.
- stbuehler 7y ago
  That is exactly what `async` blocks are about: they do support self-referential structs, and `Pin` is what allows us to use those in a safe way, as `Pin`ned data can't be moved again (unless the data is `Unpin`, which self-referential structs are not), so the self-references are safe.