6 ms·

I don't quite follow. What exactly is the overhead that other languages have for futures that is eliminated here?

tmandry 7y ago
Most languages allocate every future (and sub-future, and sub-sub-future) separately on the heap. This leads to some overhead, allocating and deallocating space to store our task state. In Rust, you can "inline" an entire chain of futures into a single heap allocation.
- pjmlp 7y ago
  In .NET something similar is possible via ValueTask.
  - Matthias247 7y ago
    ValueTask is more of an optimization for scenarios where the Future (or Task<T>) is often intended to be ready immediately (synchronously). For those cases its wasteful to first allocate a continuation on the heap, and then not to use it - because the code after the await can directly be executed. The introduction of ValueTask allowed C# code to only perform dynamic allocations whenever the task definitely needed to be yielded - opposed to one allocation for every Task<T>. However it doesn't allow for guaranteed avoidance of allocations - like Rusts model can do. However on the positive side removing the allocations for those cases is probably good enough since the other yields are of low-frequency (when waiting for IO). And Rust code currently requires allocations like Box<dyn Future> in order to make certain things work (like type-erasure and dynamically dispatched interfaces) that are not necessarily required in .NET in the non-yielded case. From an ergonomic perspective I definitely prefer .NETs easy-by-default model which is still highly optimizable up to the point of avoiding all allocations. But I understand why this wouldn't have worked for Rust and it's constraints (no GC, no classes, no runtime).
    - pjmlp 7y ago
      Thanks for the thoughtful reply. You are right, however there is a scenario that you have forgotten. The possibility that the JIT might eventually optimise the code path given enough PGO data, and apply RVO or similar approaches. Naturally I don't know if RyuJIT can already do this kind of optimization, given that only recently they have strengthened their focus on this area. However this kind of optimizations are already doable in Graal, so it is possible that Microsoft also adds them to RyuJIT. Which in any case would rule out some of the Rust deployment scenarios I guess.
- dom96 7y ago
  Has anyone ever done a comparison to see how much overhead this actually adds? I'd be really curious to see this represented in concrete terms.
weiming 7y ago
Going down the same rabbit hole earlier this week, found this to be a good explanation: All of the data needed by a task is contained within its future. That means we can neatly sidestep problems of dynamic stack growth and stack swapping, giving us truly lightweight tasks without any runtime system implications. ... Perhaps surprisingly, the future within a task compiles down to a state machine, so that every time the task wakes up to continue polling, it continues execution from the current state—working just like hand-rolled code. [1] [1] https://aturon.github.io/blog/2016/09/07/futures-design/ https://aturon.github.io/blog/2016/09/07/futures-design/
- davidw 7y ago
  > Perhaps surprisingly, the future within a task compiles down to a state machine, so that every time the task wakes up to continue polling, it continues execution from the current state How are those tasks implemented, and what's scheduling them?
  - weiming 7y ago
    You need a third-party library to provide an executor. Rust does not come with one to keep the runtime size small. The community seems to have centralized on https://tokio.rs/ https://tokio.rs/ (under the hood it uses epoll/whatever OS-specific functionality to schedule M:N) See: https://news.ycombinator.com/item?id=20722297 https://news.ycombinator.com/item?id=20722297
    - davidw 7y ago
      How's that play out in terms of, say, performing some long-running calculations, rather than something that performs IO?
      - oconnor663 7y ago
        When you want to do long-running blocking work (either computation, or synchronous IO from some library that doesn't use futures), you usually want to farm that work out to a thread pool. I think Tokio provides convenience functions for doing that. It's also possible to write a long-running computation as a future, which can yield after some smaller amount of work to let some IO get done. But I'm not sure if that's the recommended approach.
deleted 7y ago
[deleted]
zwkrt 7y ago
So for example, in Kotlin each piece of synchronous code within an async function is compiled into what is essentially a Java Runnable object, which must be allocated on the heap.
- continuational 7y ago
  Ah. What kind of asynchronous task executes so fast that a heap allocation is measurable?
  - shepmaster 7y ago
    I like to think of it from the other direction. I'm super excited to use futures and async/await in embedded hardware, where "the heap" might not even exist. Hardware interrupts can be treated as events that wake the executor. That lets me write some code like this: async fn bracketed_echo() -> ! { loop { let mut buf = 0; Serial.rx(&mut buf).await; Serial.tx(b'>').await; Serial.tx(buf).await; Serial.tx(b'<').await; } } This reads from the serial port and echos it back inside of `><`. The code reads very cleanly, does not require the heap, and should be very efficient. The caveat is that I am also trying to target a platform that Lust / LLVM doesn't support completely yet, so I haven't gotten this to actually work well. I bet the people using the Cortex processor are far ahead of me though.
    - zwirbl 7y ago
      That's exactly what I am looking forward to, the state machine generation can make a lot of code targeting embedded platforms, especially the IO part, much more comfortable. Might I ask which controller you are using? With Rust I'm still on a Cortex, but looking to apply it on other architectures in the future
  - gpderetta 7y ago
    A network read or write can be in the order of hundreds of nanoseconds when using an accelerated userspace networking stack. Allocation can be a good chunk of that.

Most languages allocate every future (and sub-future, and sub-sub-future) separately on the heap. This leads to some overhead, allocating and deallocating space to store our task state. In Rust, you can "inline" an entire chain of futures into a single heap allocation.

pjmlp 7y ago
In .NET something similar is possible via ValueTask.
- Matthias247 7y ago
  ValueTask is more of an optimization for scenarios where the Future (or Task<T>) is often intended to be ready immediately (synchronously). For those cases its wasteful to first allocate a continuation on the heap, and then not to use it - because the code after the await can directly be executed. The introduction of ValueTask allowed C# code to only perform dynamic allocations whenever the task definitely needed to be yielded - opposed to one allocation for every Task<T>. However it doesn't allow for guaranteed avoidance of allocations - like Rusts model can do. However on the positive side removing the allocations for those cases is probably good enough since the other yields are of low-frequency (when waiting for IO). And Rust code currently requires allocations like Box<dyn Future> in order to make certain things work (like type-erasure and dynamically dispatched interfaces) that are not necessarily required in .NET in the non-yielded case. From an ergonomic perspective I definitely prefer .NETs easy-by-default model which is still highly optimizable up to the point of avoiding all allocations. But I understand why this wouldn't have worked for Rust and it's constraints (no GC, no classes, no runtime).
  - pjmlp 7y ago
    Thanks for the thoughtful reply. You are right, however there is a scenario that you have forgotten. The possibility that the JIT might eventually optimise the code path given enough PGO data, and apply RVO or similar approaches. Naturally I don't know if RyuJIT can already do this kind of optimization, given that only recently they have strengthened their focus on this area. However this kind of optimizations are already doable in Graal, so it is possible that Microsoft also adds them to RyuJIT. Which in any case would rule out some of the Rust deployment scenarios I guess.
dom96 7y ago
Has anyone ever done a comparison to see how much overhead this actually adds? I'd be really curious to see this represented in concrete terms.

weiming 7y ago

Going down the same rabbit hole earlier this week, found this to be a good explanation: All of the data needed by a task is contained within its future. That means we can neatly sidestep problems of dynamic stack growth and stack swapping, giving us truly lightweight tasks without any runtime system implications. ... Perhaps surprisingly, the future within a task compiles down to a state machine, so that every time the task wakes up to continue polling, it continues execution from the current state—working just like hand-rolled code. [1] [1] https://aturon.github.io/blog/2016/09/07/futures-design/ https://aturon.github.io/blog/2016/09/07/futures-design/

davidw 7y ago
> Perhaps surprisingly, the future within a task compiles down to a state machine, so that every time the task wakes up to continue polling, it continues execution from the current state How are those tasks implemented, and what's scheduling them?
- weiming 7y ago
  You need a third-party library to provide an executor. Rust does not come with one to keep the runtime size small. The community seems to have centralized on https://tokio.rs/ https://tokio.rs/ (under the hood it uses epoll/whatever OS-specific functionality to schedule M:N) See: https://news.ycombinator.com/item?id=20722297 https://news.ycombinator.com/item?id=20722297
  - davidw 7y ago
    How's that play out in terms of, say, performing some long-running calculations, rather than something that performs IO?
    - oconnor663 7y ago
      When you want to do long-running blocking work (either computation, or synchronous IO from some library that doesn't use futures), you usually want to farm that work out to a thread pool. I think Tokio provides convenience functions for doing that. It's also possible to write a long-running computation as a future, which can yield after some smaller amount of work to let some IO get done. But I'm not sure if that's the recommended approach.

deleted 7y ago

[deleted]

zwkrt 7y ago

So for example, in Kotlin each piece of synchronous code within an async function is compiled into what is essentially a Java Runnable object, which must be allocated on the heap.

continuational 7y ago
Ah. What kind of asynchronous task executes so fast that a heap allocation is measurable?
- shepmaster 7y ago
  I like to think of it from the other direction. I'm super excited to use futures and async/await in embedded hardware, where "the heap" might not even exist. Hardware interrupts can be treated as events that wake the executor. That lets me write some code like this: async fn bracketed_echo() -> ! { loop { let mut buf = 0; Serial.rx(&mut buf).await; Serial.tx(b'>').await; Serial.tx(buf).await; Serial.tx(b'<').await; } } This reads from the serial port and echos it back inside of `><`. The code reads very cleanly, does not require the heap, and should be very efficient. The caveat is that I am also trying to target a platform that Lust / LLVM doesn't support completely yet, so I haven't gotten this to actually work well. I bet the people using the Cortex processor are far ahead of me though.
  - zwirbl 7y ago
    That's exactly what I am looking forward to, the state machine generation can make a lot of code targeting embedded platforms, especially the IO part, much more comfortable. Might I ask which controller you are using? With Rust I'm still on a Cortex, but looking to apply it on other architectures in the future
- gpderetta 7y ago
  A network read or write can be in the order of hundreds of nanoseconds when using an accelerated userspace networking stack. Allocation can be a good chunk of that.