5 ms·

> by the fact that async isn't in the standard library I wish async/await were not in the language

by worik 3y ago

Ygg2 3y ago
However other people wished it was. So here we are.
sliken 3y ago
So just ignore the async use cases? Or force tokio or similar to support it? My biggest pain point so far is channels, I like them in go because they are straight forward to use, allow concurrency, aren't a minefield of problems, and are easy to reason with. I wanted to scan N directories (in parallel) -> encrypt and checksum in (parallel) -> queue files for upload. With go channels it's straight forward. I was looking for MPMC (like go channels) to allow multiple producers to enqueue and multiple consumers to dequeue. Rust doesn't do it and tokio doesn't do it. I could use crossbeam[0], flume[1], or async[2]. Not sure what limitations those have and what if any limitations in compatibilities with other crates I need will be. [0] https://docs.rs/crossbeam/latest/crossbeam/channel/index.html https://docs.rs/crossbeam/latest/crossbeam/channel/index.htm... [1] https://docs.rs/flume/latest/flume/ https://docs.rs/flume/latest/flume/ [2] https://docs.rs/async-channel/latest/async_channel/ https://docs.rs/async-channel/latest/async_channel/
- worik 3y ago
  > So just ignore the async use cases? Or force tokio or similar to support it? Hell no! Do it differently
  - aw1621107 3y ago
    Differently how? And perhaps more importantly, what tradeoff(s) would those differences entail?
    - Ygg2 3y ago
      Green threads/Virtual threads are one solution, but then you need to ship a runtime for them, then you pay for what you don't use. Or don't standardize on them and have a split ecosystem of half libs supporting use `green-thread.rs` and the other half supporting `rust-gthreads`.
      - api 3y ago
        I wonder if you could have hidden green threads from the application entirely. Have a "full" standard library where std::thread is M:N green threads and I/O is implemented behind the scenes that way, and a "thin" version where std::thread is OS threads and I/O is just passed through to the OS. (Then of course no_std for embedded.) Of course the problem then would be that Rust makes it trivially easy to call C code. As soon as someone in a green thread runtime called libc I/O functions directly they'd be in for a weird surprise when the call froze all threads on their worker. Of course the "full" M:N runtime could be instrumented to print a warning when this occurs unless explicitly told not to, and std::thread in the "full" stdlib could allow explicit creation of full OS threads outside the runtime. There is an attempt at doing this out there: https://github.com/Xudong-Huang/may https://github.com/Xudong-Huang/may
        aw1621107 3y ago
        > Have a "full" standard library where std::thread is M:N green threads and I/O is implemented behind the scenes that way, and a "thin" version where std::thread is OS threads and I/O is just passed through to the OS. (Then of course no_std for embedded.) That sounds vaguely like what Rust used to have with its I/O functions before its green thread implementation was removed [0]? Not sure it's quite the same thing, though: > In today's [2014] Rust, there is a single I/O API -- std::io -- that provides blocking operations only and works with both threading models. Rust is somewhat unusual in allowing programs to mix native and green threading, and furthermore allowing some degree of interoperation between the two. > [snip] > In this setup, libstd works directly against the runtime interface. When invoking an I/O or scheduling operation, it first finds the current Task, and then extracts the Runtime trait object to actually perform the operation. > On native tasks, blocking operations simply block. On green tasks, blocking operations are routed through the green scheduler and/or underlying event loop and nonblocking I/O. [0]: https://github.com/rust-lang/rfcs/blob/master/text/0230-remove-runtime.md https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...
  - deleted 3y ago
    [deleted]
- LegionMammal978 3y ago
  FWIW, either async-channel's types or flume's async APIs should work for your use case. Both are completely agnostic to the async runtime, since the futures are triggered by internal events (recv() waits for send() and vice versa), rather than external events like I/O which are generally coordinated through the runtime.

> by the fact that async isn't in the standard library I wish async/await were not in the language

by worik 3y ago

> by the fact that async isn't in the standard library I wish async/await were not in the language

Ygg2 3y ago

However other people wished it was. So here we are.

sliken 3y ago

So just ignore the async use cases? Or force tokio or similar to support it? My biggest pain point so far is channels, I like them in go because they are straight forward to use, allow concurrency, aren't a minefield of problems, and are easy to reason with. I wanted to scan N directories (in parallel) -> encrypt and checksum in (parallel) -> queue files for upload. With go channels it's straight forward. I was looking for MPMC (like go channels) to allow multiple producers to enqueue and multiple consumers to dequeue. Rust doesn't do it and tokio doesn't do it. I could use crossbeam[0], flume[1], or async[2]. Not sure what limitations those have and what if any limitations in compatibilities with other crates I need will be. [0] https://docs.rs/crossbeam/latest/crossbeam/channel/index.html https://docs.rs/crossbeam/latest/crossbeam/channel/index.htm... [1] https://docs.rs/flume/latest/flume/ https://docs.rs/flume/latest/flume/ [2] https://docs.rs/async-channel/latest/async_channel/ https://docs.rs/async-channel/latest/async_channel/

worik 3y ago
> So just ignore the async use cases? Or force tokio or similar to support it? Hell no! Do it differently
- aw1621107 3y ago
  Differently how? And perhaps more importantly, what tradeoff(s) would those differences entail?
  - Ygg2 3y ago
    Green threads/Virtual threads are one solution, but then you need to ship a runtime for them, then you pay for what you don't use. Or don't standardize on them and have a split ecosystem of half libs supporting use `green-thread.rs` and the other half supporting `rust-gthreads`.
    - api 3y ago
      I wonder if you could have hidden green threads from the application entirely. Have a "full" standard library where std::thread is M:N green threads and I/O is implemented behind the scenes that way, and a "thin" version where std::thread is OS threads and I/O is just passed through to the OS. (Then of course no_std for embedded.) Of course the problem then would be that Rust makes it trivially easy to call C code. As soon as someone in a green thread runtime called libc I/O functions directly they'd be in for a weird surprise when the call froze all threads on their worker. Of course the "full" M:N runtime could be instrumented to print a warning when this occurs unless explicitly told not to, and std::thread in the "full" stdlib could allow explicit creation of full OS threads outside the runtime. There is an attempt at doing this out there: https://github.com/Xudong-Huang/may https://github.com/Xudong-Huang/may
      - aw1621107 3y ago
        > Have a "full" standard library where std::thread is M:N green threads and I/O is implemented behind the scenes that way, and a "thin" version where std::thread is OS threads and I/O is just passed through to the OS. (Then of course no_std for embedded.) That sounds vaguely like what Rust used to have with its I/O functions before its green thread implementation was removed [0]? Not sure it's quite the same thing, though: > In today's [2014] Rust, there is a single I/O API -- std::io -- that provides blocking operations only and works with both threading models. Rust is somewhat unusual in allowing programs to mix native and green threading, and furthermore allowing some degree of interoperation between the two. > [snip] > In this setup, libstd works directly against the runtime interface. When invoking an I/O or scheduling operation, it first finds the current Task, and then extracts the Runtime trait object to actually perform the operation. > On native tasks, blocking operations simply block. On green tasks, blocking operations are routed through the green scheduler and/or underlying event loop and nonblocking I/O. [0]: https://github.com/rust-lang/rfcs/blob/master/text/0230-remove-runtime.md https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...
- deleted 3y ago
  [deleted]
LegionMammal978 3y ago
FWIW, either async-channel's types or flume's async APIs should work for your use case. Both are completely agnostic to the async runtime, since the futures are triggered by internal events (recv() waits for send() and vice versa), rather than external events like I/O which are generally coordinated through the runtime.