4 ms·

I don't like this at all. Having to rely on futures::join! means that I don't have the flexibility to control the execution of these things unless Rust adds tha

I don't like this at all. Having to rely on futures::join! means that I don't have the flexibility to control the execution of these things unless Rust adds that specific utility, right? In JS, for example, the `bluebird` library is a third party utility for managing execution of functions. You can do things like const results = await Promise.map(users, user => saveUserToDBAsync(user), { concurrency: 5}); And I pass in thousands of users, and can specify `concurrency: 5` to know that it will be execute no more than 5 simultaneously. Implementation of this behavior in user space is trivial in JS, is it possible in Rust?

hguant 7y ago
It feels like you're complaining about things in the Rust language without taking the time to understand how the language idioms work. RTFM. Additionally, you're making snarky comments about how you don't like how the base language doesn't handle something like JS...then reference a third party JS library. Base JS doesn't solve your 'problem' either. To answer your question, async/await provides hooks for an executor (tokio being the most common) to run your code. You things like that in the executor. https://docs.rs/tokio/0.2.0-alpha.6/tokio/executor/index.html https://docs.rs/tokio/0.2.0-alpha.6/tokio/executor/index.htm...
- ralusek 7y ago
  I reference a third-party library to show that you have full control of this stuff in user-space. Implementing Promise.all or Promise.map in userspace is trivial. I'm not complaining about things without taking the time to understand how the language works, I'm giving examples of things that don't seem possible based off of my understanding of how the language works...in hopes that someone will either clarify or accept that this is a shortcoming.
  - roblabla 7y ago
    Rust and JS have very very different execution models. You can absolutely control how many futures are allowed to make progress at the same time fully in userspace. If you're joining N futures, and want to only allow M futures to make progress at a time, make an adapter that only calls the poll function of M futures at a time, until those futures call ready. Rust gives you all the flexibility you need here. It might not be trivial yet because all the adapters might not be written yet, but that's purely a maturity problem. The `join` macro does nothing magical. Go check out its implementation, and it will make it obvious how to implement a concurrency argument.
- deleted 7y ago
  [deleted]
steveklabnik 7y ago
The join macro is also implemented “in user space”.
- ralusek 7y ago
  Thank you, so what is the syntax that is used in order to execute without awaiting? Do you create a thread for each?
  - roblabla 7y ago
    This is where things get tricky: Rust didn't standardize an event loop - it only standardized common traits that allow implementing one, and a way to communicate whether a computation needs more time or already has a result. If what you want to do is run multiple CPU-bound computation and have a central event loop awaiting the result, then yes, you'll need to spawn threads and use some kind of channel to transfer the state and result. If what you want is to run multiple IO-bound queries, then you'll want to use the facilities of the event loop of your choice (tokio, async-std, etc...) to register the intent that you're waiting for more data on a file-descriptor. The "proper" way to execute without awaiting it on the current future is usually to spawn another future on the event loop. The syntax to do that with tokio is use tokio; let my_future = some_future(); tokio::spawn(my_future);
  - Rusky 7y ago
    You do not create a thread for each- that would defeat most of the purpose of futures. Instead you call `Future::poll`, which runs a future until it blocks again, and provide it a way to signal when it is ready. That signal would be handed off to an event loop (which tracks things executing on other hardware like network or disk controllers) or another part of the program (which will be scheduled eventually).
Sean1708 7y ago
The macro is just a wrapper around this (and a couple of other functions): https://docs.rs/futures/0.3.0/futures/future/fn.join.html https://docs.rs/futures/0.3.0/futures/future/fn.join.html And from a quick scan of the source it doesn't look like anything there is impossible to implement in userspace: https://docs.rs/futures-util/0.3.0/src/futures_util/future/join.rs.html#11-118 https://docs.rs/futures-util/0.3.0/src/futures_util/future/j...
conradev 7y ago
The async map you laid out above could be accomplished with a `Stream` in async Rust. You can turn the array into a `Stream`, have a map operation to return a future, and then use the `buffered` method to run N at once: https://docs.rs/futures/0.3.0/futures/stream/trait.StreamExt.html#method.buffered https://docs.rs/futures/0.3.0/futures/stream/trait.StreamExt... Not only does the `futures` crate provide most things you'd ever want, it also has no special treatment – you can implements your own combinators in the same way that `futures` implements them if you need something off of the beaten path.
Rusky 7y ago
Fortunately, `futures::join!` isn't provided by Rust- it's provided by library code. I am not aware (off the top of my head) of an existing equivalent to your `Promise.map` example, but it is also implementable in userspace in Rust. The primitive operation provided by a Rust `Future` is `poll`. Calling `some_future.poll(waker)` advances it forward if possible, and stashes `waker` somewhere for it to be signaled when `some_future` is ready to run again. So the implementation of `join` is fairly straightforward: It constructs a new future wrapping its arguments, which when polled itself, re-polls each of them with the same waker it was passed. There are also more elaborate schemes- e.g. `FuturesUnordered` uses a separate waker for each sub-future, so it can handle larger numbers of them at some coordination cost.
littlestymaar 7y ago
In Rust, everything is in ‘user space’ since nothing is managed by the language itself, you have full control on how the Futures are executed. Even the executor (the “event loop” in js parlance) is a third party library. You can't get more control in js than what Rust gives you.

hguant 7y ago

It feels like you're complaining about things in the Rust language without taking the time to understand how the language idioms work. RTFM. Additionally, you're making snarky comments about how you don't like how the base language doesn't handle something like JS...then reference a third party JS library. Base JS doesn't solve your 'problem' either. To answer your question, async/await provides hooks for an executor (tokio being the most common) to run your code. You things like that in the executor. https://docs.rs/tokio/0.2.0-alpha.6/tokio/executor/index.html https://docs.rs/tokio/0.2.0-alpha.6/tokio/executor/index.htm...

ralusek 7y ago
I reference a third-party library to show that you have full control of this stuff in user-space. Implementing Promise.all or Promise.map in userspace is trivial. I'm not complaining about things without taking the time to understand how the language works, I'm giving examples of things that don't seem possible based off of my understanding of how the language works...in hopes that someone will either clarify or accept that this is a shortcoming.
- roblabla 7y ago
  Rust and JS have very very different execution models. You can absolutely control how many futures are allowed to make progress at the same time fully in userspace. If you're joining N futures, and want to only allow M futures to make progress at a time, make an adapter that only calls the poll function of M futures at a time, until those futures call ready. Rust gives you all the flexibility you need here. It might not be trivial yet because all the adapters might not be written yet, but that's purely a maturity problem. The `join` macro does nothing magical. Go check out its implementation, and it will make it obvious how to implement a concurrency argument.
deleted 7y ago
[deleted]

steveklabnik 7y ago

The join macro is also implemented “in user space”.

ralusek 7y ago
Thank you, so what is the syntax that is used in order to execute without awaiting? Do you create a thread for each?
- roblabla 7y ago
  This is where things get tricky: Rust didn't standardize an event loop - it only standardized common traits that allow implementing one, and a way to communicate whether a computation needs more time or already has a result. If what you want to do is run multiple CPU-bound computation and have a central event loop awaiting the result, then yes, you'll need to spawn threads and use some kind of channel to transfer the state and result. If what you want is to run multiple IO-bound queries, then you'll want to use the facilities of the event loop of your choice (tokio, async-std, etc...) to register the intent that you're waiting for more data on a file-descriptor. The "proper" way to execute without awaiting it on the current future is usually to spawn another future on the event loop. The syntax to do that with tokio is use tokio; let my_future = some_future(); tokio::spawn(my_future);
- Rusky 7y ago
  You do not create a thread for each- that would defeat most of the purpose of futures. Instead you call `Future::poll`, which runs a future until it blocks again, and provide it a way to signal when it is ready. That signal would be handed off to an event loop (which tracks things executing on other hardware like network or disk controllers) or another part of the program (which will be scheduled eventually).

Sean1708 7y ago

The macro is just a wrapper around this (and a couple of other functions): https://docs.rs/futures/0.3.0/futures/future/fn.join.html https://docs.rs/futures/0.3.0/futures/future/fn.join.html And from a quick scan of the source it doesn't look like anything there is impossible to implement in userspace: https://docs.rs/futures-util/0.3.0/src/futures_util/future/join.rs.html#11-118 https://docs.rs/futures-util/0.3.0/src/futures_util/future/j...

conradev 7y ago

The async map you laid out above could be accomplished with a `Stream` in async Rust. You can turn the array into a `Stream`, have a map operation to return a future, and then use the `buffered` method to run N at once: https://docs.rs/futures/0.3.0/futures/stream/trait.StreamExt.html#method.buffered https://docs.rs/futures/0.3.0/futures/stream/trait.StreamExt... Not only does the `futures` crate provide most things you'd ever want, it also has no special treatment – you can implements your own combinators in the same way that `futures` implements them if you need something off of the beaten path.

Rusky 7y ago

Fortunately, `futures::join!` isn't provided by Rust- it's provided by library code. I am not aware (off the top of my head) of an existing equivalent to your `Promise.map` example, but it is also implementable in userspace in Rust. The primitive operation provided by a Rust `Future` is `poll`. Calling `some_future.poll(waker)` advances it forward if possible, and stashes `waker` somewhere for it to be signaled when `some_future` is ready to run again. So the implementation of `join` is fairly straightforward: It constructs a new future wrapping its arguments, which when polled itself, re-polls each of them with the same waker it was passed. There are also more elaborate schemes- e.g. `FuturesUnordered` uses a separate waker for each sub-future, so it can handle larger numbers of them at some coordination cost.

littlestymaar 7y ago

In Rust, everything is in ‘user space’ since nothing is managed by the language itself, you have full control on how the Futures are executed. Even the executor (the “event loop” in js parlance) is a third party library. You can't get more control in js than what Rust gives you.