3 ms·

I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to

I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to add tasks to an already running pool of tasks and posted my findings here: https://github.com/dimitarvp/rust-async-examples/blob/main/examples/add_workers_during_runtime_1.rs https://github.com/dimitarvp/rust-async-examples/blob/main/e... (there's #2 as well with some more comments and a different approach). The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things with async Rust to me is both a red flag and me being diligent BUT it should be more obvious. And promoted in docs. Rust started suffering from "you got all the nuts and bolts in place, now build your own solution, son" syndrome which I grew to dislike. Too low-level. I wouldn't mind something akin to e.g. Golang's flowmatic library (check the first two examples at the top of the README): https://github.com/carlmjohnson/flowmatic https://github.com/carlmjohnson/flowmatic

insanitybit 3y ago
I'd like to think that I've worked on some "real projects" and I've never run into these issues tbh. It's just hard for me to wrap my mind around. Like I said, I get wanting more libraries (although I think they exist? A local pool exists and you can spawn tasks in it, using tokio) but I'm just not seeing a fundamental problem. I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. But I don't really understand its purpose either.
- pdimitar 3y ago
  > but I'm just not seeing a fundamental problem. Well, I mentioned it a few times in this thread: things are mixed in non-intuitive ways -- async/await, tokio, and various crates (including StreamExt which was not at all obvious and it took me a while to finally find mentioned in a few blog posts and GitHub gists). It just introduces friction and nowadays I am more averse to tech that requires more homework. Admittedly a personal preference of course but I don't think it's an invalid concern either. > I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. Well, I needed such a pattern in a project that had to start as quickly as at all possible and only start and schedule the most critical work -- and then start adding all the other work as the first batch of work was already in progress. Would you write it differently and if so, how?
  - insanitybit 3y ago
    > Would you write it differently and if so, how? It's really hard to say because I don't fully know your use case. Maybe your way was the right way but I think I probably would have done something differently. Maybe a priority queue of jobs with a static pool of workers.
    - pdimitar 3y ago
      Yeah, fair enough, I admit I haven't dabbled with any Rust job queues yet.
Nullabillity 3y ago
`flowmatic.Do` is equivalent to `future::try_join` (https://docs.rs/futures/latest/futures/future/fn.try_join.html https://docs.rs/futures/latest/futures/future/fn.try_join.ht...), and `.Each` is equivalent to `stream::iter().map().buffer_unordered()` (https://docs.rs/futures/latest/futures/stream/fn.iter.html https://docs.rs/futures/latest/futures/stream/fn.iter.html, https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.map https://docs.rs/futures/latest/futures/stream/trait.StreamEx..., https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.buffer_unordered https://docs.rs/futures/latest/futures/stream/trait.StreamEx...). `.ManageTasks` doesn't seem to have a clear analogue that I can think of right now, that's an interesting one. `.TaskPool` looks like `StreamExt::buffer_unordered` unless I'm missing something. The code example in the README didn't really say a lot for someone who isn't already well-versed in the library.
- pdimitar 3y ago
  That's on me for making blanket statements -- my apologies. Yep, I know about the first two and I used them regularly. My issue in general is that async Rust involves using several things at the same time (not just the two keywords) and this should be surfaced much earlier in any intro material and the maintainers of the language (or the libraries, or both) should just double down on whatever they feel is the best way to do async Rust. The stance of "you have freedom, make your choice and assemble your own LEGO" is not very productive. I want Rust to start being a bit more opinionated. Though I'll recognize this is just a personal taste but I still have to defend it by saying that as a programmer who is paid to deliver, I want to be able to deliver in predictable timelines and not having to go off on an adventure to learn all the intricacies of async Rust before I am able to write a feature.
FridgeSeal 3y ago
> The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things While I resonate with this, because I also went through a period of struggling with async stuff, I genuinely think this is because async is just hard in general. Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. A lot of the async stuff, how it actually worked, and how to actually use it, only clicked for me when I played around with Glommio, which runs an executor-per-core, and some of the constraints it imposed made understanding it all somewhat easier.
- pdimitar 3y ago
  Thanks a lot of the Glommio mention, that's an instant star and I'll review it in more details Soon™. > Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. Yeah, very well put. Indeed it's very powerful and sure it's confusing. I need guard rails. And I need my hands slapped much more. "Looks like you're trying X -- this is how you do it, you idiot" would work well. :D I am almost not joking even.

insanitybit 3y ago

I'd like to think that I've worked on some "real projects" and I've never run into these issues tbh. It's just hard for me to wrap my mind around. Like I said, I get wanting more libraries (although I think they exist? A local pool exists and you can spawn tasks in it, using tokio) but I'm just not seeing a fundamental problem. I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. But I don't really understand its purpose either.

pdimitar 3y ago
> but I'm just not seeing a fundamental problem. Well, I mentioned it a few times in this thread: things are mixed in non-intuitive ways -- async/await, tokio, and various crates (including StreamExt which was not at all obvious and it took me a while to finally find mentioned in a few blog posts and GitHub gists). It just introduces friction and nowadays I am more averse to tech that requires more homework. Admittedly a personal preference of course but I don't think it's an invalid concern either. > I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. Well, I needed such a pattern in a project that had to start as quickly as at all possible and only start and schedule the most critical work -- and then start adding all the other work as the first batch of work was already in progress. Would you write it differently and if so, how?
- insanitybit 3y ago
  > Would you write it differently and if so, how? It's really hard to say because I don't fully know your use case. Maybe your way was the right way but I think I probably would have done something differently. Maybe a priority queue of jobs with a static pool of workers.
  - pdimitar 3y ago
    Yeah, fair enough, I admit I haven't dabbled with any Rust job queues yet.

Nullabillity 3y ago

`flowmatic.Do` is equivalent to `future::try_join` (https://docs.rs/futures/latest/futures/future/fn.try_join.html https://docs.rs/futures/latest/futures/future/fn.try_join.ht...), and `.Each` is equivalent to `stream::iter().map().buffer_unordered()` (https://docs.rs/futures/latest/futures/stream/fn.iter.html https://docs.rs/futures/latest/futures/stream/fn.iter.html, https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.map https://docs.rs/futures/latest/futures/stream/trait.StreamEx..., https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.buffer_unordered https://docs.rs/futures/latest/futures/stream/trait.StreamEx...). `.ManageTasks` doesn't seem to have a clear analogue that I can think of right now, that's an interesting one. `.TaskPool` looks like `StreamExt::buffer_unordered` unless I'm missing something. The code example in the README didn't really say a lot for someone who isn't already well-versed in the library.

pdimitar 3y ago
That's on me for making blanket statements -- my apologies. Yep, I know about the first two and I used them regularly. My issue in general is that async Rust involves using several things at the same time (not just the two keywords) and this should be surfaced much earlier in any intro material and the maintainers of the language (or the libraries, or both) should just double down on whatever they feel is the best way to do async Rust. The stance of "you have freedom, make your choice and assemble your own LEGO" is not very productive. I want Rust to start being a bit more opinionated. Though I'll recognize this is just a personal taste but I still have to defend it by saying that as a programmer who is paid to deliver, I want to be able to deliver in predictable timelines and not having to go off on an adventure to learn all the intricacies of async Rust before I am able to write a feature.

FridgeSeal 3y ago

> The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things While I resonate with this, because I also went through a period of struggling with async stuff, I genuinely think this is because async is just hard in general. Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. A lot of the async stuff, how it actually worked, and how to actually use it, only clicked for me when I played around with Glommio, which runs an executor-per-core, and some of the constraints it imposed made understanding it all somewhat easier.

pdimitar 3y ago
Thanks a lot of the Glommio mention, that's an instant star and I'll review it in more details Soon™. > Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. Yeah, very well put. Indeed it's very powerful and sure it's confusing. I need guard rails. And I need my hands slapped much more. "Looks like you're trying X -- this is how you do it, you idiot" would work well. :D I am almost not joking even.