5 ms·

If you want golang in Rust just use channels and tasks? Or threads? I don't find async Rust difficult at all, I'm having a hard time really empathizing with th

If you want golang in Rust just use channels and tasks? Or threads? I don't find async Rust difficult at all, I'm having a hard time really empathizing with this to the extent of needing breaking changes. To me, async from a lang perspective is virtually done - in 2024 I suspect all of the various impl Trait and async Trait stuff will be done and at that point I don't see anything left.

pdimitar 3y ago
I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to add tasks to an already running pool of tasks and posted my findings here: https://github.com/dimitarvp/rust-async-examples/blob/main/examples/add_workers_during_runtime_1.rs https://github.com/dimitarvp/rust-async-examples/blob/main/e... (there's #2 as well with some more comments and a different approach). The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things with async Rust to me is both a red flag and me being diligent BUT it should be more obvious. And promoted in docs. Rust started suffering from "you got all the nuts and bolts in place, now build your own solution, son" syndrome which I grew to dislike. Too low-level. I wouldn't mind something akin to e.g. Golang's flowmatic library (check the first two examples at the top of the README): https://github.com/carlmjohnson/flowmatic https://github.com/carlmjohnson/flowmatic
- insanitybit 3y ago
  I'd like to think that I've worked on some "real projects" and I've never run into these issues tbh. It's just hard for me to wrap my mind around. Like I said, I get wanting more libraries (although I think they exist? A local pool exists and you can spawn tasks in it, using tokio) but I'm just not seeing a fundamental problem. I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. But I don't really understand its purpose either.
  - pdimitar 3y ago
    > but I'm just not seeing a fundamental problem. Well, I mentioned it a few times in this thread: things are mixed in non-intuitive ways -- async/await, tokio, and various crates (including StreamExt which was not at all obvious and it took me a while to finally find mentioned in a few blog posts and GitHub gists). It just introduces friction and nowadays I am more averse to tech that requires more homework. Admittedly a personal preference of course but I don't think it's an invalid concern either. > I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. Well, I needed such a pattern in a project that had to start as quickly as at all possible and only start and schedule the most critical work -- and then start adding all the other work as the first batch of work was already in progress. Would you write it differently and if so, how?
    - insanitybit 3y ago
      > Would you write it differently and if so, how? It's really hard to say because I don't fully know your use case. Maybe your way was the right way but I think I probably would have done something differently. Maybe a priority queue of jobs with a static pool of workers.
      - pdimitar 3y ago
        Yeah, fair enough, I admit I haven't dabbled with any Rust job queues yet.
- Nullabillity 3y ago
  `flowmatic.Do` is equivalent to `future::try_join` (https://docs.rs/futures/latest/futures/future/fn.try_join.html https://docs.rs/futures/latest/futures/future/fn.try_join.ht...), and `.Each` is equivalent to `stream::iter().map().buffer_unordered()` (https://docs.rs/futures/latest/futures/stream/fn.iter.html https://docs.rs/futures/latest/futures/stream/fn.iter.html, https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.map https://docs.rs/futures/latest/futures/stream/trait.StreamEx..., https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.buffer_unordered https://docs.rs/futures/latest/futures/stream/trait.StreamEx...). `.ManageTasks` doesn't seem to have a clear analogue that I can think of right now, that's an interesting one. `.TaskPool` looks like `StreamExt::buffer_unordered` unless I'm missing something. The code example in the README didn't really say a lot for someone who isn't already well-versed in the library.
  - pdimitar 3y ago
    That's on me for making blanket statements -- my apologies. Yep, I know about the first two and I used them regularly. My issue in general is that async Rust involves using several things at the same time (not just the two keywords) and this should be surfaced much earlier in any intro material and the maintainers of the language (or the libraries, or both) should just double down on whatever they feel is the best way to do async Rust. The stance of "you have freedom, make your choice and assemble your own LEGO" is not very productive. I want Rust to start being a bit more opinionated. Though I'll recognize this is just a personal taste but I still have to defend it by saying that as a programmer who is paid to deliver, I want to be able to deliver in predictable timelines and not having to go off on an adventure to learn all the intricacies of async Rust before I am able to write a feature.
- FridgeSeal 3y ago
  > The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things While I resonate with this, because I also went through a period of struggling with async stuff, I genuinely think this is because async is just hard in general. Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. A lot of the async stuff, how it actually worked, and how to actually use it, only clicked for me when I played around with Glommio, which runs an executor-per-core, and some of the constraints it imposed made understanding it all somewhat easier.
  - pdimitar 3y ago
    Thanks a lot of the Glommio mention, that's an instant star and I'll review it in more details Soon™. > Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. Yeah, very well put. Indeed it's very powerful and sure it's confusing. I need guard rails. And I need my hands slapped much more. "Looks like you're trying X -- this is how you do it, you idiot" would work well. :D I am almost not joking even.
rklaehn 3y ago
I think this blog post by tomaka gives a good summary of the current issues with async rust. Certainly much more comprehensive than what you can write in a hacker news post: https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d54d63934a1c https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d... It highlights some of the same problems: - “Just spawn a new task” - The Send trait isn’t what it means anymore
- insanitybit 3y ago
  I guess I just don't run into these issues. Maybe because I write my code to be effectively single threaded already. I almost never actually need a channel, mutex, etc, except in very specific and scoped areas. I wonder why I haven't hit these issues despite writing web services in Rust for years.
  - jacquesm 3y ago
    Sounds to me like you are dealing with this at the process boundary, more like actor and message based software. This is a mature approach and will serve you well. The typical context for stuff like async/await is code that should be running in a separate process so it can block but doesn't. Then it gets ugly fast.
    - insanitybit 3y ago
      Yes, I suspect you're right. I do not like building giant monoliths with internal task systems. I prefer using async/await on the inside and microservices with rpcs or, ideally, streams for communication. I think perhaps people are trying to push too much into a single process?
      - jacquesm 3y ago
        > I think perhaps people are trying to push too much into a single process? That's exactly what's happening. A bit of Erlang/Elixir experience would help a lot of people to create architectures (even in other languages) that do not require such kludges. If your application starts to take on the kind of complexity that you normally deal with a monolithic OS kernel you're doing something terribly wrong.

pdimitar 3y ago

I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to add tasks to an already running pool of tasks and posted my findings here: https://github.com/dimitarvp/rust-async-examples/blob/main/examples/add_workers_during_runtime_1.rs https://github.com/dimitarvp/rust-async-examples/blob/main/e... (there's #2 as well with some more comments and a different approach). The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things with async Rust to me is both a red flag and me being diligent BUT it should be more obvious. And promoted in docs. Rust started suffering from "you got all the nuts and bolts in place, now build your own solution, son" syndrome which I grew to dislike. Too low-level. I wouldn't mind something akin to e.g. Golang's flowmatic library (check the first two examples at the top of the README): https://github.com/carlmjohnson/flowmatic https://github.com/carlmjohnson/flowmatic

insanitybit 3y ago
I'd like to think that I've worked on some "real projects" and I've never run into these issues tbh. It's just hard for me to wrap my mind around. Like I said, I get wanting more libraries (although I think they exist? A local pool exists and you can spawn tasks in it, using tokio) but I'm just not seeing a fundamental problem. I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. But I don't really understand its purpose either.
- pdimitar 3y ago
  > but I'm just not seeing a fundamental problem. Well, I mentioned it a few times in this thread: things are mixed in non-intuitive ways -- async/await, tokio, and various crates (including StreamExt which was not at all obvious and it took me a while to finally find mentioned in a few blog posts and GitHub gists). It just introduces friction and nowadays I am more averse to tech that requires more homework. Admittedly a personal preference of course but I don't think it's an invalid concern either. > I found the code you wrote very confusing so I can see why, when you come back to it, you find it confusing. Well, I needed such a pattern in a project that had to start as quickly as at all possible and only start and schedule the most critical work -- and then start adding all the other work as the first batch of work was already in progress. Would you write it differently and if so, how?
  - insanitybit 3y ago
    > Would you write it differently and if so, how? It's really hard to say because I don't fully know your use case. Maybe your way was the right way but I think I probably would have done something differently. Maybe a priority queue of jobs with a static pool of workers.
    - pdimitar 3y ago
      Yeah, fair enough, I admit I haven't dabbled with any Rust job queues yet.
Nullabillity 3y ago
`flowmatic.Do` is equivalent to `future::try_join` (https://docs.rs/futures/latest/futures/future/fn.try_join.html https://docs.rs/futures/latest/futures/future/fn.try_join.ht...), and `.Each` is equivalent to `stream::iter().map().buffer_unordered()` (https://docs.rs/futures/latest/futures/stream/fn.iter.html https://docs.rs/futures/latest/futures/stream/fn.iter.html, https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.map https://docs.rs/futures/latest/futures/stream/trait.StreamEx..., https://docs.rs/futures/latest/futures/stream/trait.StreamExt.html#method.buffer_unordered https://docs.rs/futures/latest/futures/stream/trait.StreamEx...). `.ManageTasks` doesn't seem to have a clear analogue that I can think of right now, that's an interesting one. `.TaskPool` looks like `StreamExt::buffer_unordered` unless I'm missing something. The code example in the README didn't really say a lot for someone who isn't already well-versed in the library.
- pdimitar 3y ago
  That's on me for making blanket statements -- my apologies. Yep, I know about the first two and I used them regularly. My issue in general is that async Rust involves using several things at the same time (not just the two keywords) and this should be surfaced much earlier in any intro material and the maintainers of the language (or the libraries, or both) should just double down on whatever they feel is the best way to do async Rust. The stance of "you have freedom, make your choice and assemble your own LEGO" is not very productive. I want Rust to start being a bit more opinionated. Though I'll recognize this is just a personal taste but I still have to defend it by saying that as a programmer who is paid to deliver, I want to be able to deliver in predictable timelines and not having to go off on an adventure to learn all the intricacies of async Rust before I am able to write a feature.
FridgeSeal 3y ago
> The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things While I resonate with this, because I also went through a period of struggling with async stuff, I genuinely think this is because async is just hard in general. Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. A lot of the async stuff, how it actually worked, and how to actually use it, only clicked for me when I played around with Glommio, which runs an executor-per-core, and some of the constraints it imposed made understanding it all somewhat easier.
- pdimitar 3y ago
  Thanks a lot of the Glommio mention, that's an instant star and I'll review it in more details Soon™. > Done properly, it yields an immense amount of power, quite efficiently, but also opens up a lot of “degrees of freedom” about it can be operated, which leads to confusion. Yeah, very well put. Indeed it's very powerful and sure it's confusing. I need guard rails. And I need my hands slapped much more. "Looks like you're trying X -- this is how you do it, you idiot" would work well. :D I am almost not joking even.

rklaehn 3y ago

I think this blog post by tomaka gives a good summary of the current issues with async rust. Certainly much more comprehensive than what you can write in a hacker news post: https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d54d63934a1c https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d... It highlights some of the same problems: - “Just spawn a new task” - The Send trait isn’t what it means anymore

insanitybit 3y ago
I guess I just don't run into these issues. Maybe because I write my code to be effectively single threaded already. I almost never actually need a channel, mutex, etc, except in very specific and scoped areas. I wonder why I haven't hit these issues despite writing web services in Rust for years.
- jacquesm 3y ago
  Sounds to me like you are dealing with this at the process boundary, more like actor and message based software. This is a mature approach and will serve you well. The typical context for stuff like async/await is code that should be running in a separate process so it can block but doesn't. Then it gets ugly fast.
  - insanitybit 3y ago
    Yes, I suspect you're right. I do not like building giant monoliths with internal task systems. I prefer using async/await on the inside and microservices with rpcs or, ideally, streams for communication. I think perhaps people are trying to push too much into a single process?
    - jacquesm 3y ago
      > I think perhaps people are trying to push too much into a single process? That's exactly what's happening. A bit of Erlang/Elixir experience would help a lot of people to create architectures (even in other languages) that do not require such kludges. If your application starts to take on the kind of complexity that you normally deal with a monolithic OS kernel you're doing something terribly wrong.