35 ms·

Async Rust doesn't have to be hard

277 points by drogus 4y ago

wolfspaw 4y ago
Great post/points. Really enjoyed your rebuttal. Your version of the dispatcher really shows a simple and intuitive way to code without any explicit lifetimes or zero-alloc-shenanigans.
Hirrolot 4y ago
I am the author of the original post. Unfortunately, before publishing anything, it's very hard to predict all possible misinterpretations of my text. > I really wish the author clearly pointed out that they write the article from a point of view of a library author trying to come up with generic and flexible APIs. Most commentators viewed the text from the perspective of application programming. You are more close to true: I am a library author and the dispatcher example was concerned with the problems of library maintainers. However, I wrote this post mainly to talk about _language design_. Rust is ill-suited for generic `async` programming, because when you enter `async`, you observe that many other language features suddenly break down: references, closures, type system, to name a few. From the perspective of language design, this manifests a failure to design an orthogonal language. I wanted to convey this observation in my post. Additionally, how we write libraries in a given language reveals its true potential, since libraries have to deal with the most generic code, and therefore require more expressive features from language designers. This also affects mundane application programmers though: the more elegant libraries you have, the more easily you can write your application code. Example: language's inexpressiveness doesn't allow you to have a generic runtime interface and change Tokio to something else in one line of code, as we do for loggers. One gentleman also outlined a more comprehensive list of the `async` failures in Rust [1]. This pretty much sums up all the bad things you have to deal with in generic `async` code. UPD: I added an explanation section [2] to my original post. Thank you for your feedback, this is very appreciated. [1] https://www.reddit.com/r/rust/comments/v3cktw/comment/ib0mp49/?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/rust/comments/v3cktw/comment/ib0mp4... [2] https://hirrolot.github.io/posts/rust-is-hard-or-the-misery-of-mainstream-programming.html#update-addressing-misinterpretations https://hirrolot.github.io/posts/rust-is-hard-or-the-misery-...
- dgb23 4y ago
  My intuition is that async is fundamentally the wrong abstraction. First, you want to get rid of function coloring and make coordination itself explicit. Then you build an abstraction over that, where you can declare or infer whether an operation is commutative or associative and generate/select scheduler logic from that. Right?
  - steveklabnik 4y ago
    That is a valid way to design things, as long as it doesn’t clash with other goals. It’s not clear that this is possible given all of the other constraints involved that Rust is attempting to fit together. Doesn’t mean it’s impossible, but the possibility is an open question. This is partially because some of it is subjective! For example, the whole idea that “function coloring” is inherently bad is not a given in a language like Rust. Languages that want to reach down into the lower levels often make costs fairly explicit. Async and sync functions are significantly different, and so some may argue that in a Rust context, this is a good thing, not a bad one. It’s the same idea with values vs references: in many languages, the difference is papered over, not shown to the end user. But in Rust, it is, and this does lead to some ceremony if you want to call a function that takes one with an argument that’s the other. But nobody is arguing that Rust should totally remove this distinction (other than the fact that references are themselves values but that’s not really relevant here…) due to some sort of two colored functions. However, some do want this for mutable vs immutable references. Tl;dr it’s not that simple, in many contexts.
  - throwawaymaths 4y ago
    This is basically what zig does "async", though it's not coordination so much as "context switching control flow points" that are made explicit. It's very low to the metal and if you think about what the machine is "actually doing" it makes a ton of sense, and it's kind of a "shift" in the way of thinking about things in the same way that pointers are. I rather dislike how it's called "async" though. I feel like "reentrant" or "detached" makes more sense, because that's what it feels like to me but if you write your own scheduler you basically have what everyone else calls async. Iirc rust takes an async function and turns it into a abstract state machine, and I feel like this is a surprisingly high level concept for a systems programming language
jph 4y ago
Async does have to be hard, sometimes, at least right now. Iterators, closures, selects, and more are IMHO hard, or absent, or not intuitive. I know these are being worked on-- thank you to the language developers.
- drogus 4y ago
  I think it depends on what you call hard. Things you listed usually make things unergonomic, not necessarily hard. You can still do a lot of stuff with async Rust, but it often requires a lot more boilerplate and compromises. Granted, it can be hard, but I wrote a lot of async code, including async streams, traits, saving `Future`s for later execution etc and usually you don't need anywhere near as much complexity as was presented in the first post.
ntoskrnl 4y ago
I write a decent amount of Rust and I find it productive, but I can see how it might be easy to get nerd-sniped trying to get rid of every last allocation. There's no shame in a Box/Arc. Remember that almost every language puts almost everything on the heap. Just allocate. It'll be fine. Really. I set a rule for myself that I'll spend up to one minute trying to save an allocation. Beyond that it's not worth getting sidetracked.
- lumost 4y ago
  so something I wonder, in a typical language you can easily pass by reference between components and threads. Avoiding any allocations. In rust the unspoken rule seems to be to allocate, allocate, allocate - unless you are writing a special purpose library etc. Which makes me wonder, is rust actually faster than a GC'd language when doing heavy async work?
  - lijogdfljk 4y ago
    Fwiw i rarely allocate around these "issues" and i use all async. I think your comment could be tweaked to say: > In rust the unspoken rule seems to be to allocate, allocate, allocate when you run into a lifetime issue Lifetimes work fine with async, but some types of lifetimes can be problematic, for sure.
  - bluejekyll 4y ago
    This isn’t quite accurate. It’s generally really easy in Rust to pass things by reference, and even inner async fns, this is easy. The issue with async and Futures, is that sometimes you need to capture the future and then pass that to something else to execute. In that context, shared references are hard, and just clone, or arc box like mentioned.
  - coder543 4y ago
    I think a lot of Rust users would argue they don’t even care much about performance. They just enjoy all of the correctness guarantees the compiler can enforce, as well as how ergonomic the language can feel. Being able to deploy a single static binary, and having an easy to use build tool and package manager are significant bonuses as well. Rust makes really hard problems easier when you know the compiler has your back. Rust gives you the tools to write very high performance code, but it doesn’t have to be about that. I’ve had to point out to people quite a few times that garbage collectors can improve performance… especially compared to naive implementations of manual memory management. GCs are not just a tool for lazy programmers. GCs can make allocation incredibly fast, and you get to defer cleanup work to another thread(s), which means less work in the critical path. Removing work from the critical path is how you make software faster. Every tool has tradeoffs, and GCs are a tool. GCs often use more memory as a tradeoff. I like Rust well enough, but I do wish we had a language that combined the ergonomics of Rust with the dead simple concurrency model of Go/Erlang. I haven’t tried it, but Luantic looks promising: https://github.com/lunatic-solutions/lunatic https://github.com/lunatic-solutions/lunatic As it is, we’re fortunate to have quite a few great languages and platforms these days.
dang 4y ago
Recent and related: Rust Is Hard, Or: The Misery of Mainstream Programming - https://news.ycombinator.com/item?id=31601040 https://news.ycombinator.com/item?id=31601040 - June 2022 (655 comments)
- crabbygrabby 4y ago
  If you read the article this post actually links that article and explains that it was written to address it...
  - mkl 4y ago
    But it doesn't link to the HN discussion, which is what dang, HN's moderator, is helpfully doing.
  - dang 4y ago
    Yes, that's why I posted the above. When an article is responding to an article that was recently discussed on HN (or to the HN discussion itself), such a link is particularly relevant.
the__alchemist 4y ago
Pet theory, as someone who falls into the "Loves Rust; avoids Async and generics" alluded to in the beginning of this article, the one it's replying to, and comments on the latter's thread here: Is the Async crew mostly writing web servers and other things that operate using TCP and HTTP? Lower level (eg IP, Eth, network hardware drivers) isn't well supported by rust libs. Nor is higher - we have Flask analogs, but no Django analogs. As Async ingresses in Embedded Rust, I seek answers to "is this worth the viral qualities, and API rift?". For the adjacent question re generics by default, I ask "Is the flexibility and type checking worth the API complexity, and documentation dead-ends?" Does anyone here use Async Rust in domains outside those sections of network and web programming? I've found rust to be a great fit as a cleaner, more explicit C alternative.
- bool3max 4y ago
  How do you manage to avoid generics while writing Rust?
  - whatshisface 4y ago
    You can't avoid using generics, but you can avoid writing generic data structures if the standard library is sufficient and your application won't benefit from de-duplicating code shared between similar structs.
  - the__alchemist 4y ago
    It depends on the use case. The short and simple answer is avoid libs that rely heavily on them, and use structs and enums. This gets into the area of application code vs libraries. A simple example, for say, a struct used to interact with a bus on a MCU is to use a `I2c` struct, instead of `I2c<Output<OpenDrain<PA5<Af5>>>, Output<OpenDrain<PA6<AF5>>>>` etc. God help you if the library that uses the latter doesn't document it using examples, because the auto-generated Rust docs won't help.
  - nyanpasu64 4y ago
    You can't avoid traits and generics entirely, but you can get by with far less than the norm (which is simpler, has some powerful code-reading advantages, and some expressiveness drawbacks). Personally I keep using Vec<T> and other hashmaps, I generally prefer match over .value_or().map(...) functional-style Option/Iterator chaining (though my opinion is unwelcome in some "oh-so-accepting" Rust spaces), prefer type methods over trait methods (which you can't even call unless you `use` the trait into scope), etc. Unfortunately when building generic data structures, you sometimes need complex lifetime and trait Send/Sync/Sized bounds (but I try to switch approaches and sometimes write duplicated code, whenever a particular abstraction approach starts requiring complex HRTBs and such).
olliej 4y ago
I’ve always felt the “if you use Arc” why not just use GC is a weak response. The benefit of rust is that you only pay the cost if you need it. I’m not saying “yay it’s all easy” I found concurrency frustrating in rust because it refused to let me do things that I “knew” we’re safe :) The always use Arc model is actually what swift and objc fundamentally do. Everything’s lifetime has to be threadsafe, so the refcount itself must be threadsafe, and so any refchurn is atomic. For a single thread as I understand it modern CPUs handle uncontended churn without a real perf hit. But I was writing a raytracer in swift, and once I made it multithreaded the refcount cost on my non mutating objects became a massive perf cost. It was super frustrating, and is fundamentally what would happen if you took the “Arc everything” approach. But you don’t have to, and this get perf where it’s safe and possible.
- aaaaaaaaaaab 4y ago
  >But I was writing a raytracer in swift, and once I made it multithreaded the refcount cost on my non mutating objects became a massive perf cost. Then access them through `unowned` references?
  - olliej 4y ago
    Because I was not skilled enough in swift land to know of such. I'll read up and adopt and see what happens, and try to remember to update here :D [edit: Just discovered the last thing I was trying to do was separate out some core data structures into packages. It turns out that if the pop() function on your priority heap gets left out of line, that becomes vastly more expensive than is reasonable. So fixing may take a wee bit long than expected]
    - aaaaaaaaaaab 4y ago
      Priority heap in a raytracer? :thinking_face:
      - olliej 4y ago
        Photon mapping collection :)
daenz 4y ago
I learned Rust by writing a (fuse-based) filesystem [0]...about 30k lines iirc. Rust was challenging, but not crazy difficult (it helps that I have a C++ background). I loved it. Development was slower than a dynamic language, but faster than C++, and most importantly, I felt safe. It's a really solid language. However, when I took a look at async Rust, it really did appear to be a mess. I have substantial experience with Python async/await (which is also a mess), so I'm not unfamiliar with the async concepts. Honestly, I think it's the idea of an event loop in a compiled + non-memory-managed language. You really have to think hard about where objects are living and for how long, and combine that with the illusory world of how async/await appears to work (versus how it actually works), it gets hard to conceptualize. Maybe I just didn't spend enough time with it to feel comfortable with it, but that's my hot take. Imo, Go does performant concurrency right. Rust would be smart to adopt what Go offers. 0. https://github.com/amoffat/supertag https://github.com/amoffat/supertag
- whatshisface 4y ago
  Go has a runtime and a GC, and importantly can implement an event loop outside of anything reached by tracing execution from your code's entry point. Rust's philosophy lead them to make the runtime something you bring in with a library import and start manually. Bundling Tokio with every binary would not be their way, although something like it may one day wind up in the standard library.
  - erikpukinskis 4y ago
    Does Tokio have a concurrency model similar to Go's?
    - oandrew 4y ago
      No, since rust async is stackless. There is a stackful coroutine implementation for Rust: https://github.com/Xudong-Huang/may https://github.com/Xudong-Huang/may
- steveklabnik 4y ago
  Rust cannot copy what Go does without compromising on various language design goals. What Go does is good, but what’s good for Go isn’t always what’s good for Rust. That goes the other way too :) Rust did try to have something closer to Go before 1.0, but it led to so many issues it was removed. Those issues aren’t a problem for Go.
qsdf38100 4y ago
It’s interesting that to criticize rust without being downvoted to death, one must first say how wonderful and superior to c++ it is.
- qsdf38100 4y ago
  - qsdf38100 4y ago
- linkdd 4y ago
  You mean that acknowledging the strengths of a technology and adding constructive criticism is not downvoted while claiming that "it sucks" without any argumentation is? Who would have thought...
  - ncmncm 4y ago
    No. It means that people are still touchy and insecure about Rust and its merits, and (rightfully) worried that being obliged to stand just on its merits it will fizzle. Rust still has a very great deal of evolution to get through before it is ready to compete on an even playing field. It has very little time in which to achieve this, the competition is not sitting still, and the numbers are strongly against it Rust could still become a mainstream language if it changes enough not to repel the majority of professionals who try it out on production-level problems. A clear majority of existing users would object to the needed changes.
    - linkdd 4y ago
      GC-less languages would have to break backward compatibility to implement lifetime/ownership management. Rust's goals do not fit every single problem out there. As the author said in their articles, Rust is not a general purpose language. Rust makes you care about many subjects that are solved in an opiniated way by higher-level languages. It wants you to be explicit about those problematics, it wants you to carefully consider the implications and make your own decisions. Rust made me realize that nothing is trivial: even `printf()` can fail. Certainly, this is not a good fit for the "don't be afraid to break things" philosophy. But when you are in a field where you care about that, and you need the safety, Rust is actually, to my knowledge, the only tool that can help you. Rust is complex, because programming is complex. Other languages are simpler, because the decisions have been made for you, and you can only accept it or use another language. For C and C++, you need extra tools to get to the same level of confidence about your code (just look at the SQLite toolchain/test suite). You can criticize Rust, just like you can criticize anything. You just have to be constructive about it. You'll never see the following comment being upvoted: Rust syntax is rotten garbage, it's unreadable. Why? Because it's subjective, it is definitely not constructive. But the following comment might be upvoted: Rust is a complex language that is difficult to approach for a newbie. Because it acknowledges the problems Rust is trying to solve. None of those hypothetical comments are praising Rust.
jmartin2683 4y ago
Tokio is awesome and very easy to use imho
pie_flavor 4y ago
The problem is essentially that nobody ever evaluates how good their code is, they evaluate how much better it could be. Code that has an Arc<Mutex<>> around every single type in Rust could be made a lot better. Code that does the same in Java could not, because Java doesn't have the ability to make types not wrapped in Arc<Mutex<>>. If you wrote perfect code in Java, it would be equivalent to Arc<Mutex<>> code in Rust - which is not hard to write at all, presenting minimal annoyance. Lest I be lambasted for comparing to a JIT language, that's also how everyone writes C++ these days, and for the same reasons. But that's not how anyone looks at code. They see the ability to unbox types. They see the ability to borrow instead of moving. They see the ability to modify stuff in place instead of making it immutable and then deep-copying it. In most cases the code to do so is shorter and easier. In some other cases, it's longer and more difficult. And people exclusively look at the latter cases, the fact that it's difficult to further improve past what they previously thought of as perfection, and declare Rust to be Hard™, and go back to Java where they can attain perfect code, even if it's worse than the Rust equivalent. Like nerd-sniping, except you do it to yourself.
- jstimpfle 4y ago
  > even if it's worse than the Rust equivalent. If the Java version does the same thing, but with less typing and unnecessary fluff, then it is indeed better.
  - felipellrocha 4y ago
    It isn’t, though. The point is that you *can* write it without thr Arc<Mutex<>> and gain significant speed without it. Even if your first iteration started with it.
    - thaneross 4y ago
      To add to this point, even if you season your Rust code liberally with Arc<Mutex<>>, the libraries you use probably don't, and you'll still get most of the performance benefits.
  - 4y ago
ComputerGuru 4y ago
I can get behind the general recommendations outlined in this article (with the caveat that they're only to be used if you don't need every last drop of performance, you're not writing a library, and you find the current async situation difficult) except for the complete cop-out of implementing each handler routine as not just a function (which could at least be nested/local) but as a completely separate type implementing a trait. That's fine if all your transforms are strictly defined, often reused, and you're just choosing between them but if many of your transforms are just one-offs then that's an insane amount of boilerplate and a very clunky approach. It's also antithesis to the OP's claimed "just get things done" approach since you'll always be second-guessing whether something should be a separate transform type or if it should be extending an existing one, etc.
tus666 4y ago
> So I'll start with a note for all the people intimidated by the techniques the author is trying to use in the post: when writing Rust code you almost never use this kind of stuff Never write async code? Or expect a library will always cover every use case where async might be needed?
aaaaaaaaaaab 4y ago
Given the amount of bikeshedding that went into the design of async Rust, it's mind-boggling how they ended up with this clusterfuck.
RazeLighter777 4y ago
I'm currently writing an async rust application. I think the biggest thing to avoid coding yourself into a corner with async rust is to prefer transferring ownership over using references when possible. Channels are a great way of doing this, and likewise communicating with sync code
rr808 4y ago
Async is a cancer that has spread from single thread languages like JS and Python. Its the wrong solution for multi-thread languages like Rust/Java/C++/Go.
srer 4y ago
Largely complaints of Rust seem to boil down to the programmer needing to describe object lifetime information in code. We can, as the original post did, show approaches that are overwhelmingly difficult in Rust because of this but trivial in Go. Alternative approaches, as this post shows, can be relatively straightforward. In a similar vein, a Python programmer might complain about having to explicitly describe object type information in Go code. One supposes they could show approaches that are overwhelmingly difficult in Go, but trivial in Python. Python, Go and Rust programs all do have types and object lifetimes. It is just that mistakes in type are not found until runtime in Python, and likewise mistakes in lifetime are not found until runtime in Python and Go. Personally, after years of Python I came to value describing types in code, and after years of Go I came to value describing lifetimes in code too.
beebmam 4y ago
Is there much support for debugging async Rust functions? I'm not able to get breakpoints to trigger in async Rust functions when using lldb. Am I missing something? This seems like a REALLY broken experience.

277 points by drogus 4y ago

Great post/points. Really enjoyed your rebuttal. Your version of the dispatcher really shows a simple and intuitive way to code without any explicit lifetimes or zero-alloc-shenanigans.

Hirrolot 4y ago

I am the author of the original post. Unfortunately, before publishing anything, it's very hard to predict all possible misinterpretations of my text. > I really wish the author clearly pointed out that they write the article from a point of view of a library author trying to come up with generic and flexible APIs. Most commentators viewed the text from the perspective of application programming. You are more close to true: I am a library author and the dispatcher example was concerned with the problems of library maintainers. However, I wrote this post mainly to talk about _language design_. Rust is ill-suited for generic `async` programming, because when you enter `async`, you observe that many other language features suddenly break down: references, closures, type system, to name a few. From the perspective of language design, this manifests a failure to design an orthogonal language. I wanted to convey this observation in my post. Additionally, how we write libraries in a given language reveals its true potential, since libraries have to deal with the most generic code, and therefore require more expressive features from language designers. This also affects mundane application programmers though: the more elegant libraries you have, the more easily you can write your application code. Example: language's inexpressiveness doesn't allow you to have a generic runtime interface and change Tokio to something else in one line of code, as we do for loggers. One gentleman also outlined a more comprehensive list of the `async` failures in Rust [1]. This pretty much sums up all the bad things you have to deal with in generic `async` code. UPD: I added an explanation section [2] to my original post. Thank you for your feedback, this is very appreciated. [1] https://www.reddit.com/r/rust/comments/v3cktw/comment/ib0mp49/?utm_source=share&utm_medium=web2x&context=3 https://www.reddit.com/r/rust/comments/v3cktw/comment/ib0mp4... [2] https://hirrolot.github.io/posts/rust-is-hard-or-the-misery-of-mainstream-programming.html#update-addressing-misinterpretations https://hirrolot.github.io/posts/rust-is-hard-or-the-misery-...

dgb23 4y ago
My intuition is that async is fundamentally the wrong abstraction. First, you want to get rid of function coloring and make coordination itself explicit. Then you build an abstraction over that, where you can declare or infer whether an operation is commutative or associative and generate/select scheduler logic from that. Right?
- steveklabnik 4y ago
  That is a valid way to design things, as long as it doesn’t clash with other goals. It’s not clear that this is possible given all of the other constraints involved that Rust is attempting to fit together. Doesn’t mean it’s impossible, but the possibility is an open question. This is partially because some of it is subjective! For example, the whole idea that “function coloring” is inherently bad is not a given in a language like Rust. Languages that want to reach down into the lower levels often make costs fairly explicit. Async and sync functions are significantly different, and so some may argue that in a Rust context, this is a good thing, not a bad one. It’s the same idea with values vs references: in many languages, the difference is papered over, not shown to the end user. But in Rust, it is, and this does lead to some ceremony if you want to call a function that takes one with an argument that’s the other. But nobody is arguing that Rust should totally remove this distinction (other than the fact that references are themselves values but that’s not really relevant here…) due to some sort of two colored functions. However, some do want this for mutable vs immutable references. Tl;dr it’s not that simple, in many contexts.
- throwawaymaths 4y ago
  This is basically what zig does "async", though it's not coordination so much as "context switching control flow points" that are made explicit. It's very low to the metal and if you think about what the machine is "actually doing" it makes a ton of sense, and it's kind of a "shift" in the way of thinking about things in the same way that pointers are. I rather dislike how it's called "async" though. I feel like "reentrant" or "detached" makes more sense, because that's what it feels like to me but if you write your own scheduler you basically have what everyone else calls async. Iirc rust takes an async function and turns it into a abstract state machine, and I feel like this is a surprisingly high level concept for a systems programming language

jph 4y ago

Async does have to be hard, sometimes, at least right now. Iterators, closures, selects, and more are IMHO hard, or absent, or not intuitive. I know these are being worked on-- thank you to the language developers.

drogus 4y ago
I think it depends on what you call hard. Things you listed usually make things unergonomic, not necessarily hard. You can still do a lot of stuff with async Rust, but it often requires a lot more boilerplate and compromises. Granted, it can be hard, but I wrote a lot of async code, including async streams, traits, saving `Future`s for later execution etc and usually you don't need anywhere near as much complexity as was presented in the first post.

ntoskrnl 4y ago

I write a decent amount of Rust and I find it productive, but I can see how it might be easy to get nerd-sniped trying to get rid of every last allocation. There's no shame in a Box/Arc. Remember that almost every language puts almost everything on the heap. Just allocate. It'll be fine. Really. I set a rule for myself that I'll spend up to one minute trying to save an allocation. Beyond that it's not worth getting sidetracked.

lumost 4y ago
so something I wonder, in a typical language you can easily pass by reference between components and threads. Avoiding any allocations. In rust the unspoken rule seems to be to allocate, allocate, allocate - unless you are writing a special purpose library etc. Which makes me wonder, is rust actually faster than a GC'd language when doing heavy async work?
- lijogdfljk 4y ago
  Fwiw i rarely allocate around these "issues" and i use all async. I think your comment could be tweaked to say: > In rust the unspoken rule seems to be to allocate, allocate, allocate when you run into a lifetime issue Lifetimes work fine with async, but some types of lifetimes can be problematic, for sure.
- bluejekyll 4y ago
  This isn’t quite accurate. It’s generally really easy in Rust to pass things by reference, and even inner async fns, this is easy. The issue with async and Futures, is that sometimes you need to capture the future and then pass that to something else to execute. In that context, shared references are hard, and just clone, or arc box like mentioned.
- coder543 4y ago
  I think a lot of Rust users would argue they don’t even care much about performance. They just enjoy all of the correctness guarantees the compiler can enforce, as well as how ergonomic the language can feel. Being able to deploy a single static binary, and having an easy to use build tool and package manager are significant bonuses as well. Rust makes really hard problems easier when you know the compiler has your back. Rust gives you the tools to write very high performance code, but it doesn’t have to be about that. I’ve had to point out to people quite a few times that garbage collectors can improve performance… especially compared to naive implementations of manual memory management. GCs are not just a tool for lazy programmers. GCs can make allocation incredibly fast, and you get to defer cleanup work to another thread(s), which means less work in the critical path. Removing work from the critical path is how you make software faster. Every tool has tradeoffs, and GCs are a tool. GCs often use more memory as a tradeoff. I like Rust well enough, but I do wish we had a language that combined the ergonomics of Rust with the dead simple concurrency model of Go/Erlang. I haven’t tried it, but Luantic looks promising: https://github.com/lunatic-solutions/lunatic https://github.com/lunatic-solutions/lunatic As it is, we’re fortunate to have quite a few great languages and platforms these days.

dang 4y ago

Recent and related: Rust Is Hard, Or: The Misery of Mainstream Programming - https://news.ycombinator.com/item?id=31601040 https://news.ycombinator.com/item?id=31601040 - June 2022 (655 comments)

crabbygrabby 4y ago
If you read the article this post actually links that article and explains that it was written to address it...
- mkl 4y ago
  But it doesn't link to the HN discussion, which is what dang, HN's moderator, is helpfully doing.
- dang 4y ago
  Yes, that's why I posted the above. When an article is responding to an article that was recently discussed on HN (or to the HN discussion itself), such a link is particularly relevant.

the__alchemist 4y ago

Pet theory, as someone who falls into the "Loves Rust; avoids Async and generics" alluded to in the beginning of this article, the one it's replying to, and comments on the latter's thread here: Is the Async crew mostly writing web servers and other things that operate using TCP and HTTP? Lower level (eg IP, Eth, network hardware drivers) isn't well supported by rust libs. Nor is higher - we have Flask analogs, but no Django analogs. As Async ingresses in Embedded Rust, I seek answers to "is this worth the viral qualities, and API rift?". For the adjacent question re generics by default, I ask "Is the flexibility and type checking worth the API complexity, and documentation dead-ends?" Does anyone here use Async Rust in domains outside those sections of network and web programming? I've found rust to be a great fit as a cleaner, more explicit C alternative.

bool3max 4y ago
How do you manage to avoid generics while writing Rust?
- whatshisface 4y ago
  You can't avoid using generics, but you can avoid writing generic data structures if the standard library is sufficient and your application won't benefit from de-duplicating code shared between similar structs.
- the__alchemist 4y ago
  It depends on the use case. The short and simple answer is avoid libs that rely heavily on them, and use structs and enums. This gets into the area of application code vs libraries. A simple example, for say, a struct used to interact with a bus on a MCU is to use a `I2c` struct, instead of `I2c<Output<OpenDrain<PA5<Af5>>>, Output<OpenDrain<PA6<AF5>>>>` etc. God help you if the library that uses the latter doesn't document it using examples, because the auto-generated Rust docs won't help.
- nyanpasu64 4y ago
  You can't avoid traits and generics entirely, but you can get by with far less than the norm (which is simpler, has some powerful code-reading advantages, and some expressiveness drawbacks). Personally I keep using Vec<T> and other hashmaps, I generally prefer match over .value_or().map(...) functional-style Option/Iterator chaining (though my opinion is unwelcome in some "oh-so-accepting" Rust spaces), prefer type methods over trait methods (which you can't even call unless you `use` the trait into scope), etc. Unfortunately when building generic data structures, you sometimes need complex lifetime and trait Send/Sync/Sized bounds (but I try to switch approaches and sometimes write duplicated code, whenever a particular abstraction approach starts requiring complex HRTBs and such).

olliej 4y ago

I’ve always felt the “if you use Arc” why not just use GC is a weak response. The benefit of rust is that you only pay the cost if you need it. I’m not saying “yay it’s all easy” I found concurrency frustrating in rust because it refused to let me do things that I “knew” we’re safe :) The always use Arc model is actually what swift and objc fundamentally do. Everything’s lifetime has to be threadsafe, so the refcount itself must be threadsafe, and so any refchurn is atomic. For a single thread as I understand it modern CPUs handle uncontended churn without a real perf hit. But I was writing a raytracer in swift, and once I made it multithreaded the refcount cost on my non mutating objects became a massive perf cost. It was super frustrating, and is fundamentally what would happen if you took the “Arc everything” approach. But you don’t have to, and this get perf where it’s safe and possible.

aaaaaaaaaaab 4y ago
>But I was writing a raytracer in swift, and once I made it multithreaded the refcount cost on my non mutating objects became a massive perf cost. Then access them through `unowned` references?
- olliej 4y ago
  Because I was not skilled enough in swift land to know of such. I'll read up and adopt and see what happens, and try to remember to update here :D [edit: Just discovered the last thing I was trying to do was separate out some core data structures into packages. It turns out that if the pop() function on your priority heap gets left out of line, that becomes vastly more expensive than is reasonable. So fixing may take a wee bit long than expected]
  - aaaaaaaaaaab 4y ago
    Priority heap in a raytracer? :thinking_face:
    - olliej 4y ago
      Photon mapping collection :)

daenz 4y ago

I learned Rust by writing a (fuse-based) filesystem [0]...about 30k lines iirc. Rust was challenging, but not crazy difficult (it helps that I have a C++ background). I loved it. Development was slower than a dynamic language, but faster than C++, and most importantly, I felt safe. It's a really solid language. However, when I took a look at async Rust, it really did appear to be a mess. I have substantial experience with Python async/await (which is also a mess), so I'm not unfamiliar with the async concepts. Honestly, I think it's the idea of an event loop in a compiled + non-memory-managed language. You really have to think hard about where objects are living and for how long, and combine that with the illusory world of how async/await appears to work (versus how it actually works), it gets hard to conceptualize. Maybe I just didn't spend enough time with it to feel comfortable with it, but that's my hot take. Imo, Go does performant concurrency right. Rust would be smart to adopt what Go offers. 0. https://github.com/amoffat/supertag https://github.com/amoffat/supertag

whatshisface 4y ago
Go has a runtime and a GC, and importantly can implement an event loop outside of anything reached by tracing execution from your code's entry point. Rust's philosophy lead them to make the runtime something you bring in with a library import and start manually. Bundling Tokio with every binary would not be their way, although something like it may one day wind up in the standard library.
- erikpukinskis 4y ago
  Does Tokio have a concurrency model similar to Go's?
  - oandrew 4y ago
    No, since rust async is stackless. There is a stackful coroutine implementation for Rust: https://github.com/Xudong-Huang/may https://github.com/Xudong-Huang/may
steveklabnik 4y ago
Rust cannot copy what Go does without compromising on various language design goals. What Go does is good, but what’s good for Go isn’t always what’s good for Rust. That goes the other way too :) Rust did try to have something closer to Go before 1.0, but it led to so many issues it was removed. Those issues aren’t a problem for Go.

qsdf38100 4y ago

It’s interesting that to criticize rust without being downvoted to death, one must first say how wonderful and superior to c++ it is.

qsdf38100 4y ago
- qsdf38100 4y ago
linkdd 4y ago
You mean that acknowledging the strengths of a technology and adding constructive criticism is not downvoted while claiming that "it sucks" without any argumentation is? Who would have thought...
- ncmncm 4y ago
  No. It means that people are still touchy and insecure about Rust and its merits, and (rightfully) worried that being obliged to stand just on its merits it will fizzle. Rust still has a very great deal of evolution to get through before it is ready to compete on an even playing field. It has very little time in which to achieve this, the competition is not sitting still, and the numbers are strongly against it Rust could still become a mainstream language if it changes enough not to repel the majority of professionals who try it out on production-level problems. A clear majority of existing users would object to the needed changes.
  - linkdd 4y ago
    GC-less languages would have to break backward compatibility to implement lifetime/ownership management. Rust's goals do not fit every single problem out there. As the author said in their articles, Rust is not a general purpose language. Rust makes you care about many subjects that are solved in an opiniated way by higher-level languages. It wants you to be explicit about those problematics, it wants you to carefully consider the implications and make your own decisions. Rust made me realize that nothing is trivial: even `printf()` can fail. Certainly, this is not a good fit for the "don't be afraid to break things" philosophy. But when you are in a field where you care about that, and you need the safety, Rust is actually, to my knowledge, the only tool that can help you. Rust is complex, because programming is complex. Other languages are simpler, because the decisions have been made for you, and you can only accept it or use another language. For C and C++, you need extra tools to get to the same level of confidence about your code (just look at the SQLite toolchain/test suite). You can criticize Rust, just like you can criticize anything. You just have to be constructive about it. You'll never see the following comment being upvoted: Rust syntax is rotten garbage, it's unreadable. Why? Because it's subjective, it is definitely not constructive. But the following comment might be upvoted: Rust is a complex language that is difficult to approach for a newbie. Because it acknowledges the problems Rust is trying to solve. None of those hypothetical comments are praising Rust.

jmartin2683 4y ago

Tokio is awesome and very easy to use imho

pie_flavor 4y ago

The problem is essentially that nobody ever evaluates how good their code is, they evaluate how much better it could be. Code that has an Arc<Mutex<>> around every single type in Rust could be made a lot better. Code that does the same in Java could not, because Java doesn't have the ability to make types not wrapped in Arc<Mutex<>>. If you wrote perfect code in Java, it would be equivalent to Arc<Mutex<>> code in Rust - which is not hard to write at all, presenting minimal annoyance. Lest I be lambasted for comparing to a JIT language, that's also how everyone writes C++ these days, and for the same reasons. But that's not how anyone looks at code. They see the ability to unbox types. They see the ability to borrow instead of moving. They see the ability to modify stuff in place instead of making it immutable and then deep-copying it. In most cases the code to do so is shorter and easier. In some other cases, it's longer and more difficult. And people exclusively look at the latter cases, the fact that it's difficult to further improve past what they previously thought of as perfection, and declare Rust to be Hard™, and go back to Java where they can attain perfect code, even if it's worse than the Rust equivalent. Like nerd-sniping, except you do it to yourself.

jstimpfle 4y ago
> even if it's worse than the Rust equivalent. If the Java version does the same thing, but with less typing and unnecessary fluff, then it is indeed better.
- felipellrocha 4y ago
  It isn’t, though. The point is that you *can* write it without thr Arc<Mutex<>> and gain significant speed without it. Even if your first iteration started with it.
  - thaneross 4y ago
    To add to this point, even if you season your Rust code liberally with Arc<Mutex<>>, the libraries you use probably don't, and you'll still get most of the performance benefits.
- 4y ago

ComputerGuru 4y ago

I can get behind the general recommendations outlined in this article (with the caveat that they're only to be used if you don't need every last drop of performance, you're not writing a library, and you find the current async situation difficult) except for the complete cop-out of implementing each handler routine as not just a function (which could at least be nested/local) but as a completely separate type implementing a trait. That's fine if all your transforms are strictly defined, often reused, and you're just choosing between them but if many of your transforms are just one-offs then that's an insane amount of boilerplate and a very clunky approach. It's also antithesis to the OP's claimed "just get things done" approach since you'll always be second-guessing whether something should be a separate transform type or if it should be extending an existing one, etc.

tus666 4y ago

> So I'll start with a note for all the people intimidated by the techniques the author is trying to use in the post: when writing Rust code you almost never use this kind of stuff Never write async code? Or expect a library will always cover every use case where async might be needed?

aaaaaaaaaaab 4y ago

Given the amount of bikeshedding that went into the design of async Rust, it's mind-boggling how they ended up with this clusterfuck.

RazeLighter777 4y ago

I'm currently writing an async rust application. I think the biggest thing to avoid coding yourself into a corner with async rust is to prefer transferring ownership over using references when possible. Channels are a great way of doing this, and likewise communicating with sync code

rr808 4y ago

Async is a cancer that has spread from single thread languages like JS and Python. Its the wrong solution for multi-thread languages like Rust/Java/C++/Go.

srer 4y ago

Largely complaints of Rust seem to boil down to the programmer needing to describe object lifetime information in code. We can, as the original post did, show approaches that are overwhelmingly difficult in Rust because of this but trivial in Go. Alternative approaches, as this post shows, can be relatively straightforward. In a similar vein, a Python programmer might complain about having to explicitly describe object type information in Go code. One supposes they could show approaches that are overwhelmingly difficult in Go, but trivial in Python. Python, Go and Rust programs all do have types and object lifetimes. It is just that mistakes in type are not found until runtime in Python, and likewise mistakes in lifetime are not found until runtime in Python and Go. Personally, after years of Python I came to value describing types in code, and after years of Go I came to value describing lifetimes in code too.

beebmam 4y ago

Is there much support for debugging async Rust functions? I'm not able to get breakpoints to trigger in async Rust functions when using lldb. Am I missing something? This seems like a REALLY broken experience.