17 ms·

The article glosses over async Rust and is mostly a rant about how closures are difficult in Rust. Most of the difficulty comes from Rust not having a GC yet w

• scotty79 5y ago
  From my very limited expeirience with Rust I noticed that it becomes way more easy and laid back language when you just skip using references and lifetimes nearly completely and just wrap everything in Rc<>. Then you are getting expeirience of fairly high level language with a lot of very cool constructs and features like exhaustive pattern matching and value types with a lot of auto-derived functionality. Does Rc<> help this much with async as well?
  • mrfox321 5y ago
    If you are going to wrap everything in Rc, just use kotlin or c-sharp or python?
    • 5e92cb50239222b 5y ago
      And suffer a 100 MB runtime? If anything, Rust with refcounts for everything is much closer to Swift.
      • mrfox321 5y ago
        Ah didnt realize swift has no runtime. Thanks!
        • ssokolow 5y ago
          As I remember, it uses an evolution of Objective-C's Automatic Reference Counting feature.
        • pjmlp 5y ago
          Every language has a runtime, it is what supports the basic language infrastructure, including Swift. https://github.com/apple/swift/blob/main/docs/Runtime.md https://github.com/apple/swift/blob/main/docs/Runtime.md
          • deleted 5y ago
            [deleted]
    • scotty79 5y ago
      Because Rust feels even nicer than Python and C# (I don't have any experience with kotlin but from what I've seen it's as decent as Python at least) and even with Rc<> Rust is quite light-weight and fast. One of its selling points is zero-cost abstractions and pretty powerful abstractions at that.
    • CodesInChaos 5y ago
      Rust has advantages over C#, even for high level programming: * Rust traits are more flexible than C# interfaces, especially when combined with generics (implementing traits for foreign types, associated types, each method can have its own constraints, conditional trait implementation, #derive) * Rust has much stronger thread safety guarantees (absence of data races, preventing access to a mutex's data without locking) * Options are cleaner than null (though at least C# supports non-nullable reference types nowadays) * Cleaner error handling through `Result`s. In C# you either have to use exceptions (discouraged for errors that are expected to happen), or use ad-hoc approaches which don't benefit from syntax sugar like the `?` operator. * Support for discriminated unions ("enums") * I like cargo better than msbuild+nuget ("it just works", feature-flags + conditional dependencies, distribution as source instead of binaries) On the other hand C# has better IDEs and you don't have to deal with the rigors of ownership and the borrow checker. In particular I struggle with LINQ style functional programming in Rust, since the lifetimes of closures and the fields they close over are difficult to reason about. I haven't used Kotlin, but expect it to be similar to C# in its strengths and weaknesses. Python is not an option for me, since I like static typing.
      • hailwren 5y ago
        I haven't used C#, so I can't comment on the IDEs there. However, it's surprising to me that you can get a better experience than VScode + rust-analyzer. It feels like magic when I use it.
      • quotemstr 5y ago
        I don't agree that people should avoid exceptions for expected errors. The idea that they should has twisted the error handling landscape into a pretzel and is responsible for some ugly bits of Rust design.
        • tialaramex 5y ago
          Exceptions are a mechanism for error handling that comes with control flow changes. But Rust already has perfectly nice control flow mechanisms (look at std::ops::ControlFlow for example) and perfectly nice error handling. So you can use either, or both, as necessary, you don't need this particular Frankenstein's Monster. Using Exceptions for things that aren't actually exceptional is perverse.
    • marcus_cemes 5y ago
      Correct me if I'm wrong, but: - GC pauses, Rc<T> does not, it's simply a deallocation by the last reference holder - Rc<T> still has predictable memory allocation/deallocation, GC is not guaranteed to run until needed - More efficient memory use, no need to keep track of allocated objects - The rest of the Rust language is a pleasure to use (imo) - Python is slow for certain applications, and not concurrent - Kotlin is kind of more niche than Rust, it's very popular in the Android community, like C# is popular on Windows Just my 2 cents, I love progamming in all languages. There are some use-cases where a high level language is absolutely the way to go. For others, Rust provides much more control with a handy escape hatch. Absolutely don't go wrapping everything in Rc<T> like a madman, only when the reduced complexity is more beneficial than dealing with references/lifetimes.
      • Quekid5 5y ago
        > GC pauses, Rc<T> does not, it's simply a deallocation by the last reference holder While the last bit is true, it may actually end up having to do lots of work. Think large object graphs where the last reference to any of it goes out of scope. Also, as a matter of terminology: Rc is a form of garbage collection. It's also a pretty bad general GC strategy at that -- which is why one doesn't just slap an Rc on everything.
      • throwaway894345 5y ago
        > GC pauses, Rc<T> does not, it's simply a deallocation by the last reference holder There are pauseless GCs (e.g., Azul for JVM), and Go kicked off a trend of super-low-latency GC. Also, RC deallocation is O(N) while GC is usually O(1) (ignoring pedantry about how RC is a type of GC). Further, RC can't handle cycles automatically. > Rc<T> still has predictable memory allocation/deallocation, GC is not guaranteed to run until needed Deterministic GC exists, but admittedly isn't widespread. For most non-critical real time systems (e.g., video games) a low latency GC is probably sufficient. > More efficient memory use, no need to keep track of allocated objects I'm not sure if this is true? Presumably each RC has an int for its reference counter? I'm not sure what the bookkeeping overhead is for tracing GCs, but I'm guessing it's not O(N)? > The rest of the Rust language is a pleasure to use (imo) Agreed, but the borrow checker affects everything so this is a pretty small consolation in practice.
    • throwaway894345 5y ago
      While I understand that with Graal and .Net you can ostensibly make native, static binaries for Kotlin or C#, I'm very skeptical that it works well in practice. In particular, I'm guessing it will feel like swimming upstream, fighting an ecosystem and build tooling which mostly assume you're running on a VM. And then there's the question of performance... And of course, with Python your code will run 100x slower than either of the above, your dependency management will suck, you won't be able to statically compile, and to top it all off everyone will give you recommendations for your ailments which take a long time to try out and inevitably will fail miserably for one glaring reason or another. By way of example: "just rewrite the slow bits in C" -> you rewrite the slow bits in C -> your code is now slower because the marshaling costs exceed the gains + your build system is dramatically more complex and you have the sheer joy of debugging segfaults and undefined behavior (yeah, I know Cython exists).
      • thinkharderdev 5y ago
        With the modern JVM you actually have to work quite hard to write native code (Rust/C++/C) that outperforms an equivalent Java/Kotlin/Scala implementation. And it is quite easy to perform worse with a native implementation. Of course that is subject to various caveats: 1. Anything running on the JVM will need a 50-500ms of startup time. 2. The JVM implementation will not reach max performance until the runtime has optimized and JIT'd the relevant code paths. 3. There is memory overhead of the VM itself so your runtime will (all else equal) probably require more memory. If you do need to use Graal to generate native images though it can actually be quite nice. You can run locally (or in benchmarking environments) on the VM and get all the tooling and metrics that come along with that, but build a native binary to actually deploy. I agree that it can be kind of a pain though.
        • jandrewrogers 5y ago
          > With the modern JVM you actually have to work quite hard to write native code (Rust/C++/C) that outperforms an equivalent Java/Kotlin/Scala implementation. For what kind of code? I've never experienced comparable performance between JVM and C++ implementations, and I've had the misfortune of writing a couple parallel implementations in recent years where it was literally an "apples to apples" comparison. C++ is faster by default and isn't particularly close, even if you ignore Java's startup and warmup time. I've never seen a native implementation run slower than a JVM one in my entire career, which has included a lot of Java. This is the expected outcome and easily explainable in technical terms. Performance in modern systems is dominated by memory handling efficiency, where C++ is very strong and JVM is not.
  • spoiler 5y ago
    You probably need Arc<> in the context of async, but if it's a normal closure, Rc<> will do. However, that's not needed always. You largely wanna move stuff into closures in my experience, since the closure ends up owning the data. Arc comes with a small performance overhead (Rc too, but less so I think?), so you don't want to use it all the time. When using Arc, you'll usually need a Mutex too, which adds its own cost, etc. There are ways to partially avoid this costs, but they can add complexity (eg channels). Sometimes you might be able to use an Async-flavor of the synchronization types instead, which come with more acceptable performance compromises in Async contexts.
  • a_rustacean 5y ago
    `Rc` freezes the value as long as it's shared unless you use interior mutability. It's fine when you want to share immutable data, but if you ever need to mutate its contents you will have to deal with awkward cases and situations, and at runtime! Wrapping everything in `Rc` is not a good default strategy. Instead figure out an architecture that works well with the borrow checker! This normally means thinking hard about your data structures and the ownership model of your application. I love Rust precisely because of this. You can leverage the compiler to drive your development and design. If it's hard, then it's probably wrong!
    • hailwren 5y ago
      Are there any good resources for design patterns in Rust? I've gone through Rust by example and am writing mid-size rust programs now but would love to check my work against the experts.
      • ssokolow 5y ago
        There's The Unofficial Rust Design Patterns Book... https://rust-unofficial.github.io/patterns/ https://rust-unofficial.github.io/patterns/ (More generally, there's the Little Book of Rust Books to find things like that: https://lborb.github.io/book/ https://lborb.github.io/book/ ) ...and someone asked about good code to learn from on Reddit a day or two ago and was advised to read basically any code by dtolnay or burntsushi, two of the big wizards who feel like they've written half the Rust ecosystem at times.
      • mgaunard 5y ago
        Rust also just enforces good C++ practices, so you could just get yourself familiar with the 40 years history of C++.
    • dan00 5y ago
      > I love Rust precisely because of this. You can leverage the compiler to drive your development and design. If it's hard, then it's probably wrong! Yes, I had the same experience writing code in Rust and Haskell. If the design works nicely with the language, then also the design is better, more modular, simpler dependencies and easier to understand and extend.
  • dgb23 5y ago
    Then you are essentially introducing a verbose, unoptimized garbage collector?
    • throwaway894345 5y ago
      And you still run into all sorts of ergonomic issues when you inevitably need to dereference your Rc<> pointer. That said, I sympathize with the parent's desire for a proper Rust-lite with a GC: C-family syntax, great tooling, great documentation, great standard library and ecosystem, native+static compilation by default. Of course, someone will ignore those criteria and come in suggesting OCaml/Reason...
      • native_samples 5y ago
        Take a look at Kotlin Native. It compiles to native code, the standard library is pretty good, and the latest versions have introduced a GC so it's now much easier to convert Java to it (Java can be rewritten into Kotlin automatically at the source level). However I'd just use Graal native image. Most libraries work and it's far easier to write a config or tweak one that doesn't, than not have access to the library at all!
      • xpressvideoz 5y ago
        > Of course, someone will ignore those criteria and come in suggesting OCaml/Reason... I hoped Go to be such a language, but it failed to fulfill my needs by throwing away all the PL knowledge that humanity has accumulated for decades. I still mourn for the missed opportunity by Google.
        • e12e 5y ago
          Have you looked at D? (or perhaps zig - but I think D would be closer to a "sane" go).
        • throwaway894345 5y ago
          I considered mentioning "Go", and while it would be nice to have a Go with generics + sum types, it's also very nice having Go as it is today. In other words, it would be nice to have a "Go with those things" and a "Go without them" (predictable rebuttal: "But if Go supports those things, you would only have to use them when you wanted to!" <- not true, because in the real world we have to use the libraries available to us). Frankly though, the type system is absurdly overemphasized. Squabbling over type systems is penny wise and pound foolish if you're missing the fundamentals. Allow me to quote myself from a thread a couple of days ago: > Agreed. It still surprises me that so many other languages fail at the fundamentals (minimal learning curve, static binaries, fast builds, reproducible dependency management, great tooling, great stdlib + ecosystem, etc) and yet many devotees of those languages have positively hyperventilated about Go's error handling and type system for 12 years. Go is finally getting generics and I'm sort of cautiously excited about it (like I was when Apple added the TouchBar to MacBook Pros), but any net benefit is going to be positively negligible in comparison to the degree in which Go raised the bar on the fundamentals. https://news.ycombinator.com/item?id=29179681 https://news.ycombinator.com/item?id=29179681
  • sweeneyrod 5y ago
    You can get those cool features in languages such as OCaml/F#/Scala/Haskell, and then you don't have to worry about managing memory because you have a GC.
    • scotty79 5y ago
      Sure but all of them are weird. Rust is something you can pick up in an afternoon if you have expeirience in C++-like languages.
      • pjmlp 5y ago
        From the point of view of somehow that has taught C++ as TA, worked with C and C++ during several years in some heavy contexts, an afternoon won't do it.
  • mgaunard 5y ago
    Reference counting is mostly a cop-out for people who can't design code properly and reason about ownership and lifetimes. It essentially makes your code Java and defeats the point of the language. I would expect most decent teams to reject your code during code reviews.
• Fiahil 5y ago
  The article is also conflating synchronous single-threaded, synchronous multi-threaded and asynchronous programming. Each have their own usage, and no, a multi-threaded program is not the same as an asynchronous one. For example, using threads and channels instead of async/await is not a design flaw if your workload is mostly about large, blocking computations on a read-only shared state with no I/O. In that situation, lifetime and closures will not pose any issue and won't get in the way because you're not mutating anything and communicate with owned messages. The async ecosystem is evolving and will be better tomorrow than it is today. Saying that Rust programming as a whole is a mess because async/await is harder to use than necessary, is short-sighted. It's like saying you don't like apples, but you only chewed on the branch of the tree.
  • convolvatron 5y ago
    its really strange that there are two languages running around together. one which is very opinionated in how to manage memory in a stack discipline and another which just uses reference counts. they don't quite mix. so you need to be aware of which one you're (implicitly using), and you may need library functions for both colors. you have to admit this adds some additional mental overhead. but what got me when trying to understand the language was just that there _was_ such a division. I actually tried to apply the lifetime model to asynch objects.
    • md8z 5y ago
      IME, using reference counts there is also not a good idea. In practice I think it's better to treat async tasks just like goroutines and then use channels to communicate.
      • convolvatron 5y ago
        I've found that when I've structured things that way in the past I do have some performance issues if the grain is very small. more importantly to me is that those machines dont really compose that well over larger scales - that is to add a new component I need to look at the overall scheduling and connectedness of the existing components and need to rework them to order to handle a new control flow. it is a nice model. i think it isn't appropriate for some things, so I tend not to reach for it by default.
• skohan 5y ago
  I think it's not entirely fair to paint problems with rust's async features as an aversion to low-level-ness. If anything, I think the issue with Rust's async is that it tries to be too high level. In my experience with async rust, most of the difficulty comes from "spooky errors at a distance". I.e, you're writing some code which feels completely normal, and then suddenly you are hit with a large, obtuse error message about types and traits you didn't even know you were working with. The reason being, something went wrong with all the inference involved in implicitly wrapping everything in futures. And often it's not even related to the line of code you just wrote. So in these cases, I think the issue is that rust tries very hard to wrap async in a very smooth, sugared syntax. It works fine in a lot of cases, but due to the precise nature of Rust, there are a cases where it doesn't work. And because the system relies on a lot of magic inference, the programmer has to hold a lot of assumptions and rules in their mind to work effectively with async. I am not an expert, but from my experience it feels like async rust was a bit rushed. It should have been introduced as a more explicit syntax first, with sugar on top once the guard rails had better been figured out through experimentation. That is under the assumption that future-based async in rust is a good idea at all. Rust's ownership model becomes intuitive very quickly in the single-threaded case, but at times it feels like a square peg in a round hole where async is concerned.
  • gameswithgo 5y ago
    This may be a problem with async generally unless the language is designed specifically around async like Go is, which some profound tradeoffs to make it happen. (not criticizing that I think Go did a great job)
    • skohan 5y ago
      I don't disagree. After working with Rust's model, which forces you to confront a lot of the tradeoffs and complexity, I'm more inclined to think async is a feature which should really be considered from the ground up when a language is being designed, to avoid painting oneself into a corner in the design space.
      • 5y ago
• Lhiw 5y ago
  > With that in mind, I see this article as not understanding the goals and tradeoffs of Rust. The author would be happier writing in a higher-level language than Rust. This is a really unfair call to make. The author rightfully so goes to great lengths to avoid talking about those trade offs, specifically because they're irrelevant. Async in rust is hard is the complaint. It's not suggesting there aren't good technical reasons for it. Just that maybe implementing something that turned out to make a mess in the end wasn't the right move. Instead, maybe other options should have been explored. The article listed what, 3 different ways to do Async? You suggesting there isn't possibly a better way to do it because of zero cost trade offs is just gaslighting.
• jmull 5y ago
  > I see this article as not understanding the goals and tradeoffs of Rust. The author would be happier writing in a higher-level language than Rust. That's a pointless conclusion. The author's criticisms of Rust's tradeoffs are invalid because those are the tradeoffs Rust made. A perfect circle!
  • NoraCodes 5y ago
    That would be circular, but that's not what the parent is saying. Rather, eta's post has one central point - she even bolds it for us: > Rust is not a language where first-class functions are ergonomic. And this... I mean, I don't agree with her, but that might be because I've been immersed in Rust for half a decade. But Rust's tradeoffs are based around four things. It is a: - performant - reliable (incl. memory safe) - productive - systems programming language "Systems programming language" means no garbage collector. "Reliable" means you can't just pass around references to memory willy-nilly. I'm sure closures and their associated traits (Fn(), "call it as many times as you want", FnMut(), "call it only when certain safety conditions are satisfied", and FnOnce(), "call it once") could have been fine-tuned more, but they _do_ achieve the goal of general, usable (imo) first class functions. I don't see another design that would have done this.
    • ectopod 5y ago
      These things can be simultaneously true. The closure design is as good as it could be given the constraints, and it is not ergonomic.
    • pjmlp 5y ago
      > "Systems programming language" means no garbage collector An opinion not shared by Xerox PARC folks, ETHZ or Microsoft Research.
• Rusky 5y ago
  > But such a type doesn't really mention the lifetimes of whatever it captures, so in practice it doesn't work well in Rust. It can: Box<dyn Fn(Arg) -> R + 'a> This is just the general syntax for any trait object with a lifetime.
• karmakaze 5y ago
  I don't use Rust but was curious if this was a real or imagined problem--it seems to be the latter. Two points: 1. the example could have made a pure callback function that takes a mut& db param and pass thaf param to the 'do'er function. Why is this not required anyway? i.e. How does Rust know/decide who owns the mut& db if both the closure and the function that creates the closure can reference it? 2. as mentioned, the complaints about async seems to apply to closures in general even if used synchronously, AFAICT. Best to think of this post as quirks when working with closures in Rust. It would have been far better to list each with workarounds.
  • brabel 5y ago
    > the example could have made a pure callback function that takes a mut& db param and pass thaf param to the 'do'er function. No. If you did that, you could only pass functions in that took the db as a closure argument, nothing else... you're thinking about this example only, and forgetting the problem is with the general case of capturing whatever is needed for the callback to work.
    • karmakaze 5y ago
      Ok, so then the problem statement is how to do dependency injection of lifecycle references into closures. This is specific and asking quite a lot of a single entrypoint function. The closure 'at some point' (either at the end, or coordinated give then get calls) would have to transfer ownership. At this point you'd likely be better off making an interface and not a single entry/exit closure.
• pjmlp 5y ago
  C++ lambdas are much better than Rust ones, because you can explicitly decide what to copy inside of the closure. As for the author being happier writing in a higher level language, it kind of proves the point that Rust's main target is the domain where any sort of automatic memory management aren't a viable option. Pushing Rust outside of this domain is only trying to fit a square peg into a round hole.
  • pitaj 5y ago
    C++ lambdas need that feature because they don't have lifetimes or a borrow checker. Rust closures don't need that feature because rust has a borrow checker that ensures you aren't referencing something you shouldn't be. You can explicitly decide what to copy inside a Rust closure as well, you just use a `move` closure and create references for anything you need referenced outside the closure instead.
    • pjmlp 5y ago
      Move closures assume everything is movable, in C++ you can control what actually takes place, and yes there is the possibility to get it wrong.
      • remram 5y ago
        Parent is saying that you can declare your closure 'move' and move references rather than values, therefore getting the mixed behavior you described. You don't have to move all values.
        • pjmlp 5y ago
          If you want to copy in such cases in Rust, you need to copy to dummy variables that are then moved into the closure, a common pattern in Gtk-rs that even deserves a macro to simplify the boilerplate.
• ridiculous_fish 5y ago
  Some of the pain inflicted by async Rust is incidental, not due to this GC choice. Pin is the biggest culprit. Pin exists so that references captured in Futures may be implemented via raw pointers. This implies that a Future contains pointers to itself, hence Pin. The cost of Pin is that it forces you to write unsafe code as a matter of course, the so-called pin-projections. [1] Look at the requirements for structural pinning: they are quite complicated. References captured in Futures probably could have been implemented differently: base + offset, or relocations, or limit what can be captured, or always Box the state for a Future that captures a reference. Those would have avoided a GC and been even safer, since it wouldn't require unsafe code on the part of struct implementors. 1: https://doc.rust-lang.org/std/pin/#projections-and-structural-pinning https://doc.rust-lang.org/std/pin/#projections-and-structura...
  • pcwalton 5y ago
    I don't think base + offset would have worked because then references would have to be codegen'd differently depending on whether they're in a future or not; this would have impacted separate compilation as you can pass references to other functions (including across FFI). Relocations only work at load time (either dynamic linking or link time), so they wouldn't work. Limiting what can be captured was tried with the old futures crate and it was a real nuisance. Boxed futures violate the zero-cost abstractions principle, which is the reason why many Rust users are using the language in the first place. Pin is an evil, but it's the least evil of the possible options.
    • ridiculous_fish 5y ago
      "Relocations" was meant as an analogy. The idea is, each time a future is entered, it checks if its base pointer has changed. If so, it fixes up its own pointers, similar to how a linker fixes up self-pointers in a dynamic library. There's obvious limitations but it would handle common cases efficiently. Anyways when I first tried async, I encountered structural pinning immediately, and I was surprised to learn that users are just expected to write unsafe code to participate in this system. Maybe it really is the best tradeoff but seems at odds with the rest of the language.
• Matthias247 5y ago
  I agree that this one isn't about async Rust at all. The main comparison is drawn in > And then fast forward a few years and you have an entire language ecosystem built on top of the idea of making these Future objects that actually have a load of closures inside And this sentence is wrong. There's next to no `Future` implementations which are built on top of callbacks. And the reason why that's the way it is is exactly the one the author mentions: Callbacks don't work very well with Rusts ownership model.