6 ms·
Arc and Mutex in Rust
- gnulinux 4y agoI want to note here that throughout the internet and HN there is this ongoing wisdom that although Rust is very useful, valuable etc writing code in it is very hard because it constantly feels like you're wrestling the compiler. Well, maybe there is some truth to it, but I want to say that having started learning Rust in 2022, I found it's multi-threadedness model very refreshing. I thought it was such a delight to write code in, and personally loved it at least one order of magnitude than Go's goroutines. It's safe, embedded in type system (Send etc), has tons of standard tools (Arc, Mutex, mspci channel etc...) that are hard to shoot your foot with. When people talk about Rust's complicated nature, I wish more emphasis is made on how it completely eliminates entire categories of bugs even within a multi-threaded context. As a corrolary, it really makes some parts of programming hard in other languages, nearly trivial in Rust.
- brink 4y ago> writing code in it is very hard because it constantly feels like you're wrestling the compiler I've been doing Rust for about two years. This difficulty eventually subsides and writing "safe" code becomes second-nature. I rarely wrestle with the compiler any more.
- geodel 4y agoAgree. For other languages it would be Stockholm syndrome but in Rust's case it is enlightenment.
- stormbrew 4y agoFor me I'd describe it as having transitioned from "fighting with the compiler" to "having a conversation with the compiler." If I'm doing anything at all non trivial there's a back and forth there still but I appreciate and understand the feedback more now.
- rascul 4y agoIt took me about a month of free time coding to quit fighting the compiler. I'm not sure who won, but we work together now. It's a real good feeling when you can finally write 100 lines or so that rustc takes on the first try. Even if it was all comments...
- puffoflogic 4y agoA likely large contributor to others' experience not matching yours is async: Newcomers often refuse to try rust's excellent native threading support and jump right to the badly tacked-on, lets-write-javascript-in-rust monstrosity called async. Async "won" in the rust ecosystem because of superior branding but it is by far the worst solution to parallelism in rust. It has, by necessity, 100% of the problems and limitations of threading, such as they are, plus many more of its own. It's impossible that async could ever be easier than threading and yet countless newcomers are lured in and convinced they don't understand threads therefore need async. "I don't understand how to fly a Cessna therefore I need to fly an F-15."
- remram 4y agoAsync has been in development for a few years during which the experience wasn't great. The popularity of Rust made it so many people wanted to try it anyway, with a plethora of frameworks being developed and replaced and not working when the next version of futures/tokio/hyper came out. The experience now that futures are in the standard library and tokio etc are actually stable is much better. Most async libraries are thread-safe, contrary to the JavaScript or Python asyncio worlds, so you are free to call call block_on(func(...)) in whatever thread if that is your preferred coding style and the library doesn't already expose a blocking interface. The low-level "zero-cost" future composition is available to you if you need to squeeze a lot of performance but you absolutely don't have to wrestle with the "monstrosity" if you don't want to.
- geodel 4y agoWell, users are obsessed with Async because Rust authors themselves are obsessed with it. The amount of discussions around design and development of Async feature is endless. The common theme is how a normal crappy Go app will become super app with Rust's zero cost async support. I can understand they chose this path to gain market share by having tons of web services written in Rust. And async might be great fit for it. But now one can't just go around and say 'Oh, Rust async is not for plebs they can continue with something much simpler to fulfill their needs'.
- cmrdporcupine 4y agoYep and the worst is that it seems that now 3rd party dependencies basically force it because they've also made that choice. And one thing in your stack is async, everything else ends up following.
- Chio 4y agoTo add my own few cents to this very common discussion Rust is hard is the same way that most programming languages that rely "non-standard" paradigms are hard because they challenge your current mental model of programming. However "different", instead of "hard", would be a better way to describe these sort of languages like Rust, Prolog, and Haskell (as well as a ton of other languages). If you look at Rust from this perspective then it becomes much more valuable to consider learning because it teach you a different way to modelling your data, and how to break-up your code; and while a language like Java may not force you to consider the borrow checker you will probably be a better overall Engineer knowing how to make it happy even when working in other languages. On the topic discussed in the actual article: I can't wait for scoped threads to finally go stable since being forced for throw an Arc around almost everything you want to share between threads does not really suit Rust which really emphasise zero-copy in their standard library.
- josephg 4y agoI think a lot of the complaints about rust's difficulty come from people trying to use async in rust. I don't think the problem there is that rust is "different". The problem is that async rust isn't ready yet. Rust's ecosystem pushes users toward using async for IO, and async support in the language & compiler is wildly incomplete. And it has been for years - the last big async feature (the async keyword) landed 2.5 years ago [1]. But its barely usable as-is. Building software with async today feels like trying to build a house out of daggers. Traits? Nope. Closures? Nope. Regular types? Only if they're Pinned. Your types are pinned, right? Iterators over async objects? Nope that doesn't work. Library support? Without traits, libraries are limited in what they can provide. Async rust? Batteries are not included. We need TAIT, GAT, async closures, async iterators / streams, and a bunch of other little quality of life improvements before I'd consider it "done". I don't know where async's momentum went, but its a bit disappointing how halfbaked async is given how nice the rest of the language is. [1] https://areweasyncyet.rs/ https://areweasyncyet.rs/
- Chio 4y agoI agree wholeheartedly on your points regarding async, which is a topic I've tried to avoid so far in all of my Rust projects because of all the reasons you've already raised above and you really don't need it for most use-cases.
- josephg 4y agoI agree. I've been writing a lot of javascript for the past few years. I've come to love async in javascript - async makes my javascript code fast, readable and reliable. Moving to rust, a lot of the struggle I've experienced has come from trying to write rust the way I would javascript - by using rust's async primitives. Async support in rust is complex, immature and incomplete. And its been this way for years. For a project a few years ago, I tried making an HTTP server which supported a custom SSE-style endpoint. The resulting code was atrocious - it was long, complicated, and I needed to invoke deep lifetime magic that I still don't fully understand to make it compile. I just want to implement both sides of an async stream - but no; thats apparently too hard. I got hundreds of lines in, and the code still wasn't finished. I rewrote it in about 20 lines of javascript and it works great. I've decided to stay away from async rust until the language & ecosystem mature. Synchronous rust is a delight. I'm going to need to do more IO soon, and I'm dreading it. I don't really want to use threads, but I think it might be the right choice given the current state of async rust.
- likeabbas 4y agoDoes anyone know what the final state of Rust’s async will look like? It seems like everyone is dreading it now but will it ever get to a point that people like writing it?
- lumost 4y agoasync is final unfortunately. My experience with rust so far is that there is a small set of program structures which Rusts type system works well for, and a large set of programs and structures where it becomes an unholy mess to make work e.g. deep lifetime magic, complex async/traits scope, etc. If you stick on the golden path then the compiler has your back, deviate, and then you are stuck in a deep hell of compiler suggestions that don't pan out. One observation I've had is that complex lifetime problems are usually solved by copy's or the use of unsafe in most production code. I would love to see a future equivalent of "clean code" for Rust, as the current situation is reminiscent of early java programming.
- 4y ago
- pkulak 4y agoConcurrency is hard. Before I started using Rust I was totally on the immutability train, and I still think it's a great strategy. But immutability everywhere also involves jumping through hoops and fighting with the compiler. Though for some reason no one talks about that strategy the same way they talk about Rust's. I think that, no matter what strategy you settle on, it's a bit more work to guarantee correct concurrency.
- klabb3 4y ago> Concurrency is hard. Absolutely true. This is the submerged part of the iceberg for any programming language. Whenever I see a new language, I skip the hello worlds and the syntax sugar and go straight to see how they solved concurrency and parallelism. Chances are you're gonna need concurrency. If you do, it'll likely be the largest source of complexity in your program at large. Sound concurrency is paramount, and a huge challenge for PL designers. Rust has solved some major problems and deserves recognition for that. It is already influencing new languages, and we will be better off in the long term. That said, in order to achieve their extremely ambitious goals, they needed to push the type system to its absolute limits, and in a painfully user-facing way. When you are writing async Rust today, you're spending a large part of your limited attention/complexity budget trying to satisfy the compiler, even when your mental model of the business logic is sound. This is what people mean when they say async Rust is hard.
- igorkraw 4y agoOn the flip side, having used C++, python+javascript (ahhhh) and Rust concurrency, I found the assurance and debuggabilty the type system gives you very relaxing. I can simply break down the problem into "model business logic" and "make the compiler happy" and if I have solved both problems without escape hatches it's not easy to mess up concurrency in Rust (in my opinion). And the ecosystem might be complex, but it's not unnecessarily complicated: basically everything is there for a reason and allow you to hack as much as possible while staying in the promises that rust gives you.
- klabb3 4y ago
- mamcx 4y agoTotally! I return to Swift/Kotlin for new UIs and wrestle with their errors. Man, "wrestling the compiler" in Rust is SOOOO much better! ... and this is even considering very simple things (like see how they choke in simple syntax error with so small help provided)
- Thaxll 4y agoIt's great until you touch async to which Go is way better and easier to use.
- hintymad 4y agoHow are Rust's concurrency tools compared with those of Java? In Java we rarely need to use mutex at all. Instead, programmers use high-level synchronizers (latches, barriers, phasers, exchanges, and etc) and containers (concurrent collections, executors, various pools and flows, and etc). I don't have extensive experience in Rust, and somehow I saw Mutex everywhere.
- inamberclad 4y agoGenerally, I've found that wrestling the compiler involves doing something that should really be done in another way.
- 3np 4y agoWhat resources did you use to get up to speed, both with the language and which libraries to use?
- vgel 4y agoOne option the author didn't mention is that threads can join with a value, so instead of needing scope it could have looked like (on my phone so syntax is probably not quite right): let user = todo!(); let handle: JoinHandle<User> = spawn(move || { // do stuff with user // hand back user user }); // wait for thread to stop and retake ownership of user let user = handle.join(); I used this recently to remove locking in a worker thread and it was a big performance boost, and made the code nicer -- win/win.
- pornel 4y agoVery good article, and these types are essential to have in your Rust "vocabulary". I've seen programmers fight the borrow checker a fight they can't win, because `Arc<Mutex<Object>>` has a non-zero overhead and looks uglier than a plain `Object` or `&Object`. However, if the program has object lifetimes that aren't statically known and/or shared mutable state, these are simply the correct types to use (e.g. event-based systems that can attach listeners have to use these). Rust is just being very explicit about ownership and putting all overheads in your face. In something like Python there's no extra syntax for this, because every object is automatically under a lock and has a refcount.
- Communitivity 4y agoEvery language is hard when you are working with hard problems (e.g. concurrency). I've also found Rust to be hard until I got a decent understanding of the borrow checker and how it works. After that, the only times I found Rust to be hard were when I was trying to make it conform to my way of thinking, rather than changing how I approached the problem to match the way Rust was designed. I've had a similar experience in other complex and well-designed languages such as Clojure, Erlang, and Haskell. In my experience the benefits of Rust (safety-guarantees, speed, good ease of use vs complexity tradeoffs) much outweigh the quirks and learning curve.
- pipeline_peak 4y agoReturn type definitions are optional as Rust programmer's might get confused by difficult words like "void".
- conradludgate 4y agoIt's just implicit for `-> ()`. Nothing magical
- singhrac 4y agoI really liked this article, but if I can provide some sort of drive-by advice (also for others looking to write technical content), providing some examples of where to use various traits would really help, e.g. what kind of data structure one would implement !Send on.
- drogus 4y agoThat's a good point and I think I will add a paragraph or too about that. The simplest answer is: as long as you're not doing any unsafe Rust you should just use whatever is derived from types within a type. If you have a struct where all fields are `Send`, then the struct will be `Send` too. If you add anything that is `!Send` (like `Cell`), it will be `!Send` and you don't have to explicitly mark is as `!Send`. The most common exception to this is when you implement a type that has a `!Send` attribute, but you still mark it as `Send`, because you implement an API that guards against misusing the type. I honestly can't think of any practical example of the opposite (ie. you have a type with all `Send` attributes), but that would probably be similar: you have a type with all `Send` attributes, but you implement some kind of API that makes it unsafe to sent the type between threads.
- lenkite 4y agoA Rust newbie here. Can someone explain why rustc complains that the clojure may outlive the function ? I mean we are joining on both thread handles so the main function has to block until they finish right ? Shouldn't RAII get rid of the clojure objects ?
- marcosdumay 4y agoBecause the compiler doesn't know to look for this way to wait for the end of the thread. The language is Turing complete, but the compiler can only look for a finite set of subprograms that wait for the thread, so it can never catch them all. The compiler could have a special case for looking at joins. But that's extra complexity that gets you almost nothing in return. Instead, Rust developers decided that it's better to do it in a library and created the scoped thread the article talks about.
- lenkite 4y agoThanks for explaining, though since the code is formally correct - taking into account thread joins can't really be considered a special case. I suspect thread spawning and joining would need be a language feature instead of a library feature to enable this.
- marcosdumay 4y agoYes, the code is correct. That comes from the undecidability of any interesting property in Turing complete code. It means that if you are doing static analysis on your code, you have to decide if you are going with false errors or mark bad code a correct, because no analysis can always give you the correct result. On this case, you get a false error.
- steveklabnik 4y agoIt doesn't need to be a language feature. Scoped threads are a library implementation of the idea of joining right away. The issue is that spawn is specifically the most general API; one where you may never even join the thread at all, hence its type signature requiring a 'static bound. Rust doesn't suddenly go "oh you're not actually using the full power this signature gives you, so I'll allow it," because that would make the code extremely fragile, be tough to implement, all sorts of things. Really, this is about using the API that communicates the semantic you want, so that the compiler can check it.
- hintymad 4y agoI think a programmer can get over the "Rust is hard" part after writing certain amount of code. The type system and the borrow checker will just become intuitive enough for a programmer to grasp. I do have a question about productivity, though: what added productivity does Rust bring over languages like Go or Java for writing RPC/HTTP services, given that Rust requires an engineer to think about memory management all the time, while GC languages don't?
- ostenning 4y agoTheres a lot of criticism here about “async rust”. I find this wording strange, I use rust asynchronously with rtic (realtime interrupt concurrency) for low level work. Its really excellent. Are peoples criticisms specifically about inline async?