13 ms·
> I don’t generally find writing Rust to be significantly harder than using a GC’d language. I’ve been trying to figure out why that is. That's quite surprisin
by ridiculous_fish 8y ago
> I don’t generally find writing Rust to be significantly harder than using a GC’d language. I’ve been trying to figure out why that is.
That's quite surprising! Here's some examples of where things would be easier to write if Rust had a GC:
1. Any sort of lock free algorithm. This is a big one - hazard pointers and the like are much harder than letting the GC clean up.
2. Data structures which may contain "back" pointers (e.g. parent pointers in a tree).
3. Data structures which may be cyclic. For example the classic LRU cache is best implemented with a cyclic linked list which is hard to express in Rust.
4. Any sort of refactoring that may adjust ownership. E.g. going from T to Rc<T>. GC requires fewer choices so these refactorings are easier.
Surely this pain is real, even if Rustaceans think it's worth tolerating?
In fairness precious resource cleanup (file descriptors, etc) is easier without a GC.
- gpm 8y ago1. Rust is such a godsend for concurrent programming that any pain associated with lack of GC is worth it. We also have GC like tricks to make it not that bad: https://docs.rs/crossbeam-epoch/0.6.0/crossbeam_epoch/ https://docs.rs/crossbeam-epoch/0.6.0/crossbeam_epoch/ 2. I agree on this one. The basic solution is "well instead of a pointer use a index into a vector" which has some correctness advantages (but less than "normal" rust does), but generally a bit of extra programmer pain. 3. See 2. 4. The compiler takes good enough care of you during things like this that it's basically painless.
- idubrov 8y ago#4 highly depends on a project size. On big projects, some of these decisions might become effectively one way doors. I've been struggling a bit with these issues on the project of ~70k lines. I cannot even imagine what the refactoring would look like if we had, let's say, 1 million LOC. To be fair, though, we use Rust the way it wasn't specifically designed for (large "enterprise" software, think Java-like enterprise). I think, potentially, Rust could offer a much better story for this kind of software (assuming we are not mad and the issues we are facing are not because we doing something completely wrong :) ). In my opinion, the key thing would be to allow building "bridges" between pieces of the system which are "ownership-incompatible", so your decisions around ownership are not "one way doors" anymore (at the cost of translation / adapter layer). Some random things which I think would be helpful: 1. Better self-referential structs, to allow going from "owned A + borrowed B" into "fully owned A+B" (rental crate helps here, though). Basically, hiding lifetimes in scenarios where you cannot easily change the original data structure to "own". 2. GATs. Honestly, this one is my speculation, but it seems like certain patterns which are hard to express now (abstraction of a "mutable reference", for example) would be possible with GATs. In our case, this would allow to bridge the gap between "trait object" world and "parametric over trait" world. The issue I was having is it is hard to express "mutably borrow from self" with traits (this is something similar to the issue "streaming iterator" crates solves). I was able to hack something using arbitrary self types, but it's quite... hacky. 3. Trait objects stable(r) ABI. Again, purely my speculation, but would allow to go back from "trait reference" world into "trait object" world. I won't go into details here, but trait objects want to "borrow" from something and it is not always easily possible (think that favorite vector+indices data structure) -- being able to "fake" those borrows would be nice (maybe). Issues #2 and #3 specifically happens around deciding on data structure: regular structs have one set of tradeoffs, vectors with indexes -- another. In a big enterprise software, I would like to have an option to use whatever works in a particular spot and still have it API-compatible to the rest of the system.
- scottlamb 8y ago> #4 highly depends on a project size. On big projects, some of these decisions might become effectively one way doors. The transformations between T, Box<T>, Rc<T>, Arc<T>, etc. are mechanical, so I expect someone will write a refactoring tool that makes a giant PR for you automatically. (Subject to certain limits, like if you're actually cloning the ref-counted pointer, it's indeed harder to go back.) Would that satisfy your need? > To be fair, though, we use Rust the way it wasn't specifically designed for (large "enterprise" software, think Java-like enterprise). IMHO, this is a valid use case for Rust. I'm not saying everyone should stop using Java (in some cases I think it's significantly faster to write) but Rust has some strong performance advantages and no data races in safe code.
- idubrov 8y ago>Would that satisfy your need? It's not always possible to change the data structure -- different ways of modeling data have different trade-offs. So, for me it is about not having to make a choice than about tools that will help you to change your mind. Also, it could be something like structure coming from a 3rd-party crate using borrowing and you want to stick it into "Arc" of some sort. Or put it (with the thing it borrows from) into a lifetime-less struct, so you don't have to care about these lifetimes. >IMHO, this is a valid use case for Rust. I very much hope so :)
- scottlamb 8y ago> It's not always possible to change the data structure -- different ways of modeling data have different trade-offs. I agree with "different ways of modeling data have different trade-offs", but I don't understand how that leads to "it's not always possible to change the data structure". I revisit trade-offs all the time. Could you explain? I might need a concrete example. > Also, it could be something like structure coming from a 3rd-party crate using borrowing and you want to stick it into "Arc" of some sort. Or put it (with the thing it borrows from) into a lifetime-less struct, so you don't have to care about these lifetimes. Yeah, certainly the refactoring becomes harder (maybe implausible to do automatically) when you can't change both sides in one PR, and when you have to convince someone else to change their interface / bump the major version. It still can be done (partially?) by hand at least; it's just a matter of cost/benefit.
- mmirate 8y ago1. As your other reply said, crossbeam is Rust's gold-standard for this. 2. Make the front-pointers be Rc's or Arc's, then use the downgrade() method to construct the back-pointers. 3. Array-based ring buffer instead, perhaps? 4. Like any refactoring, the compiler errors guide you to the places that you haven't yet changed.
- pcwalton 8y agoCrossbeam makes lock-free data structures easier to write than in any other mainstream language I know of. Regarding graphs, just use indices and vectors. It's often better to use indices for graph nodes anyway, for example in games, where an ECS design using IDs is generally preferable to direct references to objects. For a while, I was kind of obsessed with showing that Rust could do doubly linked trees and graphs just as well as C++ could. I now realize that this was a mistake, and I made a big mess of a lot of code in the process. Having a single owner and using IDs for "secondary" references is often preferable even in languages that easily allow multiple strong references to objects. In the small, direct references and an OO style can be convenient, but in the large, you often want to break up your graph code into components and systems anyway, and it's kind of nice to have the language push you to front loading that kind of design.
- ecnahc515 8y agoCrossbeam is awesome. That said; I think his point is more on the "how hard is it to write a crossbeam-like library in Rust compared to a language with a GC".
- ridiculous_fish 8y agoFrom crossbeam docs [1]: "Programming languages that come with garbage collectors solve this problem trivially." [1]: https://docs.rs/crossbeam-epoch/0.6.0/crossbeam_epoch/ https://docs.rs/crossbeam-epoch/0.6.0/crossbeam_epoch/
- pcwalton 8y agoIt's not just the core mechanism of crossbeam-epoch that I'm talking about: it's the entire ecosystem that surrounds it. Crossbeam-epoch adds some syntactic overhead compared to GC, but it comes with a whole bunch of tools that most GC'd languages don't have, including a compiler that ensures that you take locks when you're supposed to.
- chc 8y agoI know this works, but whenever I hear this advice, I can't help thinking: Aren't indexes and vectors basically just a way of smuggling pointers past the borrow checker?
- steveklabnik 8y agoThe other posters have sorta said this, but the key is Cargo, and strong generics. I don’t write these things, I use a package where someone else has written them. The difficulty of writing this stuff does not impact my day-to-day getting stuff done. I don’t write Hashes in Ruby, I don’t write HashMaps in Rust.
- leshow 8y agoI'd just like to add, most engineers aren't writing data structures all day. There are certainly some sharp corners, some of these Rust will help you with though: re: multithreaded algorithms. Robust libraries are helpful also.
- hardwaresofton 8y agoI'm a relatively new Rustacean but surely that statement is hyperbole -- one of Rust's only drawbacks is it's learning curve. That statement more reflects how many feel (myself included) after working with rust for a little while, and internalizing the beneficial nature of the extra pain (being more strict about how you pass around memory, etc), and getting used to being more explicit. Also, other languages have difficulties that eclipse and are more endemic than rust's so there's probably consideration of that too.
- steveklabnik 8y agoIt’s not really hyperbole; I don’t find Rust particularly painful. Heck, I used to say “rust will never be a good choice for web apps” but by now I’ve written several. Times change!
- wokwokwok 8y agoYou can down vote this comment all you like, but it's just Steve's opinion, and you can't be wrong about your own opinion. :) However, this is, anecdotally, what people say after they start using rust heavily. I've heard it from quite a lot of people. It's like turning on all the jslint/tslint's as hard errors. Yes, it's painful to start with... but, as you relax, and let the compiler to the hard work of checking things, you can confidently write code without worrying about a certain entire set of domain concerns, because the tooling is taking care of it. Sure, I'd love it if there was a 'alt-enter, fix this' for rust errors in [my editor of choice for rust], and no one is saying, 'oh hey, rust is super easy to learn!'. ...but more and more (as the ecosystem matures) rust is being used to Get Stuff Done, not just to write little toys; and the people using it to Get Stuff Done aren't complaining about how hard it is, they're <3'ing it. (Of course, I disagree on the web-apps front; I feel like that's still very painful with the 15+ frameworks out there fighting / breaking changes / being abandoned; but at the very least its starting to look like a few plausible stable options are emerging; and I admit this is survivor-bias, where the people who didn't like it gave up before they started to get productive... but, I think the idea that 'rust is hard always' isn't true at all)
- blt 8y agoNon sequitur, but in my experience the parent pointer in a tree can almost always be passed as an argument in whatever recursive function is traversing the tree.
- dbaupp 8y agoThis doesn't work for a mutating traversal in general (although it can work if the tree structure is split from the data being stored, so only a subset of the parent needs to be passed into the recursive calls).
- CoolGuySteve 8y agoIn a lockfree data structure, how do you ensure GC stalls never happen in the real-time code? I write a lot of lockfree C++ that requires 100 nanosecond-magnitude latencies 100% of the time. But in doing so, my algorithms are always implemented with a cache coherent wait-free allocator pool as well. I don’t see how you can achieve the same thing in a GC language without a lot of manual tuning or a similar memory pool design.
- ridiculous_fish 8y agoIf you need sub-microsecond latencies then the requirements can be summarized as "don't allocate, don't deallocate." Now the memory management technique is irrelevant, because the goal is to avoid memory management at all costs. Yes you'll need a memory pool design in whatever language. But most GC'd languages only collect at allocation sites, and so if you don't allocate you won't trigger a GC.
- pjmlp 8y agoMultiple options are possible depending on the language. 1 - Disabling the GC altogether during access to the data structure 2 - Ensuring that the GC wont run during the region (TryNoGCRegion() in .NET or a real-time thread in Real-Time Java) 3 - Or if the language supports it, keep part of the structure away from the GC, using RAII to keep track of the blocks
- frankmcsherry 8y ago> 4. Any sort of refactoring that may adjust ownership. E.g. going from T to Rc<T>. GC requires fewer choices so these refactorings are easier. This pain is real. I've been using Rust for four years now, and it hasn't gotten any easier. And unfortunately, the best strategy I have for refactoring is still "start refactoring; then see what issues come up and if I have to revert".
- pjmlp 8y ago> In fairness precious resource cleanup (file descriptors, etc) is easier without a GC. Not really, as proven by languages like Mesa/Cedar, Modula-3 or more recently D, Swift, ParaSail or Chapel. Somehow I think many CS degrees fail at teaching a proper history of programming languages. Just because there is a GC involved, doesn't mean that it the only language feature available for resource management. As for the other points, that is why I would advocate the use of other ML like compiled languages like Swift, OCaml, Haskell if having making use of a GC is not an impediment for the application being deployed. If memory management is absolute a no-go, e.g. MISRA, High Integrity, DOJ certifications, real time kernels, high performance graphics, then Rust is a very good tool.
- dbaupp 8y agoPrecious resource management is easier without pervasive sharing and with (modern) C++ or Rust style ownership. Most GC (or RC'd like Swift) languages end up attaching resources to their heap object type, with the free sharing that implies. Even in a reference counted implementation, code can squirrel away a strong pointer to keep a resource alive beyond it's obvious syntactic region, or keep trying to access after it has been invalidated. Those languages do offer some tools and conventions that help, but they're typically opt in.
- asplake 8y ago> or more recently D, Swift, ParaSail or Chapel. Gotta love Python's 'with' statement too, not that you'd necessarily choose Python where resources are precious, but it depends on what the resources are...
- jonathanstrange 8y agoI have read The Rust Programming Language from start to beginning and afterwards decided to use Go instead because Rust does not have a Garbage Collector. If somebody added a GC to it, including syntactic sugar that doesn't make it feel 'glued on', then I'd be happy to use it. Something else ultimately turned me away from investing more time into it, though: Chapter 17 of the book, which spends a great deal of time trying to advertise why its OOP support is so impoverished. In comparison, Donovan & Kernighan's The Go Programming Language is much more honest about the (many) misfeatures of Go. That being said, already existing libraries and Rust's tooling are excellent, and so I'll surely return to the language. Not because it has no optional GC but despite of it.
- steveklabnik 8y agoWhat did you find dishonest? Chapter 17 was one of the hardest to write.
- jonathanstrange 8y agoFirst of all, I didn't use the word "dishonest", there is nothing dishonest about your book. It is a fine book. :) What I found problematic was the section on "Inheritance as a Type System and as Code Sharing", especially the paragraph "Inheritance has recently fallen out of favor..." While there is some truth to the claim, you forget to mention that there are also metric tons of highly successful OOP based software libraries out there, that it is perfectly feasible to use inheritance (and multiple inheritance) in good and productive ways, and that the Rust community is still quibbling about how to implement GUI frameworks in a "Rust way" and how to interface to foreign OOP libraries in general because of Rust's OOP limitations. It is a drawback not to have full OOP with inheritance and multiple dynamic dispatch. A non-biased and informed language user may choose Rust despite this limitation, but not because of it. Another thing I found problematic was the "newtype pattern" pp. 439-441. Whether the wrapper is optimized away or not, the newtype pattern is clearly a cumbersome solution for a language limitation. At least to a beginner who has seen many other languages, this just looks like a horrible workaround for the lack of scalar subtypes, as e.g. Ada has them. To be honest, pp. 440-441 were probably the final reason why I decided to postpone writing software in Rust. (Coincidentally, Ada also allows you to define type aliases that are discussed on the following pages, but in Ada they are normally considered bad style and only used for clearly identifiable abbreviations.) That being said, I found your book very informative and overall good reading, and I'm planning to come back to Rust once I have the time.