10 ms·

Changing the Rules of Rust

190 points by kevincox 3y ago

j16sdiz 3y ago
rust is _almost_ as complex as C++. Introducing these new "edition" would grow the complexity exponentially. I hope they would think twice about the effects on new learners before even experimenting
- weinzierl 3y ago
  "rust is _almost_ as complex as C++." I don't agree, but even if it were true, C++ has so much accidental complexity (because of its legacy) that you cannot meaningfully compare it with Rust, which by and large is very well thought out. In general, if something is complex in Rust that is because it's in the nature of the problem and not because of some quirkiness in the solution. Borrowing is the prime example. People find it complex, but just because the C++ compiler doesn't check the rules for you, doesn't mean you have follow them in C++ as well (which is tedious and error-prone) or know exactly why you don't have to follow them (which is even harder). I admit though, that the Leakpocalypse (which the article talks about, even if it doesn't call it like that) is an area where accidental complexity lurks.
  - vlovich123 3y ago
    Pin is another one. I still can’t wrap my head around it. I’m sure it’s simple and the docs are great, but having read them multiple times I’m not really more clear. I probably need to see a bunch of hands on examples. I wish they’d gone the Move route instead of rushing to 1.0 and then retrofitting in Pin. But I agree, in general things in rust are a lot more thought out and experiments are relegated to unstable which is fine with breaking changes. I do wish though that they had a better versioning story in the language for making backward incompatible changes more a matter of course. It’s great already for “user space” code via editions, but as the author notes making certain kinds of changes brings up risk and uncertainty because there’s not a well trodden path to support changing core semantics in the language and having it work gracefully without needing a migration en masse.
    - majewsky 3y ago
      > I wish they’d gone the Move route instead of rushing to 1.0 and then retrofitting in Pin. That is easy to say, and I don't disagree with the sentiment, but there are valid reasons for setting a 1.0 release date and cutting corners to get there. If they had not done that, there probably was a real risk of running an infinite loop of "let's wait for Move to be implemented", "great, now let's wait for Leak to be implemented", "great, now let's wait for..." and a 1.0 never gets put out. Rust may not be perfect, but at least it's stable and people are actually using it.
    - cmrdporcupine 3y ago
      Pin used to confuse me, but now it's clearer. It's right in the name: Pin this thing to where it is in memory, never try to move it. Rust can and does move things around as it feels the need. So you need something like Pin if what you've built can't move. It's confusing if you're coming from a C++ background because in C++ something only "moves" if you explicitly std::move it. In Rust its effectively the reverse. A crude way of thinking about Rust's call pattern is that it's basically the opposite: as if every call was a std::move operation. So Pin is basically just saying: you can't move this, don't try. The memory can't be moved. And it's mostly used in corner cases where some kind of explicit memory management is involved. Most Rust users should not have to worry about it. It's one of those things that becomes more of a concern if you're writing library code for low level datastructures, or interfacing with C/C++ libraries.
      - laurencerowe 3y ago
        > Most Rust users should not have to worry about it. It's one of those things that becomes more of a concern if you're writing library code for low level datastructures, or interfacing with C/C++ libraries. Don’t all Rust users writing async code have to worry about Pin?
        brigadier132 3y ago
        Not really, the async libraries hide the implementation details from you. It's an issue for library developers.
  - cwzwarich 3y ago
    One advantage C++ has over Rust is that there are multiple fairly compliant C++ implementations, and the process of making them created some shared understanding of how the language actually works from an implementation perspective. There are several aspects of the current Rust language that were effectively defined by arbitrary decisions made by the implementors, which aren't really spelled out anywhere besides the code.
    - tialaramex 3y ago
      Do you have examples? I think in many ways Rust is doing a better job of tying down the tricky details than C++ has. There is no effort to explain how pointer provenance works in C++ at all. The state of the art is if you write a provenance bug, someone from a compiler vendor will gesture at DR/260 which says provenance exists but WG14 (so C, not even C++) declines to explain how it works, and that's all you get. In contrast Aria's "Experiment" (https://doc.rust-lang.org/std/ptr/index.html#strict-provenance https://doc.rust-lang.org/std/ptr/index.html#strict-provenan...) has gone rather well. The vast majority of people who want to do acrobatics with pointers can do what Aria says you have to do to get coherent behaviour, and by following her rules they can make code which works in Rust.
      - uecker 3y ago
        WG14 produced a document clarifying provenance in C (which Aria knew): https://open-std.org/JTC1/SC22/WG14/www/docs/n3005.pdf https://open-std.org/JTC1/SC22/WG14/www/docs/n3005.pdf
        tialaramex 3y ago
        Given your username, I assume I'm addressing Martin Uecker and I want to commend you for putting the work in on this problem over a long period. While a draft TS is closer to actually resolving a 20+ year old problem it's not there yet and this draft is, if you pardon me, in my opinion more than three months work from publishable. In contrast, Aria went from "We should see whether strict rules are workable" to the current set of Rust nightly features relatively quickly. If I want to attempt pointer stunts in C, N3005 is probably worth reading first, but it isn't actionable from what I can tell. Contrast Aria's changes, I can go try that (in nightly Rust) and see it works or why I can't / mustn't do what I wanted under Strict Provenance and who I ought to talk to about that. I wish you luck for C 29 (?) but that's a distant future, and we're contrasting C++ here, so as with #embed you might take this obvious must-have for C29 and find that somehow WG21 are still so far behind they can't do the same for C++ 29.
- rapsey 3y ago
  C++ versions are practically a new language. Last two Rust editions required no changes on any of my code. Why are people constantly harping on this obviously untrue idea. New Rust versions and editions make the language _simpler_.
  - kaashif 3y ago
    > C++ versions are practically a new language. Last two Rust editions required no changes on any of my code. This juxtaposition kind of makes it seem like that new C++ versions did require changes to your code, did they? My understanding is that C++ maintains backwards compatibility.
    - rapsey 3y ago
      Rust editions are allowed to break backwards compatibility. If you have 2015 edition code and change your Cargo.toml and set it to 2021 edition, the code may no longer compile. My code went from 2015 to 2018 and 2021 by simply changing Cargo.toml. No code changes were necessary and it was still completely idiomatic code. Nothing was deprecated or even outdated. An idiomatic C++11 solution is not likely to be the idiomatic C++17 solution however. Every version of C++ makes deep changes to the actual language and adds a ton of features. Whereas Rust versions and editions tend to make the language simpler as they generally remove limitations.
      - Conscat 3y ago
        Every release of C++ since 11 has relaxed limitations on `constexpr`, and several relaxed limitations on `template`. Many other features have had various rules relaxed in certain releases, including `for` loops, lambdas, labels, moves, and atomics. It's true that there are usually new features which arguably make older code no longer idiomatic, but modernizing code is always optional, so what's really the issue there?
        rapsey 3y ago
        The issue is the language gets more complex over time. The origin of the discussion was the statement Rust is just as complex as c++, which is completely wrong. Rust changes in small increments and even when they decide on a new edition that can break compatibility, it still barely changes.
- duped 3y ago
  Rust already has editions. There have been three releases since 1.0.
lukebitts 3y ago
Very interesting read! I really wish we could be unburdened by backwards compatibility so big changes like these are possible, but I realize that would open its own can of worms
- Taek 3y ago
  Honestly backwards compatibility in rust is already half-assed anyway. Trying to use a fixed version of the compiler (important in some contexts such as security software) is a huge pain in the ass.
  - Ar-Curunir 3y ago
    Backwards incompatibility in the language is different from backwards incompatibility in libraries.
  - IshKebab 3y ago
    I've never had any notable issues. I've had far fewer compatibility issues with compiling crates than I have compiling C++ libraries or installing Python packages.
  - mirashii 3y ago
    It’s about a 10 line nix file, a 10 line dockerfile, a one line rust-toolchain.toml.
  - ekidd 3y ago
    Rust has an excellent story for running old code on new compilers. For example, I've been using Rust in production since just after 1.0, so about 8 years now. I maintain 10-20 libraries and tools, some with hundreds of dependencies. If I update a project with 200 dependencies that hasn't been touched in 3 years, that's 3x200 = 600 "dependency-years" worth of old code being run on a new compiler. And usually it either works flawlessly, or it can be fixed in under 5 minutes. Very occasionally I'm forced to update my code to use a new library version that requires some code changes on my end. I've also maintained long-lived C, C++, Ruby and Python projects. Those projects tend to have far more problems with new compilers or runtimes than Rust, in my experience. Updating a large C++ program to newest version of Visual C++, for example, can be a mess. However, Rust obviously does not support running new code on old compilers. And because stable Rust is almost always a safe upgrade, many popular libraries don't worry too much about supporting 3-year-old compilers. (This is super irritating for distros like Debian.) Which if you're working on a regulated embedded system that requires you to use a specific old compiler, well, it's a problem. Realistically, in that case you'll want to vendor and freeze all your dependencies and back-port specific security fixes as needed. If you're not allowed to update your compiler, you probably shouldn't be mass-updating your dependencies, either. Basically, Rust got so exceptionally good at running old code on new compilers that the library ecosystem developed the habit of dropping support for old compilers too quickly for some downstream LTS maintainers. And as a library maintainer, I'm absolutely part of the problem—I don't actually care about supporting 5 year old compilers for free. On some level, I probably should, but...
Ygg2 3y ago
I think eventually or at some point there might be a need for Rust 2.0 even if it won't be called Rust. Think something like JavaScript and TypeScript. A language that's a super set except the former would be soft deprecated. I know this is highly unpopular opinion, but I don't consider permanent backwards compatibility a good thing.
- vimpunk 3y ago
  What parts of Rust would you like changed, though? I think the current version, with its minor warts, is a very pleasant and consistent experience for the most part -- very much unlike C++. So I'm not sure what I'd change to such a drastic degree that it'd require such breaking changes.
  - lawn 3y ago
    async still feels half-baked and not pleasant like the rest of the language. Now, it may be something that still can be fixed and improved upon, but a Rust 2.0 with nicer async experience would be a big win.
  - Taek 3y ago
    I would definitely want to see async totally overhauled. Also it would be great if major components of the standard library were all version 1. And if I want to risk going too far, I'd also advocate for refactoring the type system to be slightly less generic in favor of chasing faster compile times. Also, ideally the lead designer of SafeLang2.0 would be much more heavy on engineering experience than PL theory. Rust made a lot of tradeoffs that were beautiful in PL theory but are annoying to use on a day to day basis. I honestly think rust is equal parts amazing ideas and mediocre ideas, and I think a slighky less ambitious language would still be almost as safe (world class compared to any other language) but 10x more fun to use.
    - gmnash 3y ago
      A complete overhaul of async sounds nice in the abstract, but without exact details it’s difficult to see how it would be better than what exists. Similarly, I’m curious as to what the details of a “slightly less generic” type system entails.
amluto 3y ago
I wonder if the new-edition approach could be done by interpreting pre-2024 code as if everything had a Leak bound. So instead of a not-Leak type being disallowed when passed to a pre-2024 module, it would not pass type checking because everything in the pre-2024 module required Leak.
- jeff-davis 3y ago
  The article addresses this: "However - and this is the big, enormous caveat of this section - this would rely on the compiler effectively implementing some kind of unbreakable firewall, to prevent a type that does not implement Leak from ever getting used with pre-2024 code."
  - amluto 3y ago
    What I mean is: instead of having a “firewall” in the compiler, is there a way to interpret pre-2024 code such that it has correct but conservative Leak bounds? Then there would be more confidence that, if a mixed-edition program type-checks, then it’s correct.
    - ComputerGuru 3y ago
      It took me a minute, but the firewall is talking about the reverse direction of preventing post-2024 code without Leak being used in pre-2024 code.
- AndrewDucker 3y ago
  How would it know the code was pre 2024?
nynx 3y ago
I’m actually glad the Move and Leak didn’t get in. They both have fairly niche usecases but they’d have to be spattered all over basically all rust code.
- jeff-davis 3y ago
  Why do you say they'd have to be splattered over basically all rust code? The post compares them to Send and Sync, and I don't have to use those very frequently (though I don't write a lot of rust code, either).
  - nynx 3y ago
    People use Arc and Rc all the time, which would need +?Leak annotations. Things like Option would, I think, also need + ?Move + ?Leak annotations.
    - bryanlarsen 3y ago
      If it is an unobtrusive as the existing ?Trait annotations such as ?Sized it's not very much of a burden.
    - deleted 3y ago
      [deleted]
  - mlindner 3y ago
    Because features are always used in the worst (most evil?) way possible by inexperienced programmers. One of the benefits of Rust is that it makes many evil things either impossible or easily detectable.
jeff-davis 3y ago
The Leak trait could be interesting when it comes to C code that calls rust code that calls C code that might longjmp() over the rust code in the middle. Right now longjmp()ing over rust code is not a great idea for a couple reasons, but because leaks are currently always allowed in rust, it arguably follows the rules at least in some cases (doing so could cause problems, of course). If the Leak trait were introduced, then skipping destructors would be clearly the wrong thing to do if there are types on the stack that don't implement Leak. That would clearly require some way to either prevent longjmp() from ever jumping over rust code (e.g. always catch it in C code and turn it into a special return code), or find some way to catch the longjmp and turn it into an unwind in rust (e.g. a panic?) and then turn it back into a longjmp to go back into C.
- connicpu 3y ago
  Honestly, longjmp is just extremely problematic for any code that isn't pure C. Even just adding C++ into the mix can easily make longjmp something that can't be used without invoking UB. "If replacing std::longjmp with throw and setjmp with catch would invoke a non-trivial destructor for any automatic object, the behavior of such std::longjmp is undefined."[1] [1]: https://en.cppreference.com/w/cpp/utility/program/longjmp https://en.cppreference.com/w/cpp/utility/program/longjmp
  - jeff-davis 3y ago
    setjmp/longjmp exist in a lot of real world C code, so there is a lot of value in allowing rust to deal with longjmp in a defined way at least in some narrow cases. And I don't think it's impossible -- it probably requires some special wrappers (perhaps bindgen could generate them?) that would call setjmp before entering the C code. setjmp has its own set of problems, but I don't believe those are impossible to solve, either.
- LegionMammal978 3y ago
  > Right now longjmp()ing over rust code is not a great idea for a couple reasons, but because leaks are currently always allowed in rust, it arguably follows the rules at least in some cases (doing so could cause problems, of course). In the general case, skipping destructors of objects you don't own (through longjmp(), killing a single thread, etc.) has never been allowed in Rust: at any particular point in the program, you're only allowed to skip destructors of objects that you currently own, or can otherwise gain ownership of. For instance, the thread::scope() API, after running the spawning thread's closure, uses a destructor* to join all the created threads. However, if a program could longjmp() past that destructor, then the spawning thread could access its variables again while the created threads are still using them, resulting in a data race. (This is the same issue that the original thread::scoped() API had.) let mut x = 0; let mut jmp_buf = JmpBuf::default(); if setjmp(&mut jmp_buf) == 0 { std::thread::scope(|s| { s.spawn(|| loop { x += 1; }); longjmp(&jmp_buf, 1); }); } else { loop { println!("{x}"); } /* the created thread is still changing the value of x! */ } That is to say, longjmp() has always been a very unsafe operation, which can only be sound if you're in control of every single destructor which it skips. A Leak trait wouldn't change anything in that regard. * Or rather, a catch_unwind() followed by a resume_unwind(), which is mostly equivalent to a destructor in this context.
notnullorvoid 3y ago
Here's an idea, for a kind of soft rollout. Implement a optional feature for 2021-edition crates that allow specifying Leak. The trait has no effect in 2021. In next-edition: - 2021 crates that turned Leak feature uses it just as a next-edition crate would. Only things explicitly marked with Leak in the crate can leak. - 2021 crate that didn't opt-in to Leak feature, automatically marks everything in the crate with Leak.
- bryanlarsen 3y ago
  I think the big caveat mentioned in the editions section (https://without.boats/blog/changing-the-rules-of-rust/#editions https://without.boats/blog/changing-the-rules-of-rust/#editi...) still applies to your suggestion. The author concludes in that section "I don’t know if this is even possible to implement, probably it would have pretty bad effect on compile times at". Your refinement seems like a nice addition to the author's suggestion, if it is possible and doesn't effect compile times too much. Hopefully somebody smarter than me takes a run at the problem.
- kmeisthax 3y ago
  from __future__ import Leak
hedora 3y ago
The lack of a Leak-style trait is so painful for me that I’d strongly consider switching to a different language that was basically just rust + Leak. (Adding move and removing pin would be nice to have too.) The problem with not having Leak is that it forces all futures to be ‘static, which effectively disables the borrow checker whenever you cross an async spawn boundary. Arc is the standard solution, but, for multi-threaded code, it can be expensive (though also more predictable, in fairness) than having a GC. The only other solution (which I prefer) is to use an unsafe version of spawn that assumes the future it returns will not be leaked. This certainly breaks the rust promise of supporting safe, zero cost abstractions.
- sabageti 3y ago
  I have seen many comments like this, and I would like to understand better I feel like I'm missing something. In my daily use case(web api) of async rust we use rarely spawn. What are the use case of intensive spawn usage ? And there is tokio::spawn_local() that doesn't require Send
- scottlamb 3y ago
  > Arc is the standard solution, but, for multi-threaded code, it can be expensive (though also more predictable, in fairness) than having a GC. Do you have measurements to back that up? I would expect it to be a lot cheaper than GC because GCed language such as Java/Go/etc. GC every heap allocation (and Java in particular doesn't have unboxed objects, so there are a lot of heap allocations!) where Arc<RequestState> would bundle a bunch of things together. That said, I'm not in love with the Arc solution either. It's not just the performance impact. The programming model of scoped concurrency is just much more pleasant. Giving up that and then also having to deal with putting everything into an Arc, and likely also then wrapping it in a MutexLock, even when it should really be one thread at a time dealing with the value in question, just doesn't do anything good for readability and writability of code...
  - charcircuit 3y ago
    >because GCed language such as Java/Go/etc. GC every heap allocation GC is not run on every allocation.
kmeisthax 3y ago
Why would Move/Leak being tied to editions hurt compile times? Presumably old editions would have the required trait bounds added immediately after parsing, and the existing trait machinery would handle that. You'd need extra checks for error messages so you can tell people using !Leak/!Move with old code why they have extra trait bounds, but that's about that.
- Rusky 3y ago
  The idea is that this would be a huge number of extra bounds that need to be processed. This is one of the more computationally expensive parts of the frontend.
minroot 3y ago
Everytime I see Rust, I feel like terrorism happening in PL
thayne 3y ago
> this would rely on the compiler effectively implementing some kind of unbreakable firewall, to prevent a type that does not implement Leak from ever getting used with pre-2024 code Couldn't this be done just by changing how 2021 code is compiled, such that: All types impl Leak All generic type parameters have a Leak constraint added All trait objects are changed to include "+ Leak" In other words, treat pre 2024 code as if it was the equivalent code in the presense of the Leak trait, requiring that all types are Leak.
- withoutboats3 3y ago
  That's what my post is suggesting. The tricky part is making sure you don't sneak a !Leak type from 2024 code into some 2021 code.
  - pornel 3y ago
    If you inserted `+ Leak` on every trait bound in 2021 code at parsing time, then I don't think you'd need any extra firewall later. You'd just treat all code the same, as supporting Leak, and process trait bounds as usual.
    - withoutboats3 3y ago
      Ariel Ben-Yehuda has sent me an example which shows how much harder it is than that, and which this wouldn't cover. I'm going to write a follow up blog post. EDIT: https://without.boats/blog/follow-up-to-changing-the-rules-of-rust/ https://without.boats/blog/follow-up-to-changing-the-rules-o...
      - pornel 3y ago
        With the solution I imagine, this example wouldn't compile. In the crate bar, edition 2021, the code written as: fn foo<T>(input: T) would actually parse as if: fn foo<T: Leak>(input: T) An implicit Leak bound would be inserted into AST of every trait bound in the 2021 edition. Then all code from all editions would be interpreted the same. This way, the 2021 code wouldn't compile, because the 2024 trait requires `<T>` bound, and the 2021 code can only express `<T: Leak>` and nothing less. 2021 code could not work at all with any 2024 code that allows !Leak. libstd would have to add `+ Leak` to all existing APIs. Upgrading to 2024 edition would need a migration to change the bounds.
Klasiaster 3y ago
I think a "linter" that does formal verification would be a good solution. People use things like clippy already and my hope is that using one of the verification tools such as Kani could become equally widespread for checking common things like absence of panics and leaks.
xpe 3y ago
Since a good number of people will arrive at this thread without a lot of surrounding context, would some of us 'in the know' please share some remarks about: 1. Who should read this? Who should care? Why? 2. Help me situate this relative to other Rust commentary, ideas, and recommendations. Where does this sit? 3. What's been happening in the Rust world that may have motivated boats to write this blog post now? Context: For many years, I've enjoyed and written a lot of Rust, but I don't necessarily keep up with the 'inner sanctum' conversations.
- ameliaquining 3y ago
  This is mostly about async Rust. Today, async Rust works, but it's a lot jankier and less polished than the rest of the language; in some ways, it feels kind of beta-quality. The maintainers are extremely aware of this problem and are making serious efforts to fix it. These efforts are expected to take years, because it is a difficult technical problem (most other languages solve it in ways that compromise runtime performance or low-level programmer control, which Rust prioritizes) and because a community-driven open source project has limited resources and takes time to arrive at decisions. This blog post discusses one potential technical direction for addressing a subset of the problems with async Rust, and in particular why the need to preserve backwards compatibility with earlier design decisions, made before async Rust was designed, makes it harder than it might otherwise have been. If you don't use async Rust, you can mostly ignore this. If you do use async Rust, and are interested in how it might (backwards-compatibly!) evolve in the coming years, then the report of the Async Foundations Working Group is probably the best starting point to understand what's going on: https://rust-lang.github.io/wg-async/welcome.html https://rust-lang.github.io/wg-async/welcome.html
  - pclmulqdq 3y ago
    With respect to Rust, a big part of the issue here is that Rust is roughly attempting to turn Javascript into a systems programming language. While `async` as a language feature is a key part of the ideas of JS, it's essentially fundamentally incompatible with 0-overhead systems programming (despite most systems programming using asynchronous concepts). Asynchronous systems code is usually done completely stackless, using state machines and queues of objects, and batched functions that run to completion to transition objects to the next state. This gives you maximum cache-friendliness, and since good performance fundamentally comes from limiting your options, making unbounded state growth hard is a pretty good way to get it. Async functions with stacks are already a performance compromise in comparison. You can do this type of programming in Rust by just not using async. Unfortunately, some fraction of Rust maintainers are pulling back towards Rust being a web backend language (see the earlier comment about Javascript), and that really needs a concept of stackful async, but really doesn't need the same level of performance as a kernel or a filesystem. Rust green threads would even have worked really well for this use case, despite being a huge mismatch with traditional systems programming. In addition, those async functions need some sort of runtime anyway, so it's not even really a standalone language feature distinct from the packages that enable it. So here we are, in a situation around async that makes nobody happy. C++ coroutines, despite the hate they get, and Go green threads, despite the performance, actually seem to be closer to the mark.

190 points by kevincox 3y ago

rust is _almost_ as complex as C++. Introducing these new "edition" would grow the complexity exponentially. I hope they would think twice about the effects on new learners before even experimenting

weinzierl 3y ago
"rust is _almost_ as complex as C++." I don't agree, but even if it were true, C++ has so much accidental complexity (because of its legacy) that you cannot meaningfully compare it with Rust, which by and large is very well thought out. In general, if something is complex in Rust that is because it's in the nature of the problem and not because of some quirkiness in the solution. Borrowing is the prime example. People find it complex, but just because the C++ compiler doesn't check the rules for you, doesn't mean you have follow them in C++ as well (which is tedious and error-prone) or know exactly why you don't have to follow them (which is even harder). I admit though, that the Leakpocalypse (which the article talks about, even if it doesn't call it like that) is an area where accidental complexity lurks.
- vlovich123 3y ago
  Pin is another one. I still can’t wrap my head around it. I’m sure it’s simple and the docs are great, but having read them multiple times I’m not really more clear. I probably need to see a bunch of hands on examples. I wish they’d gone the Move route instead of rushing to 1.0 and then retrofitting in Pin. But I agree, in general things in rust are a lot more thought out and experiments are relegated to unstable which is fine with breaking changes. I do wish though that they had a better versioning story in the language for making backward incompatible changes more a matter of course. It’s great already for “user space” code via editions, but as the author notes making certain kinds of changes brings up risk and uncertainty because there’s not a well trodden path to support changing core semantics in the language and having it work gracefully without needing a migration en masse.
  - majewsky 3y ago
    > I wish they’d gone the Move route instead of rushing to 1.0 and then retrofitting in Pin. That is easy to say, and I don't disagree with the sentiment, but there are valid reasons for setting a 1.0 release date and cutting corners to get there. If they had not done that, there probably was a real risk of running an infinite loop of "let's wait for Move to be implemented", "great, now let's wait for Leak to be implemented", "great, now let's wait for..." and a 1.0 never gets put out. Rust may not be perfect, but at least it's stable and people are actually using it.
  - cmrdporcupine 3y ago
    Pin used to confuse me, but now it's clearer. It's right in the name: Pin this thing to where it is in memory, never try to move it. Rust can and does move things around as it feels the need. So you need something like Pin if what you've built can't move. It's confusing if you're coming from a C++ background because in C++ something only "moves" if you explicitly std::move it. In Rust its effectively the reverse. A crude way of thinking about Rust's call pattern is that it's basically the opposite: as if every call was a std::move operation. So Pin is basically just saying: you can't move this, don't try. The memory can't be moved. And it's mostly used in corner cases where some kind of explicit memory management is involved. Most Rust users should not have to worry about it. It's one of those things that becomes more of a concern if you're writing library code for low level datastructures, or interfacing with C/C++ libraries.
    - laurencerowe 3y ago
      > Most Rust users should not have to worry about it. It's one of those things that becomes more of a concern if you're writing library code for low level datastructures, or interfacing with C/C++ libraries. Don’t all Rust users writing async code have to worry about Pin?
      - brigadier132 3y ago
        Not really, the async libraries hide the implementation details from you. It's an issue for library developers.
- cwzwarich 3y ago
  One advantage C++ has over Rust is that there are multiple fairly compliant C++ implementations, and the process of making them created some shared understanding of how the language actually works from an implementation perspective. There are several aspects of the current Rust language that were effectively defined by arbitrary decisions made by the implementors, which aren't really spelled out anywhere besides the code.
  - tialaramex 3y ago
    Do you have examples? I think in many ways Rust is doing a better job of tying down the tricky details than C++ has. There is no effort to explain how pointer provenance works in C++ at all. The state of the art is if you write a provenance bug, someone from a compiler vendor will gesture at DR/260 which says provenance exists but WG14 (so C, not even C++) declines to explain how it works, and that's all you get. In contrast Aria's "Experiment" (https://doc.rust-lang.org/std/ptr/index.html#strict-provenance https://doc.rust-lang.org/std/ptr/index.html#strict-provenan...) has gone rather well. The vast majority of people who want to do acrobatics with pointers can do what Aria says you have to do to get coherent behaviour, and by following her rules they can make code which works in Rust.
    - uecker 3y ago
      WG14 produced a document clarifying provenance in C (which Aria knew): https://open-std.org/JTC1/SC22/WG14/www/docs/n3005.pdf https://open-std.org/JTC1/SC22/WG14/www/docs/n3005.pdf
      - tialaramex 3y ago
        Given your username, I assume I'm addressing Martin Uecker and I want to commend you for putting the work in on this problem over a long period. While a draft TS is closer to actually resolving a 20+ year old problem it's not there yet and this draft is, if you pardon me, in my opinion more than three months work from publishable. In contrast, Aria went from "We should see whether strict rules are workable" to the current set of Rust nightly features relatively quickly. If I want to attempt pointer stunts in C, N3005 is probably worth reading first, but it isn't actionable from what I can tell. Contrast Aria's changes, I can go try that (in nightly Rust) and see it works or why I can't / mustn't do what I wanted under Strict Provenance and who I ought to talk to about that. I wish you luck for C 29 (?) but that's a distant future, and we're contrasting C++ here, so as with #embed you might take this obvious must-have for C29 and find that somehow WG21 are still so far behind they can't do the same for C++ 29.
rapsey 3y ago
C++ versions are practically a new language. Last two Rust editions required no changes on any of my code. Why are people constantly harping on this obviously untrue idea. New Rust versions and editions make the language _simpler_.
- kaashif 3y ago
  > C++ versions are practically a new language. Last two Rust editions required no changes on any of my code. This juxtaposition kind of makes it seem like that new C++ versions did require changes to your code, did they? My understanding is that C++ maintains backwards compatibility.
  - rapsey 3y ago
    Rust editions are allowed to break backwards compatibility. If you have 2015 edition code and change your Cargo.toml and set it to 2021 edition, the code may no longer compile. My code went from 2015 to 2018 and 2021 by simply changing Cargo.toml. No code changes were necessary and it was still completely idiomatic code. Nothing was deprecated or even outdated. An idiomatic C++11 solution is not likely to be the idiomatic C++17 solution however. Every version of C++ makes deep changes to the actual language and adds a ton of features. Whereas Rust versions and editions tend to make the language simpler as they generally remove limitations.
    - Conscat 3y ago
      Every release of C++ since 11 has relaxed limitations on `constexpr`, and several relaxed limitations on `template`. Many other features have had various rules relaxed in certain releases, including `for` loops, lambdas, labels, moves, and atomics. It's true that there are usually new features which arguably make older code no longer idiomatic, but modernizing code is always optional, so what's really the issue there?
      - rapsey 3y ago
        The issue is the language gets more complex over time. The origin of the discussion was the statement Rust is just as complex as c++, which is completely wrong. Rust changes in small increments and even when they decide on a new edition that can break compatibility, it still barely changes.
duped 3y ago
Rust already has editions. There have been three releases since 1.0.

lukebitts 3y ago

Very interesting read! I really wish we could be unburdened by backwards compatibility so big changes like these are possible, but I realize that would open its own can of worms

Taek 3y ago
Honestly backwards compatibility in rust is already half-assed anyway. Trying to use a fixed version of the compiler (important in some contexts such as security software) is a huge pain in the ass.
- Ar-Curunir 3y ago
  Backwards incompatibility in the language is different from backwards incompatibility in libraries.
- IshKebab 3y ago
  I've never had any notable issues. I've had far fewer compatibility issues with compiling crates than I have compiling C++ libraries or installing Python packages.
- mirashii 3y ago
  It’s about a 10 line nix file, a 10 line dockerfile, a one line rust-toolchain.toml.
- ekidd 3y ago
  Rust has an excellent story for running old code on new compilers. For example, I've been using Rust in production since just after 1.0, so about 8 years now. I maintain 10-20 libraries and tools, some with hundreds of dependencies. If I update a project with 200 dependencies that hasn't been touched in 3 years, that's 3x200 = 600 "dependency-years" worth of old code being run on a new compiler. And usually it either works flawlessly, or it can be fixed in under 5 minutes. Very occasionally I'm forced to update my code to use a new library version that requires some code changes on my end. I've also maintained long-lived C, C++, Ruby and Python projects. Those projects tend to have far more problems with new compilers or runtimes than Rust, in my experience. Updating a large C++ program to newest version of Visual C++, for example, can be a mess. However, Rust obviously does not support running new code on old compilers. And because stable Rust is almost always a safe upgrade, many popular libraries don't worry too much about supporting 3-year-old compilers. (This is super irritating for distros like Debian.) Which if you're working on a regulated embedded system that requires you to use a specific old compiler, well, it's a problem. Realistically, in that case you'll want to vendor and freeze all your dependencies and back-port specific security fixes as needed. If you're not allowed to update your compiler, you probably shouldn't be mass-updating your dependencies, either. Basically, Rust got so exceptionally good at running old code on new compilers that the library ecosystem developed the habit of dropping support for old compilers too quickly for some downstream LTS maintainers. And as a library maintainer, I'm absolutely part of the problem—I don't actually care about supporting 5 year old compilers for free. On some level, I probably should, but...

Ygg2 3y ago

I think eventually or at some point there might be a need for Rust 2.0 even if it won't be called Rust. Think something like JavaScript and TypeScript. A language that's a super set except the former would be soft deprecated. I know this is highly unpopular opinion, but I don't consider permanent backwards compatibility a good thing.

vimpunk 3y ago
What parts of Rust would you like changed, though? I think the current version, with its minor warts, is a very pleasant and consistent experience for the most part -- very much unlike C++. So I'm not sure what I'd change to such a drastic degree that it'd require such breaking changes.
- lawn 3y ago
  async still feels half-baked and not pleasant like the rest of the language. Now, it may be something that still can be fixed and improved upon, but a Rust 2.0 with nicer async experience would be a big win.
- Taek 3y ago
  I would definitely want to see async totally overhauled. Also it would be great if major components of the standard library were all version 1. And if I want to risk going too far, I'd also advocate for refactoring the type system to be slightly less generic in favor of chasing faster compile times. Also, ideally the lead designer of SafeLang2.0 would be much more heavy on engineering experience than PL theory. Rust made a lot of tradeoffs that were beautiful in PL theory but are annoying to use on a day to day basis. I honestly think rust is equal parts amazing ideas and mediocre ideas, and I think a slighky less ambitious language would still be almost as safe (world class compared to any other language) but 10x more fun to use.
  - gmnash 3y ago
    A complete overhaul of async sounds nice in the abstract, but without exact details it’s difficult to see how it would be better than what exists. Similarly, I’m curious as to what the details of a “slightly less generic” type system entails.

amluto 3y ago

I wonder if the new-edition approach could be done by interpreting pre-2024 code as if everything had a Leak bound. So instead of a not-Leak type being disallowed when passed to a pre-2024 module, it would not pass type checking because everything in the pre-2024 module required Leak.

jeff-davis 3y ago
The article addresses this: "However - and this is the big, enormous caveat of this section - this would rely on the compiler effectively implementing some kind of unbreakable firewall, to prevent a type that does not implement Leak from ever getting used with pre-2024 code."
- amluto 3y ago
  What I mean is: instead of having a “firewall” in the compiler, is there a way to interpret pre-2024 code such that it has correct but conservative Leak bounds? Then there would be more confidence that, if a mixed-edition program type-checks, then it’s correct.
  - ComputerGuru 3y ago
    It took me a minute, but the firewall is talking about the reverse direction of preventing post-2024 code without Leak being used in pre-2024 code.
AndrewDucker 3y ago
How would it know the code was pre 2024?

nynx 3y ago

I’m actually glad the Move and Leak didn’t get in. They both have fairly niche usecases but they’d have to be spattered all over basically all rust code.

jeff-davis 3y ago
Why do you say they'd have to be splattered over basically all rust code? The post compares them to Send and Sync, and I don't have to use those very frequently (though I don't write a lot of rust code, either).
- nynx 3y ago
  People use Arc and Rc all the time, which would need +?Leak annotations. Things like Option would, I think, also need + ?Move + ?Leak annotations.
  - bryanlarsen 3y ago
    If it is an unobtrusive as the existing ?Trait annotations such as ?Sized it's not very much of a burden.
  - deleted 3y ago
    [deleted]
- mlindner 3y ago
  Because features are always used in the worst (most evil?) way possible by inexperienced programmers. One of the benefits of Rust is that it makes many evil things either impossible or easily detectable.

jeff-davis 3y ago

The Leak trait could be interesting when it comes to C code that calls rust code that calls C code that might longjmp() over the rust code in the middle. Right now longjmp()ing over rust code is not a great idea for a couple reasons, but because leaks are currently always allowed in rust, it arguably follows the rules at least in some cases (doing so could cause problems, of course). If the Leak trait were introduced, then skipping destructors would be clearly the wrong thing to do if there are types on the stack that don't implement Leak. That would clearly require some way to either prevent longjmp() from ever jumping over rust code (e.g. always catch it in C code and turn it into a special return code), or find some way to catch the longjmp and turn it into an unwind in rust (e.g. a panic?) and then turn it back into a longjmp to go back into C.

connicpu 3y ago
Honestly, longjmp is just extremely problematic for any code that isn't pure C. Even just adding C++ into the mix can easily make longjmp something that can't be used without invoking UB. "If replacing std::longjmp with throw and setjmp with catch would invoke a non-trivial destructor for any automatic object, the behavior of such std::longjmp is undefined."[1] [1]: https://en.cppreference.com/w/cpp/utility/program/longjmp https://en.cppreference.com/w/cpp/utility/program/longjmp
- jeff-davis 3y ago
  setjmp/longjmp exist in a lot of real world C code, so there is a lot of value in allowing rust to deal with longjmp in a defined way at least in some narrow cases. And I don't think it's impossible -- it probably requires some special wrappers (perhaps bindgen could generate them?) that would call setjmp before entering the C code. setjmp has its own set of problems, but I don't believe those are impossible to solve, either.
LegionMammal978 3y ago
> Right now longjmp()ing over rust code is not a great idea for a couple reasons, but because leaks are currently always allowed in rust, it arguably follows the rules at least in some cases (doing so could cause problems, of course). In the general case, skipping destructors of objects you don't own (through longjmp(), killing a single thread, etc.) has never been allowed in Rust: at any particular point in the program, you're only allowed to skip destructors of objects that you currently own, or can otherwise gain ownership of. For instance, the thread::scope() API, after running the spawning thread's closure, uses a destructor* to join all the created threads. However, if a program could longjmp() past that destructor, then the spawning thread could access its variables again while the created threads are still using them, resulting in a data race. (This is the same issue that the original thread::scoped() API had.) let mut x = 0; let mut jmp_buf = JmpBuf::default(); if setjmp(&mut jmp_buf) == 0 { std::thread::scope(|s| { s.spawn(|| loop { x += 1; }); longjmp(&jmp_buf, 1); }); } else { loop { println!("{x}"); } /* the created thread is still changing the value of x! */ } That is to say, longjmp() has always been a very unsafe operation, which can only be sound if you're in control of every single destructor which it skips. A Leak trait wouldn't change anything in that regard. * Or rather, a catch_unwind() followed by a resume_unwind(), which is mostly equivalent to a destructor in this context.

notnullorvoid 3y ago

Here's an idea, for a kind of soft rollout. Implement a optional feature for 2021-edition crates that allow specifying Leak. The trait has no effect in 2021. In next-edition: - 2021 crates that turned Leak feature uses it just as a next-edition crate would. Only things explicitly marked with Leak in the crate can leak. - 2021 crate that didn't opt-in to Leak feature, automatically marks everything in the crate with Leak.

bryanlarsen 3y ago
I think the big caveat mentioned in the editions section (https://without.boats/blog/changing-the-rules-of-rust/#editions https://without.boats/blog/changing-the-rules-of-rust/#editi...) still applies to your suggestion. The author concludes in that section "I don’t know if this is even possible to implement, probably it would have pretty bad effect on compile times at". Your refinement seems like a nice addition to the author's suggestion, if it is possible and doesn't effect compile times too much. Hopefully somebody smarter than me takes a run at the problem.
kmeisthax 3y ago
from __future__ import Leak

hedora 3y ago

The lack of a Leak-style trait is so painful for me that I’d strongly consider switching to a different language that was basically just rust + Leak. (Adding move and removing pin would be nice to have too.) The problem with not having Leak is that it forces all futures to be ‘static, which effectively disables the borrow checker whenever you cross an async spawn boundary. Arc is the standard solution, but, for multi-threaded code, it can be expensive (though also more predictable, in fairness) than having a GC. The only other solution (which I prefer) is to use an unsafe version of spawn that assumes the future it returns will not be leaked. This certainly breaks the rust promise of supporting safe, zero cost abstractions.

sabageti 3y ago
I have seen many comments like this, and I would like to understand better I feel like I'm missing something. In my daily use case(web api) of async rust we use rarely spawn. What are the use case of intensive spawn usage ? And there is tokio::spawn_local() that doesn't require Send
scottlamb 3y ago
> Arc is the standard solution, but, for multi-threaded code, it can be expensive (though also more predictable, in fairness) than having a GC. Do you have measurements to back that up? I would expect it to be a lot cheaper than GC because GCed language such as Java/Go/etc. GC every heap allocation (and Java in particular doesn't have unboxed objects, so there are a lot of heap allocations!) where Arc<RequestState> would bundle a bunch of things together. That said, I'm not in love with the Arc solution either. It's not just the performance impact. The programming model of scoped concurrency is just much more pleasant. Giving up that and then also having to deal with putting everything into an Arc, and likely also then wrapping it in a MutexLock, even when it should really be one thread at a time dealing with the value in question, just doesn't do anything good for readability and writability of code...
- charcircuit 3y ago
  >because GCed language such as Java/Go/etc. GC every heap allocation GC is not run on every allocation.

kmeisthax 3y ago

Why would Move/Leak being tied to editions hurt compile times? Presumably old editions would have the required trait bounds added immediately after parsing, and the existing trait machinery would handle that. You'd need extra checks for error messages so you can tell people using !Leak/!Move with old code why they have extra trait bounds, but that's about that.

Rusky 3y ago
The idea is that this would be a huge number of extra bounds that need to be processed. This is one of the more computationally expensive parts of the frontend.

minroot 3y ago

Everytime I see Rust, I feel like terrorism happening in PL

thayne 3y ago

> this would rely on the compiler effectively implementing some kind of unbreakable firewall, to prevent a type that does not implement Leak from ever getting used with pre-2024 code Couldn't this be done just by changing how 2021 code is compiled, such that: All types impl Leak All generic type parameters have a Leak constraint added All trait objects are changed to include "+ Leak" In other words, treat pre 2024 code as if it was the equivalent code in the presense of the Leak trait, requiring that all types are Leak.

withoutboats3 3y ago
That's what my post is suggesting. The tricky part is making sure you don't sneak a !Leak type from 2024 code into some 2021 code.
- pornel 3y ago
  If you inserted `+ Leak` on every trait bound in 2021 code at parsing time, then I don't think you'd need any extra firewall later. You'd just treat all code the same, as supporting Leak, and process trait bounds as usual.
  - withoutboats3 3y ago
    Ariel Ben-Yehuda has sent me an example which shows how much harder it is than that, and which this wouldn't cover. I'm going to write a follow up blog post. EDIT: https://without.boats/blog/follow-up-to-changing-the-rules-of-rust/ https://without.boats/blog/follow-up-to-changing-the-rules-o...
    - pornel 3y ago
      With the solution I imagine, this example wouldn't compile. In the crate bar, edition 2021, the code written as: fn foo<T>(input: T) would actually parse as if: fn foo<T: Leak>(input: T) An implicit Leak bound would be inserted into AST of every trait bound in the 2021 edition. Then all code from all editions would be interpreted the same. This way, the 2021 code wouldn't compile, because the 2024 trait requires `<T>` bound, and the 2021 code can only express `<T: Leak>` and nothing less. 2021 code could not work at all with any 2024 code that allows !Leak. libstd would have to add `+ Leak` to all existing APIs. Upgrading to 2024 edition would need a migration to change the bounds.

Klasiaster 3y ago

I think a "linter" that does formal verification would be a good solution. People use things like clippy already and my hope is that using one of the verification tools such as Kani could become equally widespread for checking common things like absence of panics and leaks.

xpe 3y ago

Since a good number of people will arrive at this thread without a lot of surrounding context, would some of us 'in the know' please share some remarks about: 1. Who should read this? Who should care? Why? 2. Help me situate this relative to other Rust commentary, ideas, and recommendations. Where does this sit? 3. What's been happening in the Rust world that may have motivated boats to write this blog post now? Context: For many years, I've enjoyed and written a lot of Rust, but I don't necessarily keep up with the 'inner sanctum' conversations.

ameliaquining 3y ago
This is mostly about async Rust. Today, async Rust works, but it's a lot jankier and less polished than the rest of the language; in some ways, it feels kind of beta-quality. The maintainers are extremely aware of this problem and are making serious efforts to fix it. These efforts are expected to take years, because it is a difficult technical problem (most other languages solve it in ways that compromise runtime performance or low-level programmer control, which Rust prioritizes) and because a community-driven open source project has limited resources and takes time to arrive at decisions. This blog post discusses one potential technical direction for addressing a subset of the problems with async Rust, and in particular why the need to preserve backwards compatibility with earlier design decisions, made before async Rust was designed, makes it harder than it might otherwise have been. If you don't use async Rust, you can mostly ignore this. If you do use async Rust, and are interested in how it might (backwards-compatibly!) evolve in the coming years, then the report of the Async Foundations Working Group is probably the best starting point to understand what's going on: https://rust-lang.github.io/wg-async/welcome.html https://rust-lang.github.io/wg-async/welcome.html
- pclmulqdq 3y ago
  With respect to Rust, a big part of the issue here is that Rust is roughly attempting to turn Javascript into a systems programming language. While `async` as a language feature is a key part of the ideas of JS, it's essentially fundamentally incompatible with 0-overhead systems programming (despite most systems programming using asynchronous concepts). Asynchronous systems code is usually done completely stackless, using state machines and queues of objects, and batched functions that run to completion to transition objects to the next state. This gives you maximum cache-friendliness, and since good performance fundamentally comes from limiting your options, making unbounded state growth hard is a pretty good way to get it. Async functions with stacks are already a performance compromise in comparison. You can do this type of programming in Rust by just not using async. Unfortunately, some fraction of Rust maintainers are pulling back towards Rust being a web backend language (see the earlier comment about Javascript), and that really needs a concept of stackful async, but really doesn't need the same level of performance as a kernel or a filesystem. Rust green threads would even have worked really well for this use case, despite being a huge mismatch with traditional systems programming. In addition, those async functions need some sort of runtime anyway, so it's not even really a standalone language feature distinct from the packages that enable it. So here we are, in a situation around async that makes nobody happy. C++ coroutines, despite the hate they get, and Go green threads, despite the performance, actually seem to be closer to the mark.