5 ms·

I've seen this complaint about the Rust compiler a number of times now, here and at other sites. Isn't one of the main selling points of Rust that it's supposed

by Jenny1975 11y ago

I've seen this complaint about the Rust compiler a number of times now, here and at other sites. Isn't one of the main selling points of Rust that it's supposed to make it much harder to write code that crashes? It is my understanding that the Rust compiler is mostly written in Rust. While maybe these ICEs aren't traditional segfaults that can be exploited, they're still unexpected crashes none the less! The software stops working without warning. If the best Rust experts around have so much trouble avoiding those kinds of crashes, then how is an average Rust user, or even a complete beginner like me, supposed to write Rust code that's any less susceptible to crashing?

steveklabnik 11y ago
TL;DR: don't underestimate the power of technical debt. Longer answer: The Rust compiler was written in Rust before the Rust language was even what it was today. As one particular example, the AST used to be garbage collected! And when you have a codebase that's 320,000 lines long, you don't update the entire codebase to the latest idiom, because if you did, you'd never actually ship the language. Basically no other Rust program will ever have to contend with this kind of situation. As such, while I can sympathize with this line of argument, I don't think it's representative. EDIT: I previously had 750,000 but it's actually 323,326 LOC as of right now. Whoops!
- Jenny1975 11y ago
  Legacy code and inexperience with new technologies are not problems that are unique to Rust and the Rust compiler. And presumably Rust will continue to evolve, will it not? If my team starts a large, multi-year project using Rust 1.1 today, does Rust offer anything that will help us avoid the sorts of crashes that the Rust compiler suffers from?
  - steveklabnik 11y ago
    Your language changing literally every other day is a problem unique to a pre-1.0 language. You're right that technical debt in general is not. > And presumably Rust will continue to evolve, will it not? Right, but it will evolve in a backwards-compatible way, rather than in a "the fundamental semantics of the language have changed Yet Again" kind of way.
  - Aatch 11y ago
    Just to expand on what Steve said. An evolving language is one thing. Pre-1.0 was more like many stages of violent metamorphasis. Some changes to the language required changing almost every line of code in the tree. Normal technical debt is when you find an old approach doesn't work any more and you need to fix it. Much of rustc's technical debt is from things like features disappearing from underneath it, or things designed before the features that you would use now even existed. Looking at rustc as a regular large Rust project is unfair. A new project can at least assume that the features you are using today will exist tomorrow and design with that in mind. Your designs may become obsolete due to new features, but you aren't going to have to pick between a hacky workaround and a redesign/rewrite of 1000's of lines of code.
  - veddan 11y ago
    Sure, Rust will continue to evolve. But many changes that happened in the past were on the scale of "Java 10 decides to get rid of classes".
jroesch 11y ago
Steve has hit this on the head, there are lots of places where we are using older abstractions that we have had to slowly refactor out of the compiler and like all other software development this won't be instantaneous. We definitely have some less then optimal code in the compiler from a design standpoint. I think it is also important to recognize that all compilers have this problem and they all stabilize over time. It is an issue orthogonal of safety. These aren't segmentation faults or some kind of memory error but the compiler signaling an internal invariant was violated. This can happen easily because we are writing a very large program that reads user input, interprets it, and tries to emit a program. This can happen in multiple situations. One example is the compiler may have a logic bug that violates an internal invariant resulting in an ICE (instead of emitting incorrect code). Other times it is as simple as the code we have been given is invalid and the error wasn't properly handled, and you end up seeing it in a non-user friendly way (as an ICE). We are working hard to move the compiler to a more modern, robust design as soon as possible but even with a great design it will be an iterative process that we continue to do over the lifetime of the language, just like everyone else.
n_ham 11y ago
ICEs are not crashes due to memory unsafety, they are logical errors in the compiler code. For example, here's about the worst rustc crash I've found: https://github.com/rust-lang/rust/issues/21946 https://github.com/rust-lang/rust/issues/21946. It doesn't even give you an ICE, it terminates with a stack overflow. So no, Rust cannot magically fix logic errors in large, complicated codebases, but that's not something that a complete beginner needs to worry about thankfully.
- arielby 11y ago
  That's just a vanilla "divergence" stack overflow - an essentially harmless consequence of the type-system's Turing-completeness.
  - kibwen 11y ago
    You don't even need Turing-completeness, just any form of recursion will do.
arielby 11y ago
Most crashes are in type inference, associated types handling, and error recovery, which are very complicated things, especially in combination. We also basically rushed features in for 1.0.0 (1.1/1.2 should be much better in terms of these).

steveklabnik 11y ago

TL;DR: don't underestimate the power of technical debt. Longer answer: The Rust compiler was written in Rust before the Rust language was even what it was today. As one particular example, the AST used to be garbage collected! And when you have a codebase that's 320,000 lines long, you don't update the entire codebase to the latest idiom, because if you did, you'd never actually ship the language. Basically no other Rust program will ever have to contend with this kind of situation. As such, while I can sympathize with this line of argument, I don't think it's representative. EDIT: I previously had 750,000 but it's actually 323,326 LOC as of right now. Whoops!

Jenny1975 11y ago
Legacy code and inexperience with new technologies are not problems that are unique to Rust and the Rust compiler. And presumably Rust will continue to evolve, will it not? If my team starts a large, multi-year project using Rust 1.1 today, does Rust offer anything that will help us avoid the sorts of crashes that the Rust compiler suffers from?
- steveklabnik 11y ago
  Your language changing literally every other day is a problem unique to a pre-1.0 language. You're right that technical debt in general is not. > And presumably Rust will continue to evolve, will it not? Right, but it will evolve in a backwards-compatible way, rather than in a "the fundamental semantics of the language have changed Yet Again" kind of way.
- Aatch 11y ago
  Just to expand on what Steve said. An evolving language is one thing. Pre-1.0 was more like many stages of violent metamorphasis. Some changes to the language required changing almost every line of code in the tree. Normal technical debt is when you find an old approach doesn't work any more and you need to fix it. Much of rustc's technical debt is from things like features disappearing from underneath it, or things designed before the features that you would use now even existed. Looking at rustc as a regular large Rust project is unfair. A new project can at least assume that the features you are using today will exist tomorrow and design with that in mind. Your designs may become obsolete due to new features, but you aren't going to have to pick between a hacky workaround and a redesign/rewrite of 1000's of lines of code.
- veddan 11y ago
  Sure, Rust will continue to evolve. But many changes that happened in the past were on the scale of "Java 10 decides to get rid of classes".

jroesch 11y ago

Steve has hit this on the head, there are lots of places where we are using older abstractions that we have had to slowly refactor out of the compiler and like all other software development this won't be instantaneous. We definitely have some less then optimal code in the compiler from a design standpoint. I think it is also important to recognize that all compilers have this problem and they all stabilize over time. It is an issue orthogonal of safety. These aren't segmentation faults or some kind of memory error but the compiler signaling an internal invariant was violated. This can happen easily because we are writing a very large program that reads user input, interprets it, and tries to emit a program. This can happen in multiple situations. One example is the compiler may have a logic bug that violates an internal invariant resulting in an ICE (instead of emitting incorrect code). Other times it is as simple as the code we have been given is invalid and the error wasn't properly handled, and you end up seeing it in a non-user friendly way (as an ICE). We are working hard to move the compiler to a more modern, robust design as soon as possible but even with a great design it will be an iterative process that we continue to do over the lifetime of the language, just like everyone else.

n_ham 11y ago

ICEs are not crashes due to memory unsafety, they are logical errors in the compiler code. For example, here's about the worst rustc crash I've found: https://github.com/rust-lang/rust/issues/21946 https://github.com/rust-lang/rust/issues/21946. It doesn't even give you an ICE, it terminates with a stack overflow. So no, Rust cannot magically fix logic errors in large, complicated codebases, but that's not something that a complete beginner needs to worry about thankfully.

arielby 11y ago
That's just a vanilla "divergence" stack overflow - an essentially harmless consequence of the type-system's Turing-completeness.
- kibwen 11y ago
  You don't even need Turing-completeness, just any form of recursion will do.

arielby 11y ago

Most crashes are in type inference, associated types handling, and error recovery, which are very complicated things, especially in combination. We also basically rushed features in for 1.0.0 (1.1/1.2 should be much better in terms of these).