4 ms·
When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -
by Waterluvian 7y ago
When writing a language like Rust, is the biggest challenge simply deciding what Rust's features and behaviors should be? And implementing the syntax and Rust -> LLVM compiler is really just a chore for the individuals who are super familiar with the implementation of these languages? Or is the technical implementation also genuinely challenging and non-obvious?
- kevinmgranger 7y agoThe concept of lifetime management is relatively novel and uncharted territory, if I understand correctly. There's only some prior art. So implementing that must have been an adventure and a half. And while I'm sure the folks who work on these languages are wonderfully intelligent people, let's dispel this notion that you need to be a super genius to implement a compiler or something like that! It seems magical, like one of the hardest things you could program-- but take a look through crafting interpreters, if you will: http://craftinginterpreters.com/ http://craftinginterpreters.com/ "Nothing is particularly hard if you break it down into small jobs." - Henry Ford
- Waterluvian 7y agoI walked through the Java portion of Crafting Interpreters and indeed, they can be much simpler than you might imagine in your early years. I was more just paying a compliment than suggesting that maintainers are superhuman. Everyone who ships productive code is a wizard and you can be too! I modified my original question to avoid a potential distraction from what I want to talk about. Thanks!
- xscott 7y agoDo you have any links for the prior art? I'm sincerely interested, thank you if you do. I'd also be interested in any articles (or even blog posts) that describe Rust's process for that from a compiler writer's point of view.
- tadfisher 7y agoA language like Rust aims for "zero-cost abstractions", which means the features and behaviors of the language must be evaluated in the context of the implementation.
- Waterluvian 7y agoLet me attempt to unpack this into language I think I understand: It's up to Rust's compiler to verify all the contracts made, because in order for the binary to be "zero-cost", none of those checks are being done at runtime. If you looked at the output assembly, you would see what looks like very carefully written code that shows no explicit signs of protecting itself. Ie. there's no swaths of boilerplate assembly doing borrow checking, out of bounds checking, etc.
- steveklabnik 7y agoYou're not wrong, exactly, but I'm not 100% sure this is right. The way I would put it is this: Rust has made certain commitments about performance. This means that language changes have to be made in the context of how they are implemented, because releasing a language feature that causes a significant performance degradation would make it not a good fit by definition.
- eximius 7y agoThat might be a stronger way of putting it than I would. The canonical example is iterator chains with complex logic compiling down into vectorised and unrolled loops. Powerful logical abstractions are used by the compiler to generate code that does what it says to do without the runtime cost of closures or whatever other logical but not mechanically necessary things you have. Iterators can't go out of bounds, so the compiler can elide those checks. There are still some runtime costs at the intersection of safety, ergonomics, and performance. Bounds checking, overflow checking. But they have escape hatches and are relatively rare in the language. Most things do compile out.
- steveklabnik 7y agoFirst of all, deciding features and behaviors is not simple. :) There are a number of technical implementation challenges in the compiler. It is a large project, and Rust's got a really intense stability policy. The compiler was bootstrapped very early, when the rate of change of the language itself was still "multiple things per day." This introduced significant architectural debt. There have been multiple projects that have re-written massive parts of the compiler, and more ongoing. For example, non-lexical lifetimes required inventing an entire additional intermediate language, re-writing the compiler to use it, and making sure that everything kept working while doing so. More recently, the compiler has been being re-done from a classic, multiple-pass architecture to a more Roslyn, "query-based" one. Again, this is being done entirely "in-flight", while keeping a project that's used by a lot of folks stable. The rust-analyzer has made this project even more interesting; a "librarification" strategy is being undergone to make the compiler more modular. For some numbers on this kind of thing, https://twitter.com/steveklabnik/status/1211667962379276288/photo/1 https://twitter.com/steveklabnik/status/1211667962379276288/... and https://twitter.com/steveklabnik/status/1211717308143587334/photo/1 https://twitter.com/steveklabnik/status/1211717308143587334/...
- Waterluvian 7y agoUsing the word "simply" is a quirk of mine. I'm very much trying to express that I think it's by far the hardest part. Thanks for your response!
- steveklabnik 7y agoIt's all good :) You're welcome!
- j88439h84 7y ago> Rust's got a really intense stability policy. I know the code won't stop running, but I wonder how soon it stops being idiomatic. If it's not idiomatic, it's harder to maintain due to unfamiliar style and structure. Does Rust have measures to deal with this issue?
- deleted 7y ago