16 ms·

Roc rewrites the compiler in Zig

• cma256 2y ago
  It will be interesting to compare compile times afterwards. I'm also interested if there's any Rust features that are 'taken for granted' which will make the Zig version less pleasant to read or write. Hopefully the old Rust repo is archived so I can reference it a year from now!
  • badfishblues 2y ago
    It will be impossible to ever truly do a one-to-one comparison. First, there may be a ton of code in the existing implementation that gets cut out by a mass rewrite due to better, more informed, architecture decisions. On the other hand, it may be possible that new features find their way into the rewrite that cause the fully Zig implementation to be larger than the original Rust-Zig implementation. I don't think any comparison will be fully "fair". A smaller implementation could be argued as only being possible in Rust or Zig because of its advantages over the other language. I expect results to be controversial.
    • bhansconnect 2y ago
      Yeah, the rewrite will clean up a ton of technical debt for us. So it will very much be a bad comparison.
    • gauge_field 2y ago
      I am also curios about at least some benchmark even if it is not one-to-one perfect comparison. There should be at least one case-study where they have shorter feedback loop with Zig than with Rust. It would be interesting to see that.
• hassleblad23 2y ago
  Zig seems to be a good choice here.
  • UncleOxidant 2y ago
    > Rust's compile times are slow. Zig's compile times are fast. This is not the only reason, but it's a big reason. So why not something like OCaml that has fast compile times and a lot of nice features for writing compilers? (pattern matching, etc.)
    • whytevuhuni 2y ago
      Because, like he said, compile times matter. This includes the Roc compiler too. Zig is significantly faster than OCaml.
      • LAC-Tech 2y ago
        The Ocaml compiler is significantly faster than the Zig one. It's one of the fastest I've seen.
        • whytevuhuni 2y ago
          Ah, sorry, I was referring to the speed of the resulting Roc compiler. A Roc compiler written in Zig would compile Roc code significantly faster than a Roc compiler written in OCaml.
          • LAC-Tech 2y ago
            Ah! That makes sense.
    • platinumrad 2y ago
      The same reason why they're not self-hosting. They want compiler performance to be in the top C, C++, Rust, Zig tier. OCaml isn't slow, but at best it's in the second Java, C#, etc. tier.
    • henning 2y ago
      Zig's union enums get you something kind of like pattern matching. https://zig.guide/language-basics/unions/ https://zig.guide/language-basics/unions/
• UncleOxidant 2y ago
  What is Roc? Here it is: https://www.roc-lang.org/ https://www.roc-lang.org/
• pjmlp 2y ago
  The way compile times are stressed all over the place is a very good point. Papercuts like this do make or break adoption. Even C++ is improving on that front.
  • nurettin 2y ago
    ccache made C++ usable for me.
    • Cyph0n 2y ago
      https://github.com/mozilla/sccache https://github.com/mozilla/sccache
    • pjmlp 2y ago
      It has been mostly usable to me with binary libraries, precompiled headers, incremental compiling and linking, and now modules. Although ClearMake object sharing was quite neat experience back in the day.
  • pcwalton 2y ago
    As is Rust. And the C++ compiler performance improvements in LLVM apply to Rust.
    • pjmlp 2y ago
      I was talking about C++ 20 modules and C++23 modularised standard library, and that has little to do with LLVM, rather clang. LLVM improvements will do little to improve Rust, because the major issue is the lack of parallelism and massive amount of LLVM IR that the fronted gives to the backend to handle.
      • pcwalton 2y ago
        > LLVM improvements will do little to improve Rust Falsified many times during Rust's development.
• rubenvanwyk 2y ago
  After watching YT talks on Roc using Rust and Zig, I always wondered they didn't just opt for Zig only. Interesting to see this.
  • bhansconnect 2y ago
    Yeah, the compiler was started in 2019 when zig wasn't nearly as viable of an option. On top of that, you don't necessarily know what you want until you build something. I'm part of the core team, and I'm honestly still a bit surprised we're switching to zig. I think it makes sense, it just kinda finally aligned and was decided now. I guess enough technical debt piled up and the core of the language was clearly discovered. So we really know what we want now.
• bmacho 2y ago
  Strangely they don't mention that (allegedly? According to them at least) they already use Zig for the standard library[0]. Which was also started to be written in Rust, then it got rewritten in Zig: [0] : https://www.roc-lang.org/faq#rust-and-zig https://www.roc-lang.org/faq#rust-and-zig
  • laserbeam 2y ago
    > (and to unblock updating to the latest Zig version, which we've use for our stdlib for years) It's somewhere in the middle of the list of the "Why now?" section.
  • thegeekpirate 2y ago
    They do mention it but there's only 10k lines of Zig, so it's hardly a blip in the grand scheme of things.
• baq 2y ago
  A very good discussion and hard to disagree with conclusions. Also, educational.
  • cardanome 2y ago
    Richard Feldman is an amazing educator. He got me into Elm back in the days. I am a bit sad that Roc is following similar pitfalls as Elm in its quest to be simple by rejecting to have more advanced type system features. That just does not work. In dynamic languages you can opt for minimalism because everything goes by default but in fully statically typed languages things can painful quickly. Even golang had to add generics. Still many amazing ideas in the language. The platform concept is pretty neat. Not doing implicit currying is a good move.
    • 1-more 2y ago
      If you're talking about typeclasses, Roc calls them "abilities" https://www.roc-lang.org/abilities https://www.roc-lang.org/abilities
    • Hasnep 2y ago
      But Roc already has generics, Go has managed to become very successful despite having a simpler type system than Roc.
    • bbkane 2y ago
      What particular type system featured do you miss? I think Elm proved that a restricted type system DOES WORK for large amounts of software. It's been a few years since I've written Elm, so I don't recall specific painful memories I've had with the type system. One thing that makes Go's restrictive type system more bearable is a fantastic stlib package for analyzing and generating Go code. I wonder if Roc will get anything similar.
    • forgotmypass1 2y ago
      What kind of advanced type system features?
• greener_grass 2y ago
  > While we're at it, we also want to convert it to use recursive descent; it was originally written using parser combinators because that was what I was comfortable with at the time Aren't parser combinators just one way to do recursive descent?
  • jjk7 2y ago
    Explicitly hand-writing the parsing logic allows you to get more specific in error states and messages that a combinator can't.
• Lyngbakr 2y ago
  The rationale given makes perfect sense, but isn't it risky to rewrite in a language that is yet to hit 1.0?
  • stevage 2y ago
    Much better before than after!
    • Lyngbakr 2y ago
      Why? There's the possibility that Zig will introduce breaking changes and the Roc compiler will have to be revised in light of those changes.
      • brokencode 2y ago
        I’m assuming the previous commenter thought you were referring to Roc being pre v1 and not Zig. There are probably some risks to it. And I think that you wouldn’t want to release a Roc v1 before Zig v1 as well. But if things are working well now, you could always stay on an older version of Zig while you rewrite things for breaking changes in a newer version. Still potentially a pain, but Rust is post v1 and they ended up deciding to rewrite anyway, so there are no guarantees in any approach.
  • bhansconnect 2y ago
    For sure, but we have a really good relationship with the zig folks and they are willing to help us out. On top of that, zig has gotten a lot more robust and stable of the last few releases. Upgrading is getting smoother. But yeah, it is a risk.
• IshKebab 2y ago
  This is the first justification of not using Rust that I actually agree with. Well written. I recommend reading Roc's FAQ too - it's got some really great points. E.g. I'm internally screaming YESSS! to this: https://www.roc-lang.org/faq.html#curried-functions https://www.roc-lang.org/faq.html#curried-functions But then it has other weird features too, like they seem to be really emphasising "friendliness" (great!) but then it has weird syntax like `\` for anonymous functions (I dunno where that dumb syntax came from by Nix also uses it and it's pretty awful). Omitting brackets and commas for function calls is also a bad decision if you care about friendliness. I have yet to find a language where that doesn't make the code harder to read and understand.
  • Skinney 2y ago
    The syntax came from Elm, which got it’s syntax from Haskell (where Nix also got it from) which got its syntax from ML. It’s a syntax that’s several decades old at this point. It’s different, but not harder. If you learned ML first, you’d found Algol/C-like syntax equally strange.
    • wk_end 2y ago
      (ETA: speaking strictly about anonymous functions; on rereading you might be talking about the absence of parens and commas for function application.) That's not ML syntax. Haskell got it from Miranda, I guess? In SML you use the `fn` keyword to create an anonymous function; in Ocaml, it's `fun` instead.
      • throwaway17_17 2y ago
        I believe the `\` character for functions is original to Haskell. Miranda does not have anonymous functions as a part of the language.
        • pezezin 2y ago
          The \ is a simplified lambda, because most programmers can't type λ easily.
• greener_grass 2y ago
  I can't help but feel like the Roc team has an attitude of "imperative programming for me, but not for thee". And now they are doubling down on that by moving from "OCaml meets C++" to "C, the good parts"! If FP isn't good for writing a compiler, what is it good for?
  • btreecat 2y ago
    > If FP isn't good for writing a compiler, what is it good for? Summing the Fibonacci sequence I guess.
    • agumonkey 2y ago
      other FP languages bootstrapped onto themselves IIRC
    • incrudible 2y ago
      FP is bad for computing the Fibonacci series unless you have the compiler optimization to turn it into a loop (as seen in imperative languages). To be fair, most practical FP languages have that, but I never saw the appeal for a strictly functional general purpose language. The situations where I wished one could not use imperative constructs are very domain specific.
  • munchler 2y ago
    As an FP fan, I agree. This is disappointing.
  • whlr 2y ago
    According to [1], they want to use a systems-level language for performance. [1] https://www.roc-lang.org/faq#self-hosted-compiler https://www.roc-lang.org/faq#self-hosted-compiler
    • tialaramex 2y ago
      "The split of Rust for the compiler and Zig for the standard library has worked well so far, and there are no plans to change it." I assume that statement will need updating.
• chambers 2y ago
  This decisions looks well-reasoned. It acknowledges the strengths of their current implementation, and explains why another language may serve their goals even better. I can follow along as someone interested in general software decision-making, even though I'm not particularly invested in either language. One specific thing I like is that the motivation for this rewrite is organic, i.e., not driven by an external pressure campaign. It's refreshing to see a drama-free rewrite.
• debugnik 2y ago
  > we also want to convert [the parser] to use recursive descent; it was originally written using parser combinators Almost all parser combinators are recursive descent with backtracking, they just add higher-order plumbing. I have a feeling that whatever issue they've encountered with the combinator approach could have been worked around by handwriting only a few pieces.
  • wirrbel 2y ago
    When I used parser combinations in rust years ago the compile times were really long. In also think it’s a strange reason to move away from rust as a language.
  • bhansconnect 2y ago
    This is more about simplicity, maintainability, and possiblity for new contributors to easily jump in and fix things. Our current parser is not fun for new contributors to learn. Also, I think parser combinators obsficate a lot of the tracking required for robust error messages. Personally, I find parser combinators nice for small things but painful for large and robust things.
• flooow 2y ago
  From listening to Feldman's podcast, this doesn't really come as a surprise to me. The rigor that Rust demands seems not to jibe with his 'worse is better' approach. That coupled with the fact they already switched the stdlib from Rust to Zig. The real question I have is why he chose Rust in the first place.
  • bhansconnect 2y ago
    Zig was not ready or nearly as popular back in 2019 when the compiler was started. Not to mention, Richard has a background mostly doing higher level programming. So jumping all the way to something like C or Zig would have been a very big step. Sometimes you need a stepping stone to learn and figure out what you really want.
  • christophilus 2y ago
    It’s covered in TFA, but the tldr is they started when Zig was immature.
  • norman784 2y ago
    > The real question I have is why he chose Rust in the first place. If you read the linked post carefully you will know. > Compile times aside, the strengths and weaknesses of Rust and Zig today are much different than they were when I wrote the first line of code in Roc's Rust compiler in 2019. Back then, Rust was relatively mature and Zig was far from where it is today.
• zoogeny 2y ago
  It's nice to see Zig continuing to gain support. I have no idea why I've ended up siding with Zig in the lower languages wars. I used to root for the D programming language but it seems to have got stuck and never gained a good ecosystem. I've disliked Rust from the first time I saw it and have never warmed up to it's particular trade offs. C feels unergonomic these days and C++ is overfull with complexity. Zig feels like a nice pragmatic middle ground. I actually think Rust is probably perfectly suited to a number of tasks, but I feel I would default to choosing Zig unless I was certain beyond doubt that I needed specific Rust safety features.
  • bornfreddy 2y ago
    I wanted to like Rust (memory safety and performance - what's not to like?) but both of my experiments with it ended in frustration. It seemed a way too complex language for what I needed. Recently, a coworker of mine made a great observation that made everything clear to me. I was looking for a replacement for C, but Rust is actually a replacement for C++. Totally different beast - powerful but complex. I need to see if Zig is any closer to C in spirit.
    • fc417fc802 2y ago
      Somewhat related, even when I work with C++ I use it as "C with RAII". What I actually want is a scheme (R5RS) with manual memory management and a borrow checker. I don't know how well such a monstrosity would actually work in practice, but it's what I've convinced myself that I want.
      • valorzard 2y ago
        https://github.com/carp-lang/Carp https://github.com/carp-lang/Carp
      • throwawaymaths 2y ago
        posted yesterday: https://news.ycombinator.com/item?id=42923829 https://news.ycombinator.com/item?id=42923829
    • 2y ago
• gregwebs 2y ago
  Go was built while waiting for C++ to compile- fast compilation was an implicit design goal. Rust on the other hand didn’t prioritize compile times and ended up making design decisions that make faster compilation difficult to achieve. To me it’s the biggest pain point with Rust for a large code base and that seems to be the sentiment here as well.
  • pcwalton 2y ago
    Rust always cared about compile times. Much of the motivation to switch from rustboot to rustc was compile times. (Source: I wrote much of the Rust compiler.)
    • deleted 2y ago
      [deleted]
    • eximius 2y ago
      But were any _language_ decisions discarded due to compile time concerns? I don't think anyone would claim the folks working on the rust compiler don't care. On that note, thank you for your part! I sure enjoy your work! :)
      • LegionMammal978 2y ago
        Since at least 2018, there's been a planned upgrade to the borrow checker to permit certain patterns that it currently rejects [0]. Also since 2018, there's been an implementation of the new version (called Polonius) that can be enabled with an unstable compiler flag [1]. But now it's 2025, and the new version still hasn't been enabled on stable Rust, since the Polonius implementation is seen as far too slow for larger programs. (I'm not exactly sure how horrible it really is, I haven't looked at the numbers.) A big goal of the types team since last year has been to reimplement a faster version within rustc [2]. I'd count this as a language feature (albeit a relatively minor one) that's been greatly deferred in favor of shorter compile times. [0] https://blog.rust-lang.org/inside-rust/2023/10/06/polonius-update.html https://blog.rust-lang.org/inside-rust/2023/10/06/polonius-u... [1] https://github.com/rust-lang/rust/pull/51133 https://github.com/rust-lang/rust/pull/51133 [2] https://rust-lang.github.io/rust-project-goals/2024h2/Polonius.html https://rust-lang.github.io/rust-project-goals/2024h2/Poloni...
• betimsl 2y ago
  I think Go would have been a better choice.
  • anacrolix 2y ago
    absolutely not. both because it's awful for implementing parsers, but also it introduces lots of extra hoops for interop. its build system is also terrible for targeting lots of varying platforms and feature sets. it has a custom IR that isn't helpful interacting with other systems. its runtime is really nice but it's a black box and not suitable for interop.
• wiz21c 2y ago
  I use rust a lot but, Roc is right: compilation times are not proportional to the safety you gain. At least in my project.
  • Gibbon1 2y ago
    I had an issue where the corporate spyware thought my cross compiler was potential malware and running scans on every file access. My compile times went from a few seconds to 2 minutes. And my codebase has a dozen targets. Where building, fixing, running tests, checking in would take half and hour it now took the whole afternoon. I think I'd rather stick with C and run static analysis tools on my code when generating release candidates than have to deal with that.
    • airstrike 2y ago
      That's hardly the language's fault
• stevage 2y ago
  If author is reading this, maybe worth a paragraph saying why you're not rewriting in Roc.
  • anacrolix 2y ago
    because that's hard. also it's one choice I respect them for not making: don't self host until you have actual users and momentum (unless you want to to prove a point). if Roc intends to have industry usage, keeping it in another language for now is a good move.
    • bhansconnect 2y ago
      Roc never plans to self host. We want roc the compiler in the long term to be as nice as possible for end users. A big part of that is as fast as possible. While roc can be fast, it can't compete with all the techniques that can be done in rust/zig/c/c++. It fundamentally is a higher level language to increase developer joy and reduce bugs. So it isn't the right tool when you are trying to reach the limits of hardware like this.
  • teo_zero 2y ago
    Like this one? > Roc's compiler has always been written in Rust, and we do not plan to self-host. We have a FAQ entry explaining why, and none of our reasoning has changed on that subject.
  • bhansconnect 2y ago
    https://www.roc-lang.org/faq.html#self-hosted-compiler https://www.roc-lang.org/faq.html#self-hosted-compiler
• mr_00ff00 2y ago
  I remember reading the HN thread when Fish rewrote to Rust. A big reason because “they wanted to stay current” so they rewrote from C++ to Rust. Someone made a joke comment how in a few years there would be tons of HN threads about rewriting Rust to Zig.
• sweeter 2y ago
  Yay for Zig! I'm a big Zig fan. It's seemless interop with C makes it super fun. I really enjoy writing code in a C-style way. Its still got some rough edges, but its definitely super fun and enjoyable to write imo.
• marxisttemp 2y ago
  I’ve been reading a lot about comptime in Zig and it’s really cool, it unifies and simplifies a lot of metaprogramming concepts that normally end up being the ugliest and clunkiest parts of most languages. I’ve been immersed in Swift for a couple years after working as a Go programmer, and I find myself pining for a language that’s more expressive than the latter and more concise than the former.
• nialv7 2y ago
  Whenever I see <language A> rewrites its compiler in <language B>, and A != B, I lose a bit of interest in language A.
  • Hasnep 2y ago
    Why? Roc has good reasons for not self hosting the compiler.
    • nialv7 2y ago
      Well their main reason is a compiler written in Roc won't be performant. (And their tag line is "Roc: A _fast_, friendly, functional language.") So yeah, I am not interested. Plus I feel in general not dogfooding your own language is a bad sign.
      • Hasnep 2y ago
        That excludes all languages that want to write the compiler in a lower level language than the target language itself. You can easily imagine a high level language with a fast compiler written in C where if you rewrote the compiler to be self hosted the performance would drop 10-100x. Dog-fooding is important, but it can be done outside of compiler development. If the only code the compiler team writes is the compiler, they end up designing a language that's great for implementing compilers at the expense of other use cases.
• petabyt 2y ago
  Fast compile times in Zig? Last time I tried it was super slow. Has anything changed in recent years?
  • pverghese 2y ago
    Compared to Rust its blazingly fast. Compared to Go compilation its slower.
    • metaltyphoon 2y ago
      Non it’s not. I known its not a benchmark but just try a zig init and build their hello world. It takes significantly longer
      • sweeter 2y ago
        ```sh sweet@nadeko ~ $ mkcd test sweet@nadeko ~/test $ zig init info: created build.zig info: created build.zig.zon info: created src/main.zig info: created src/root.zig info: see `zig build --help` for a menu of options sweet@nadeko ~/test $ time zig build zig build 5.08s user 0.58s system 119% cpu 4.745 total # cached sweet@nadeko ~/test $ time zig build zig build 0.01s user 0.03s system 132% cpu 0.026 total # after rewriting the main function to call a function that takes a pointer to another function sweet@nadeko ~/test $ time zig build zig build 0.02s user 0.03s system 136% cpu 0.032 total ```
      • baazaa 2y ago
        think this has something to do with zig building part of the std which other languages ship as binaries. incremental compilation will remove this small overhead.
      • jedisct1 2y ago
        The first time you run the toolchain for a given target, it compiles stuff that is then stored in a global cache.
  • rererereferred 2y ago
    Their x86 backend seems to be more complete, bypassing LLVM and allowing for fast debug builds.
• kreelman 2y ago
  The ROC webpage talks about ROC doing web servers via Rust libs... Behind the scenes, it uses Rust's high-performance hyper and tokio libraries to execute your Roc function on incoming requests. Will these be replaced with Zig networking libs?
  • isaacvando 2y ago
    Not necessarily! Each Roc app runs on a particular platform which is built in a host language for a specific domain. That host language could be Rust, Zig, C, C++, Go, Swift, etc. It's possible the basic-webserver platform will be rewritten to Zig but it doesn't need to be.
• einpoklum 2y ago
  It is interesting to note an intentional choice to go from a "safe" language to an "unsafe" language for a program whose use is not without security implications (even if it's not network-facing).
  • sionisrecur 2y ago
    What matters is that the compiled program is safe. Who cares if the compiler leaks memory? It's only going to run for a bit and then shut down, the OS will take care of it.
• 3836293648 2y ago
  Maybe I need to give it another shot, but Zig was a terribly unpleasant language last I tried to use it. Their motivation seems to justify the decision, but I'm pretty sure this will scare away potential contributors. I've been following Roc personally, and probably will continue to do so, but this definitely has killed a lot of interest for me.
• 5- 2y ago
  > Zig came up with a way around this: compile directly to LLVM bitcode (LLVM has strong backwards-compatibility on its bitcode but not on its public-facing API) and then upgrades become trivial because we can keep our existing code generation the same. i'm glad that zig helps offset what i find a rather worrying trend -- replacing protocols with libraries (and as a consequence designing new underlying protocols in view of their only being used through the vendor library). protocols are traditionally designed to facilitate an independent implementation; in fact, many standards ratification processes require several independent implementations as a prerequisite. libraries (or worse, frameworks) intermingle the actual api with their own design, and necessitate a single implementation. just the other day i wanted to display a notification a linux desktop where it wasn't convenient to depend on a library (it was a game, so not a traditional application expected to have many dependencies). the protocol (there is one, wrapped by the library) is very unpleasant, but i got it working out of spite. and of course, when there is a perfectly nice protocol available (llvm ir, in either the bitcode or text representation) why not choose it? at least on unix, where starting processes and interprocess communication is cheap. (and as an added bonus, you won't crash when llvm does.)
  • steveklabnik 2y ago
    LLVM IR is not stable, which is a prerequisite for a good protocol.
• poulpy123 2y ago
  I'm not educated enough to give an opinion but I find it weird to rewrite in a language that is so new
  • lioeters 2y ago
    Not only is it new, it has yet to reach version 1 as a language with any guaranteed level of stability. In every version release there's a disclaimer that seems to discourage its use in production software. > Zig has known bugs and even some miscompilations. > Zig is immature. Even with Zig 0.13.0, working on a non-trivial project using Zig will likely require participating in the development process. But then again, it's a great thing for the language to be used in larger public projects (like tigerbeetle, bun, ghostty). This will drive its development and adoption.
• ksec 2y ago
  I wonder if it would make sense for Ruby YJIT as well. Which is also written in Rust and started around the same time as Roc in 2019.