6 ms·

A significant part of the port was also for the built-in safety. Unsafe code just... fails to compile. Turns out there's a journey here.

Philip-J-Fry 3mo ago
The port was line for line and full of unsafe code. It didn't prevent any bugs. They're not using Rust for its strengths.
- skeledrew 3mo ago
  You should give the article[0] a read. "A large percentage of bugs from that list are use-after-free, double-free, and "forgot to free" in an error path. In safe Rust, these are compiler errors and RAII-like automatic cleanup with Drop. Compiler errors are a better feedback loop than a style guide." "At the time of writing, about 4% of Bun's Rust code sits inside an unsafe block (~13,000 unsafe keywords across ~27,000 lines / ~780,000 lines), and 78% of those blocks are a single line — a pointer that came from C++, or one call into a C library. I expect this number to go down over time as we refactor from a faithful Zig port (which had no greppable unsafe keyword) to idiomatic Rust, but we are going to continue using C & C++ libraries like JavaScriptCore so it will always have more unsafe than pure Rust projects." [0] https://bun.com/blog/bun-in-rust https://bun.com/blog/bun-in-rust
  - bunderbunder 3mo ago
    That one’s interesting to me because it speaks to both sides of the debate. Yes, safe Rust is good for catching those kinds of errors at compile time. But also, it should theoretically be very easy for a compiler to avoid those problems in Zig, too, if you are using the language the way it wants to be used. Which, from what I’ve experienced so far, does seem to take a whole lot more effort if you’re using a coding agent. I spend an incredible amount of time making sure mine doesn’t bloat our codebase with Java-flavored Python, and all the noise and defects and performance problems that it brings.
    - skeledrew 3mo ago
      > using the language the way it wants to be used. This sounds like just the coding conventions dependency they're trying to avoid.
      - bunderbunder 3mo ago
        Yup. But that also gets to the “they’re both right” angle. Zig’s convention may not be how the Bun developers wanted to work, but it’s a good choice for the kinds of projects Zig is designed for. Which overlaps a bit with, but isn't the same as, Rust’s intended use cases. For example, I might rather go with the Zig philosophy if I’ve got hard memory constraints and want to avoid dynamic memory allocation. Which, ironically, is something I’d love for my browser’s JavaScript runtime to do. But I can also respect others not wanting that.
      - TheGoddessInari 3mo ago
        IMO it's the same problem with Rust. LLMs are trained on vast quantities of code that violate Rust soundness in safe code. LLMs are prediction engines: you can move the needle (heavily) on the predictions from lazy to correct by construction, but the defaults even on high end models like Opus are always riddled with shortcuts. LLMs can produce coherent & safe Rust 1.92, LLMs can produce coherent Zig 0.16, but that's dependent on in context learning & instruction following.
        maleldil 3mo ago
        > LLMs are trained on vast quantities of code that violate Rust soundness in safe code. Why is your reasoning behind this? Most Rust code in the wild does not use unsafe. Most projects don't have a single line of unsafe.
        skeledrew 3mo ago
        > that's dependent on in context learning & instruction following Not just that. For Rust in particular, since unsafe code is uncompilable code (unless explicitly marked "unsafe"), the LLM is essentially forced to continue iterating even if the code already works but is unsafe, until it finds an implementation that's both correct (per the spec) and safe (per the compiler). That's the difference from Zig, where "make the code safe" is essentially part of the spec, and so may be missed by the LLM and not be discovered by a human until there's an observable runtime issue, or CVE.
  - sfink 3mo ago
    The way I'm thinking of it is that they ported Bun from Zig to Unsafe Rust, and are still working on now porting it from Unsafe Rust to Rust (not 100% safe, but a reasonable amount).
  - Dylan16807 3mo ago
    I'm optimistic for that refactoring. But if one in 30 lines sprinkled all over everywhere is unsafe then you haven't actually done much to improve safety. When you have a large percent of fully safe functions and modules, that's when you're making a difference. Counting the lines is not a very helpful metric.
    - deleted 3mo ago
      [deleted]
    - skeledrew 3mo ago
      > one in 30 lines > Counting the lines is not a very helpful metric. Seems self-contradictory. And they did say they're working to reduce the unsafe code over time. That huge percentage of memory bugs that they were able to better triage and fix due to the port was also them improving safety.
      - Dylan16807 3mo ago
        The other part of that line is important. Measuring lines is better than nothing but the distribution is much more important. If they're scattered all over that's a bigger downside than the line reduction is an upside. If you want good metrics, measure functions and data structures without unsafe. Not lines.
        skeledrew 3mo ago
        > If you want good metrics, measure functions and data structures without unsafe. I doubt it, as it's possible a substantial amount of those unsafe areas can't even be removed, given the project's dependent on C/C++ libraries. Without proper calibration those metrics won't tell the full story, and with calibration the incentive for the story will decrease over time with the unsafe refactoring. I doubt the distribution of the unsafe code really matters that much; what matters is proper handling, wherever it is, so stability and security isn't compromised.
- samwillis 3mo ago
  They have made it very clear that the mechanical translation is the starting point, and are refactoring to remove as many unsafe declarations as possible.
- neuronexmachina 3mo ago
  That may have been the case for the initial iteration a few months ago, but is it still true?
- insanitybit 3mo ago
  They turned every hidden memory safety bug into a grep'able memory safety bug. That's step 1 and a massive win.
- atombender 3mo ago
  The original Zig code is metaphorically one big unsafe block. Even if the Rust port is made up of 3% unsafe blocks, that still means 97% of the original Zig code has been made safe (in the Rust sense).
  - audunw 3mo ago
    Since they did a one-to-one translation to start with, hypothetically, if these were the true numbers, you could probably get a static analysis tool to guarantee you that 97% of the zig code was safe and tell you which of the 3% were unsafe. Zig has a lot more syntax and conventions to encourage safe code compare to C after all. It just doesn’t have way to mark which parts are safe or unsafe. An LLM could probably also trivially give you accurate reviews saying which parts were unsafe and in need of tests or reviews. I mean, considering their LLM budgets they could probably have had nightly reviews running every night for years before spending more than their Rust rewrite. Not that I think Rust rewrite was a bad idea. Rust is a good fit for this kind of project. I say that as a Zig enthusiast. It’s just that their stated motivation and reported results are kinda BS. If they just wrote “we just like Rust and thought it’d be cool to see of LLM could do the whole rewrite”, and left it at that, I think it’d be a more honest description of the motivation. The rest is just rationalisation.
    - Zakis1 3mo ago
      No you cannot get a static analysis tool to guarantee 97% of zig code was safe. Well, you can - the tool is called the borrow checker. Do you know what's infinitely cheaper and faster than an LLM running nightly reviews of all of the code and infact mathematically provable that the code (and any new code) is memory safe? It's called the borrow checker.
      - dnautics 3mo ago
        it seems like a reasonable enterprise to intercept the data dependency graph from the zig compiler and subject it to analysis to prove memory safety parameters: (my wip experiment: ) https://github.com/ityonemo/clr https://github.com/ityonemo/clr
GuB-42 3mo ago
GP is talking about the fork of the Zig compiler, not the rewrite of Bun into Rust. Anthropic submitted to patch the Zig compiler to improve Bun compilation times, it was rejected. One reason it was rejected is that it contained AI-generated code, which is against the community guidelines, but more importantly, it would have been rejected regardless of AI use. They claimed a 4x improvement, something like going from 2 minutes to 30 seconds, but, according to the Zig team, it made compilation non-deterministic, and missed on the real improvement they worked on, which was sub-1s incremental builds. "Buz" shows that the fork submitted by Anthropic wasn't needed as Bun can now compile under 1s (incrementally), which is much better than the 30s they achieved with their patch. It is likely that this rejection, especially the "no AI" aspect of it encouraged Anthropic to move away from Zig.
- dminik 3mo ago
  Is there anything suggesting that Bun even attempted to submit their Zig patches? As far as I know, they were stuck 2 versions behind. With the amount of changes and Zig already working on incremental I'm not sure what the point would have been.
  - kristoff_it 3mo ago
    Yes it was never attempted, the whole communication on social media about the AI policy was completely detached from the reality of the engineering work being done on either side.
- dtj1123 3mo ago
  Yep, on rereading my comment I totally failed to make this clear. Cheers.

Philip-J-Fry 3mo ago

The port was line for line and full of unsafe code. It didn't prevent any bugs. They're not using Rust for its strengths.

skeledrew 3mo ago
You should give the article[0] a read. "A large percentage of bugs from that list are use-after-free, double-free, and "forgot to free" in an error path. In safe Rust, these are compiler errors and RAII-like automatic cleanup with Drop. Compiler errors are a better feedback loop than a style guide." "At the time of writing, about 4% of Bun's Rust code sits inside an unsafe block (~13,000 unsafe keywords across ~27,000 lines / ~780,000 lines), and 78% of those blocks are a single line — a pointer that came from C++, or one call into a C library. I expect this number to go down over time as we refactor from a faithful Zig port (which had no greppable unsafe keyword) to idiomatic Rust, but we are going to continue using C & C++ libraries like JavaScriptCore so it will always have more unsafe than pure Rust projects." [0] https://bun.com/blog/bun-in-rust https://bun.com/blog/bun-in-rust
- bunderbunder 3mo ago
  That one’s interesting to me because it speaks to both sides of the debate. Yes, safe Rust is good for catching those kinds of errors at compile time. But also, it should theoretically be very easy for a compiler to avoid those problems in Zig, too, if you are using the language the way it wants to be used. Which, from what I’ve experienced so far, does seem to take a whole lot more effort if you’re using a coding agent. I spend an incredible amount of time making sure mine doesn’t bloat our codebase with Java-flavored Python, and all the noise and defects and performance problems that it brings.
  - skeledrew 3mo ago
    > using the language the way it wants to be used. This sounds like just the coding conventions dependency they're trying to avoid.
    - bunderbunder 3mo ago
      Yup. But that also gets to the “they’re both right” angle. Zig’s convention may not be how the Bun developers wanted to work, but it’s a good choice for the kinds of projects Zig is designed for. Which overlaps a bit with, but isn't the same as, Rust’s intended use cases. For example, I might rather go with the Zig philosophy if I’ve got hard memory constraints and want to avoid dynamic memory allocation. Which, ironically, is something I’d love for my browser’s JavaScript runtime to do. But I can also respect others not wanting that.
    - TheGoddessInari 3mo ago
      IMO it's the same problem with Rust. LLMs are trained on vast quantities of code that violate Rust soundness in safe code. LLMs are prediction engines: you can move the needle (heavily) on the predictions from lazy to correct by construction, but the defaults even on high end models like Opus are always riddled with shortcuts. LLMs can produce coherent & safe Rust 1.92, LLMs can produce coherent Zig 0.16, but that's dependent on in context learning & instruction following.
      - maleldil 3mo ago
        > LLMs are trained on vast quantities of code that violate Rust soundness in safe code. Why is your reasoning behind this? Most Rust code in the wild does not use unsafe. Most projects don't have a single line of unsafe.
      - skeledrew 3mo ago
        > that's dependent on in context learning & instruction following Not just that. For Rust in particular, since unsafe code is uncompilable code (unless explicitly marked "unsafe"), the LLM is essentially forced to continue iterating even if the code already works but is unsafe, until it finds an implementation that's both correct (per the spec) and safe (per the compiler). That's the difference from Zig, where "make the code safe" is essentially part of the spec, and so may be missed by the LLM and not be discovered by a human until there's an observable runtime issue, or CVE.
- sfink 3mo ago
  The way I'm thinking of it is that they ported Bun from Zig to Unsafe Rust, and are still working on now porting it from Unsafe Rust to Rust (not 100% safe, but a reasonable amount).
- Dylan16807 3mo ago
  I'm optimistic for that refactoring. But if one in 30 lines sprinkled all over everywhere is unsafe then you haven't actually done much to improve safety. When you have a large percent of fully safe functions and modules, that's when you're making a difference. Counting the lines is not a very helpful metric.
  - deleted 3mo ago
    [deleted]
  - skeledrew 3mo ago
    > one in 30 lines > Counting the lines is not a very helpful metric. Seems self-contradictory. And they did say they're working to reduce the unsafe code over time. That huge percentage of memory bugs that they were able to better triage and fix due to the port was also them improving safety.
    - Dylan16807 3mo ago
      The other part of that line is important. Measuring lines is better than nothing but the distribution is much more important. If they're scattered all over that's a bigger downside than the line reduction is an upside. If you want good metrics, measure functions and data structures without unsafe. Not lines.
      - skeledrew 3mo ago
        > If you want good metrics, measure functions and data structures without unsafe. I doubt it, as it's possible a substantial amount of those unsafe areas can't even be removed, given the project's dependent on C/C++ libraries. Without proper calibration those metrics won't tell the full story, and with calibration the incentive for the story will decrease over time with the unsafe refactoring. I doubt the distribution of the unsafe code really matters that much; what matters is proper handling, wherever it is, so stability and security isn't compromised.
samwillis 3mo ago
They have made it very clear that the mechanical translation is the starting point, and are refactoring to remove as many unsafe declarations as possible.
neuronexmachina 3mo ago
That may have been the case for the initial iteration a few months ago, but is it still true?
insanitybit 3mo ago
They turned every hidden memory safety bug into a grep'able memory safety bug. That's step 1 and a massive win.
atombender 3mo ago
The original Zig code is metaphorically one big unsafe block. Even if the Rust port is made up of 3% unsafe blocks, that still means 97% of the original Zig code has been made safe (in the Rust sense).
- audunw 3mo ago
  Since they did a one-to-one translation to start with, hypothetically, if these were the true numbers, you could probably get a static analysis tool to guarantee you that 97% of the zig code was safe and tell you which of the 3% were unsafe. Zig has a lot more syntax and conventions to encourage safe code compare to C after all. It just doesn’t have way to mark which parts are safe or unsafe. An LLM could probably also trivially give you accurate reviews saying which parts were unsafe and in need of tests or reviews. I mean, considering their LLM budgets they could probably have had nightly reviews running every night for years before spending more than their Rust rewrite. Not that I think Rust rewrite was a bad idea. Rust is a good fit for this kind of project. I say that as a Zig enthusiast. It’s just that their stated motivation and reported results are kinda BS. If they just wrote “we just like Rust and thought it’d be cool to see of LLM could do the whole rewrite”, and left it at that, I think it’d be a more honest description of the motivation. The rest is just rationalisation.
  - Zakis1 3mo ago
    No you cannot get a static analysis tool to guarantee 97% of zig code was safe. Well, you can - the tool is called the borrow checker. Do you know what's infinitely cheaper and faster than an LLM running nightly reviews of all of the code and infact mathematically provable that the code (and any new code) is memory safe? It's called the borrow checker.
    - dnautics 3mo ago
      it seems like a reasonable enterprise to intercept the data dependency graph from the zig compiler and subject it to analysis to prove memory safety parameters: (my wip experiment: ) https://github.com/ityonemo/clr https://github.com/ityonemo/clr

GuB-42 3mo ago

GP is talking about the fork of the Zig compiler, not the rewrite of Bun into Rust. Anthropic submitted to patch the Zig compiler to improve Bun compilation times, it was rejected. One reason it was rejected is that it contained AI-generated code, which is against the community guidelines, but more importantly, it would have been rejected regardless of AI use. They claimed a 4x improvement, something like going from 2 minutes to 30 seconds, but, according to the Zig team, it made compilation non-deterministic, and missed on the real improvement they worked on, which was sub-1s incremental builds. "Buz" shows that the fork submitted by Anthropic wasn't needed as Bun can now compile under 1s (incrementally), which is much better than the 30s they achieved with their patch. It is likely that this rejection, especially the "no AI" aspect of it encouraged Anthropic to move away from Zig.

dminik 3mo ago
Is there anything suggesting that Bun even attempted to submit their Zig patches? As far as I know, they were stuck 2 versions behind. With the amount of changes and Zig already working on incremental I'm not sure what the point would have been.
- kristoff_it 3mo ago
  Yes it was never attempted, the whole communication on social media about the AI policy was completely detached from the reality of the engineering work being done on either side.
dtj1123 3mo ago
Yep, on rereading my comment I totally failed to make this clear. Cheers.