7 ms·

We sped up bun by 100x

54 points by sdan 6mo ago

homarp 6mo ago
with a different git client https://news.ycombinator.com/item?id=47618895 https://news.ycombinator.com/item?id=47618895 to discuss the git implementation
deleted 6mo ago
[deleted]
Night_Thastus 6mo ago
Aside: That font is really hard on my eyes. Anyone else?
- Growtika 6mo ago
  The dark mode looks better. In the light mode they have to improve the contrast. The lines in the visualisations are almost invisible, also the data table on the left https://vers.sh/hdr_legacy/images/manager_vs_manager_of_managers_clean.svg https://vers.sh/hdr_legacy/images/manager_vs_manager_of_mana...
- quantummagic 6mo ago
  Yeah. I was going to suggest Firefox Reader Mode, but it fails miserably on this particular site for some reason. Ublock Origin's "disable remote fonts", still works great on this site though.
- cdrnsf 6mo ago
  Me as well. Calling it painful would be overstating it, but it's not pleasant.
- yevbar 6mo ago
  I cross-posted to my personal blog too https://yev.bar/ziggit https://yev.bar/ziggit
hvenev 6mo ago
This blog post calls libgit2 "git's C library" as if it is in any way related to git. I don't think it is.
mfitton 6mo ago
What it cost doesn't actually say what it cost. I wonder what models they used. Napkin math of Opus for everything (probably not true) with no caching suggests $67,000. Cool article though!
- BearOso 6mo ago
  They mention Anthropic, so I assumed something similar. At $5 per 5 million tokens, 13 billion would cost $65,000. However, the image in the article shows over 17 billion used, which is $85,000. That's an entry-level programmer's yearly salary. It doesn't quite pass their code tests, and it's automatic code translation, so it's going to be a pretty direct transcription. There's still probably a lot of messy code to clean up. I'm not sure it's worth it.
- plorkyeran 6mo ago
  Anthropic owns Bun, so they presumably did not directly pay for anything.
sc68cal 6mo ago
So, they implemented a git client in zig, that had some significant speedups for their usecase. However: > The git CLI test suite consists of 21,329 individual assertions for various git subcommands (that way we can be certain ziggit does suffice as a drop-in replacement for git). <snip> > While we only got through part of the overall test suite, that's still the equivalent of a month's worth of straight developer work (again, without sleep or eating factored in).
- varispeed 6mo ago
  Reminds me when I was happy with my algorithm being super fast until I started tackling edge cases. Suddenly it's got quite slow.
  - yevbar 6mo ago
    Edge cases certainly apply with scripts depending on specific git CLI args or stdout strings may not suffice with ziggit. _However_, for the use cases that most developers or agents are looking for, ziggit should have enough features covered. Happy to fix issues or bugs if that's not the case
    - ARandumGuy 6mo ago
      > _However_, for the use cases that most developers or agents are looking for What use cases are those? How did you determine that these are the use cases most developers/agents are looking for? For me, git has a ton of features that I rarely use. But when I need them, I really need them. Any replacement that doesn't cover these edge cases is fundamentally incomplete and insufficient, even if it works fine 99% of the time.
- rao-v 6mo ago
  Wait, so they don’t have test parity with git? How do they know that they, umm … did the actual thing they were trying to do?
  - didgetmaster 6mo ago
redoh 6mo ago
[flagged]
- dingdingdang 6mo ago
  But then there's this: "When evaluating the complete bun install improvements, it came out speed-wise to about the same as the existing git usage (due to networking being the big bottleneck time-wise despite more cases being slightly faster with ziggit over multiple benchmarks). Except, it's done in 100% zig and those internal improvements pile up as projects consist of more git dependencies. All in all, it seems like a sensible upstream contribution." Sooo, after burning these 10k+ worth of tokens we find out that it's sensible to use it because the language (zig) feels good as opposed to git itself which now has +20 years of human eval on it. That seems. Well. Yeah...
  - yevbar 6mo ago
    The original target was bun since it itself is written in zig, not because of anything specific to the language When it was clear that there were benefits in filling in more of git's capabilities (ie targeting WASM), I then went and filled in more git features. It's not by any means a universal win over everything but it does have notable wins like having git operations be between 4-10x faster on arm-based MacBooks than git itself
nightpool 6mo ago
Seems like they actually sped bun up ~1x: When evaluating the complete bun install improvements, it came out speed-wise to about the same as the existing git usage (due to networking being the big bottleneck time-wise despite more cases being slightly faster with ziggit over multiple benchmarks). Except, it's done in 100% zig and those internal improvements pile up as projects consist of more git dependencies. All in all, it seems like a sensible upstream contribution. So you have to maintain a completely separate git implementation and keep that up to date with upstream git, all for the benefit of being indistinguishable on benchmarks. Oh well!
- deleted 6mo ago
  [deleted]
- BearOso 6mo ago
  You have to maintain a completely separate implementation of AI generated code that's translated from C, so not even idiomatic zig. Edit And then I go their repository and read commits like this https://github.com/hdresearch/ziggit/commit/31adc1da1693e402d8e7e4132d586e78650d6885 https://github.com/hdresearch/ziggit/commit/31adc1da1693e402... which confirms it wasn't even looked over by a human.
  - yevbar 6mo ago
    This was orchestrated and developed by agents with verifications like the codebase compiling or git's CLI test suite passing. That was so the commit authors don't all appear like blank accounts on GitHub
    - nightpool 6mo ago
      Surely "the commits are attributed to the user who creates them" is a pretty basic feature of the git CLI, and not something that you can add in as a fix later after posting your project to Github and writing a blog post about how much faster than git it is. It's very easy to be faster than git's CLI if you don't have to do any of the things that git's CLI does!
  - 6mo ago
deleted 6mo ago
[deleted]
joaohaas 6mo ago
With the recent barrage of AI-slop 'speedup' posts, the first thing I always do to see if the post is worth a read is doing a Ctrl+F "benchmark" and seeing if the benchmark makes any fucking sense. 99% of the time (such as in this article), it doesn't. What do you mean 'cloneBare + findCommit + checkout: ~10x win'? Does that mean running those commands back to back result in a 10x win over the original? Does that mean that there's a specific function that calls these 3 operations, and that's the improvement of the overall function? What's the baseline we're talking about, and is it relevant at all? Those questions are partially answered on the much better benchmark page[1], but for some reason they're using the CLI instead of the gitlib for comparisons. [1] https://github.com/hdresearch/ziggit/blob/5d3deb361f03d4aefef29426cf333782fc05d7cf/BENCHMARKS.md#full-workflow https://github.com/hdresearch/ziggit/blob/5d3deb361f03d4aefe...
- yevbar 6mo ago
  The reason being bun actually tested both using the git CLI as well as libgit2. Across the board the C library was 3x slower than just spawning calls to the git CLI. Under the hood, bun's calling these operations when doing a `bun install` and these are the places where integrating 100% gives the most boost. When more and more git deps are included in a project, these gains pile up. However, the results appear more at 1x parity when accounting for network times (ie round trip to GitHub)
- hrmtst93837 6mo ago
  [flagged]
butz 6mo ago
How does bun compare with upcoming Vite+?
- manniL 6mo ago
  Upcoming?
  - butz 6mo ago
    It is still in alpha.
jedisct1 6mo ago
Zig is a well-kept secret for writing highly efficient WebAssembly modules.
moralestapia 6mo ago
>it becomes possible to see upward of 100x speedups for some git operations. They really stretch the limits of an honest title there.
flykespice 6mo ago
AI slop with your usual hallucinated unrealistic speedups claims yawn immediate skip
cwillu 6mo ago
I think we might be getting to the point where submissions for projects that are primarily written by ai and/or ai agents need to be tagged with [agent] in the title
- yevbar 6mo ago
  If this were 2+ years ago perhaps, with industry adopting more agents in their SDLCs (ie Stripe minions or Ramp background agents), I think we're more a matter of time before we treat agent/human built products the same unless we're branding smth as artisanal human-crafted software
  - mpalmer 6mo ago
    Well, we're not there today. Generated code and prose are still trivial to spot. And even mass-produced products benefit from thoughtful design by a human. Why didn't the blog post explain anything about why the rewrite is faster? Or about Zig, or C, programming, at all?
TimTheTinker 6mo ago
These "AI rewrite" projects are beginning to grate on me. Sure, if you have a complete test suite for a library or CLI tool, it is possible to prompt Claude Opus 4.6 such that it creates a 100% passing, "more performant", drop-in replacement. However, if the original package is in its training data, it's very likely to plagiarize the original source. Also, who actually wants to use or maintain a large project that no one understands and that doesn't have a documented history of thoughtful architectural decisions and the context behind them? No matter how tightly you structure AI work, probabilistic LLM logorrhea cannot reliably adopt or make high-level decisions/principles, apply them, or update them as new data arrives. If you think otherwise, you're believing an illusion - truly. A large software project's source code and documentation are the empirical ground-truth encoding of a ton of decisions made by many individuals and teams -- decisions that need to be remembered, understood, and reconsidered in light of new information. AI has no ability to consider these types of decisions and their accompanying context, whether they are past, present, or future -- and is not really able to coherently communicate them in a way that can be trusted to be accurate. That's why I can't and won't trust fully AI-written software beyond small one-off-type tools until AI gains two fundamentally new capabilities: (1) logical reasoning that can weigh tradeoffs and make accountable decisions in terms of ground-truth principles accurately applied to present circumstances, and (2) ability to update those ground-truth principles coherently and accurately based on new, experiential information -- this is real "learning"
- yevbar 6mo ago
  > Sure, if you have a complete test suite for a library or CLI tool, it is possible to prompt Claude Opus 4.6 such that it creates a 100% passing, "more performant", drop-in replacement. This was the "validation" used for determining how much progress was made at a given point in time. Re training data concerns, this was done and shipped to be open source (under GPLv2) so there's no abuse of open source work here imo Re the tradeoffs you highlight - these are absolutely true and fair. I don't expect or want anyone to just use ziggit because it's new. The places where there performance gains (ie internally with `bun install` or as a better WASM binary alternative) are places that I do have interest or use in myself _However_, if I could interest you in one thing. ziggit when compiled into a release build on my arm-based Mac, showed 4-10x faster performance than git's CLI for the core workflows I use in my git development
carterschonwald 6mo ago
im pretty stoked about the llm harness theyre using. cause I wrote all the code thats not monopi code in that fork! despite it’s paucity of features, the changes i landed in it from my design notes actually have been so smooth in terms of comparative ux/ llm behavior that its my daily driver since ive stood it up. Previously, since early december, ive had to run a patch script on every update of claude code to make it stop undermining me. I didnt need a hilarious code leak to find the problematic strings in the minified js ;) I regard punkin-pi as a first stab at translating ideas ive had over the past 6 months for reliable llm harnesses. I hit some walls in terms of mono pi architecture for doing much more improvement with mono pi. so Im working on the next gen of agent harnesses! stay tuned!
SeriousM 6mo ago
[dead]
CodeCompost 6mo ago
Is this more vibe-coded garbage?
mpalmer 6mo ago
The title is obviously dishonest. I do not hesitate to call it a lie. The post is also not about the speed increase, it's about how proud this team is of their agent orchestration scheme. As I understand it, there is really no speed difference at all between Zig and C, just some cognitive overhead associated with doing things "right" in C. It's all machine code at bottom. So why is this rewrite faster? Why did the authors choose Zig? How has the logic or memory management changed? The authors give us absolutely no insight whatsoever into the the Zig code. No indication that they know anything about Zig, or systems programming, at all. I wish this was an exaggeration. And really. With all this agentic power at your fingertips, why wouldn't they just contribute these improvements to git itself? I can think of at least one reason, that they don't want their changes to be rejected as unhelpful or low-quality.
4b11b4 6mo ago
So uh.. about that sentence where it says you only pass part of the test suite...
queenkjuul 6mo ago
Long-ass LLM write-up that repeats itself several times, i couldn't finish reading. Big if true, etc

54 points by sdan 6mo ago

with a different git client https://news.ycombinator.com/item?id=47618895 https://news.ycombinator.com/item?id=47618895 to discuss the git implementation

deleted 6mo ago

[deleted]

Night_Thastus 6mo ago

Aside: That font is really hard on my eyes. Anyone else?

Growtika 6mo ago
The dark mode looks better. In the light mode they have to improve the contrast. The lines in the visualisations are almost invisible, also the data table on the left https://vers.sh/hdr_legacy/images/manager_vs_manager_of_managers_clean.svg https://vers.sh/hdr_legacy/images/manager_vs_manager_of_mana...
quantummagic 6mo ago
Yeah. I was going to suggest Firefox Reader Mode, but it fails miserably on this particular site for some reason. Ublock Origin's "disable remote fonts", still works great on this site though.
cdrnsf 6mo ago
Me as well. Calling it painful would be overstating it, but it's not pleasant.
yevbar 6mo ago
I cross-posted to my personal blog too https://yev.bar/ziggit https://yev.bar/ziggit

hvenev 6mo ago

This blog post calls libgit2 "git's C library" as if it is in any way related to git. I don't think it is.

mfitton 6mo ago

What it cost doesn't actually say what it cost. I wonder what models they used. Napkin math of Opus for everything (probably not true) with no caching suggests $67,000. Cool article though!

BearOso 6mo ago
They mention Anthropic, so I assumed something similar. At $5 per 5 million tokens, 13 billion would cost $65,000. However, the image in the article shows over 17 billion used, which is $85,000. That's an entry-level programmer's yearly salary. It doesn't quite pass their code tests, and it's automatic code translation, so it's going to be a pretty direct transcription. There's still probably a lot of messy code to clean up. I'm not sure it's worth it.
plorkyeran 6mo ago
Anthropic owns Bun, so they presumably did not directly pay for anything.

sc68cal 6mo ago

So, they implemented a git client in zig, that had some significant speedups for their usecase. However: > The git CLI test suite consists of 21,329 individual assertions for various git subcommands (that way we can be certain ziggit does suffice as a drop-in replacement for git). <snip> > While we only got through part of the overall test suite, that's still the equivalent of a month's worth of straight developer work (again, without sleep or eating factored in).

varispeed 6mo ago
Reminds me when I was happy with my algorithm being super fast until I started tackling edge cases. Suddenly it's got quite slow.
- yevbar 6mo ago
  Edge cases certainly apply with scripts depending on specific git CLI args or stdout strings may not suffice with ziggit. _However_, for the use cases that most developers or agents are looking for, ziggit should have enough features covered. Happy to fix issues or bugs if that's not the case
  - ARandumGuy 6mo ago
    > _However_, for the use cases that most developers or agents are looking for What use cases are those? How did you determine that these are the use cases most developers/agents are looking for? For me, git has a ton of features that I rarely use. But when I need them, I really need them. Any replacement that doesn't cover these edge cases is fundamentally incomplete and insufficient, even if it works fine 99% of the time.
rao-v 6mo ago
Wait, so they don’t have test parity with git? How do they know that they, umm … did the actual thing they were trying to do?
- didgetmaster 6mo ago

redoh 6mo ago

[flagged]

dingdingdang 6mo ago
But then there's this: "When evaluating the complete bun install improvements, it came out speed-wise to about the same as the existing git usage (due to networking being the big bottleneck time-wise despite more cases being slightly faster with ziggit over multiple benchmarks). Except, it's done in 100% zig and those internal improvements pile up as projects consist of more git dependencies. All in all, it seems like a sensible upstream contribution." Sooo, after burning these 10k+ worth of tokens we find out that it's sensible to use it because the language (zig) feels good as opposed to git itself which now has +20 years of human eval on it. That seems. Well. Yeah...
- yevbar 6mo ago
  The original target was bun since it itself is written in zig, not because of anything specific to the language When it was clear that there were benefits in filling in more of git's capabilities (ie targeting WASM), I then went and filled in more git features. It's not by any means a universal win over everything but it does have notable wins like having git operations be between 4-10x faster on arm-based MacBooks than git itself

nightpool 6mo ago

Seems like they actually sped bun up ~1x: When evaluating the complete bun install improvements, it came out speed-wise to about the same as the existing git usage (due to networking being the big bottleneck time-wise despite more cases being slightly faster with ziggit over multiple benchmarks). Except, it's done in 100% zig and those internal improvements pile up as projects consist of more git dependencies. All in all, it seems like a sensible upstream contribution. So you have to maintain a completely separate git implementation and keep that up to date with upstream git, all for the benefit of being indistinguishable on benchmarks. Oh well!

deleted 6mo ago
[deleted]
BearOso 6mo ago
You have to maintain a completely separate implementation of AI generated code that's translated from C, so not even idiomatic zig. Edit And then I go their repository and read commits like this https://github.com/hdresearch/ziggit/commit/31adc1da1693e402d8e7e4132d586e78650d6885 https://github.com/hdresearch/ziggit/commit/31adc1da1693e402... which confirms it wasn't even looked over by a human.
- yevbar 6mo ago
  This was orchestrated and developed by agents with verifications like the codebase compiling or git's CLI test suite passing. That was so the commit authors don't all appear like blank accounts on GitHub
  - nightpool 6mo ago
    Surely "the commits are attributed to the user who creates them" is a pretty basic feature of the git CLI, and not something that you can add in as a fix later after posting your project to Github and writing a blog post about how much faster than git it is. It's very easy to be faster than git's CLI if you don't have to do any of the things that git's CLI does!
- 6mo ago

deleted 6mo ago

[deleted]

joaohaas 6mo ago

With the recent barrage of AI-slop 'speedup' posts, the first thing I always do to see if the post is worth a read is doing a Ctrl+F "benchmark" and seeing if the benchmark makes any fucking sense. 99% of the time (such as in this article), it doesn't. What do you mean 'cloneBare + findCommit + checkout: ~10x win'? Does that mean running those commands back to back result in a 10x win over the original? Does that mean that there's a specific function that calls these 3 operations, and that's the improvement of the overall function? What's the baseline we're talking about, and is it relevant at all? Those questions are partially answered on the much better benchmark page[1], but for some reason they're using the CLI instead of the gitlib for comparisons. [1] https://github.com/hdresearch/ziggit/blob/5d3deb361f03d4aefef29426cf333782fc05d7cf/BENCHMARKS.md#full-workflow https://github.com/hdresearch/ziggit/blob/5d3deb361f03d4aefe...

yevbar 6mo ago
The reason being bun actually tested both using the git CLI as well as libgit2. Across the board the C library was 3x slower than just spawning calls to the git CLI. Under the hood, bun's calling these operations when doing a `bun install` and these are the places where integrating 100% gives the most boost. When more and more git deps are included in a project, these gains pile up. However, the results appear more at 1x parity when accounting for network times (ie round trip to GitHub)
hrmtst93837 6mo ago
[flagged]

butz 6mo ago

How does bun compare with upcoming Vite+?

manniL 6mo ago
Upcoming?
- butz 6mo ago
  It is still in alpha.

jedisct1 6mo ago

Zig is a well-kept secret for writing highly efficient WebAssembly modules.

moralestapia 6mo ago

>it becomes possible to see upward of 100x speedups for some git operations. They really stretch the limits of an honest title there.

flykespice 6mo ago

AI slop with your usual hallucinated unrealistic speedups claims yawn immediate skip

cwillu 6mo ago

I think we might be getting to the point where submissions for projects that are primarily written by ai and/or ai agents need to be tagged with [agent] in the title

yevbar 6mo ago
If this were 2+ years ago perhaps, with industry adopting more agents in their SDLCs (ie Stripe minions or Ramp background agents), I think we're more a matter of time before we treat agent/human built products the same unless we're branding smth as artisanal human-crafted software
- mpalmer 6mo ago
  Well, we're not there today. Generated code and prose are still trivial to spot. And even mass-produced products benefit from thoughtful design by a human. Why didn't the blog post explain anything about why the rewrite is faster? Or about Zig, or C, programming, at all?

TimTheTinker 6mo ago

These "AI rewrite" projects are beginning to grate on me. Sure, if you have a complete test suite for a library or CLI tool, it is possible to prompt Claude Opus 4.6 such that it creates a 100% passing, "more performant", drop-in replacement. However, if the original package is in its training data, it's very likely to plagiarize the original source. Also, who actually wants to use or maintain a large project that no one understands and that doesn't have a documented history of thoughtful architectural decisions and the context behind them? No matter how tightly you structure AI work, probabilistic LLM logorrhea cannot reliably adopt or make high-level decisions/principles, apply them, or update them as new data arrives. If you think otherwise, you're believing an illusion - truly. A large software project's source code and documentation are the empirical ground-truth encoding of a ton of decisions made by many individuals and teams -- decisions that need to be remembered, understood, and reconsidered in light of new information. AI has no ability to consider these types of decisions and their accompanying context, whether they are past, present, or future -- and is not really able to coherently communicate them in a way that can be trusted to be accurate. That's why I can't and won't trust fully AI-written software beyond small one-off-type tools until AI gains two fundamentally new capabilities: (1) logical reasoning that can weigh tradeoffs and make accountable decisions in terms of ground-truth principles accurately applied to present circumstances, and (2) ability to update those ground-truth principles coherently and accurately based on new, experiential information -- this is real "learning"

yevbar 6mo ago
> Sure, if you have a complete test suite for a library or CLI tool, it is possible to prompt Claude Opus 4.6 such that it creates a 100% passing, "more performant", drop-in replacement. This was the "validation" used for determining how much progress was made at a given point in time. Re training data concerns, this was done and shipped to be open source (under GPLv2) so there's no abuse of open source work here imo Re the tradeoffs you highlight - these are absolutely true and fair. I don't expect or want anyone to just use ziggit because it's new. The places where there performance gains (ie internally with `bun install` or as a better WASM binary alternative) are places that I do have interest or use in myself _However_, if I could interest you in one thing. ziggit when compiled into a release build on my arm-based Mac, showed 4-10x faster performance than git's CLI for the core workflows I use in my git development

carterschonwald 6mo ago

im pretty stoked about the llm harness theyre using. cause I wrote all the code thats not monopi code in that fork! despite it’s paucity of features, the changes i landed in it from my design notes actually have been so smooth in terms of comparative ux/ llm behavior that its my daily driver since ive stood it up. Previously, since early december, ive had to run a patch script on every update of claude code to make it stop undermining me. I didnt need a hilarious code leak to find the problematic strings in the minified js ;) I regard punkin-pi as a first stab at translating ideas ive had over the past 6 months for reliable llm harnesses. I hit some walls in terms of mono pi architecture for doing much more improvement with mono pi. so Im working on the next gen of agent harnesses! stay tuned!

SeriousM 6mo ago

[dead]

CodeCompost 6mo ago

Is this more vibe-coded garbage?

mpalmer 6mo ago

The title is obviously dishonest. I do not hesitate to call it a lie. The post is also not about the speed increase, it's about how proud this team is of their agent orchestration scheme. As I understand it, there is really no speed difference at all between Zig and C, just some cognitive overhead associated with doing things "right" in C. It's all machine code at bottom. So why is this rewrite faster? Why did the authors choose Zig? How has the logic or memory management changed? The authors give us absolutely no insight whatsoever into the the Zig code. No indication that they know anything about Zig, or systems programming, at all. I wish this was an exaggeration. And really. With all this agentic power at your fingertips, why wouldn't they just contribute these improvements to git itself? I can think of at least one reason, that they don't want their changes to be rejected as unhelpful or low-quality.

4b11b4 6mo ago

So uh.. about that sentence where it says you only pass part of the test suite...

queenkjuul 6mo ago

Long-ass LLM write-up that repeats itself several times, i couldn't finish reading. Big if true, etc