6 ms·
This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive: > This was a clean-room implementation (Claude
by NitpickLawyer 9mo ago
This is a much more reasonable take than the cursor-browser thing. A few things that make it pretty impressive:
> This was a clean-room implementation (Claude did not have internet access at any point during its development); it depends only on the Rust standard library. The 100,000-line compiler can build Linux 6.9 on x86, ARM, and RISC-V. It can also compile QEMU, FFmpeg, SQlite, postgres, redis
> I started by drafting what I wanted: a from-scratch optimizing compiler with no dependencies, GCC-compatible, able to compile the Linux kernel, and designed to support multiple backends. While I specified some aspects of the design (e.g., that it should have an SSA IR to enable multiple optimization passes) I did not go into any detail on how to do so.
> Previous Opus 4 models were barely capable of producing a functional compiler. Opus 4.5 was the first to cross a threshold that allowed it to produce a functional compiler which could pass large test suites, but it was still incapable of compiling any real large projects.
And the very open points about limitations (and hacks, as cc loves hacks):
> It lacks the 16-bit x86 compiler that is necessary to boot [...] Opus was unable to implement a 16-bit x86 code generator needed to boot into 16-bit real mode. While the compiler can output correct 16-bit x86 via the 66/67 opcode prefixes, the resulting compiled output is over 60kb, far exceeding the 32k code limit enforced by Linux. Instead, Claude simply cheats here and calls out to GCC for this phase
> It does not have its own assembler and linker;
> Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled.
Ending with a very down to earth take:
> The resulting compiler has nearly reached the limits of Opus’s abilities. I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality.
All in all, I'd say it's a cool little experiment, impressive even with the limitations, and a good test-case as the author says "The resulting compiler has nearly reached the limits of Opus’s abilities". Yeah, that's fair, but still highly imrpessive IMO.
- modeless 9mo agoThere seem to still be a lot of people who look at results like this and evaluate them purely based on the current state. I don't know how you can look at this and not realize that it represents a huge improvement over just a few months ago, there have been continuous improvements for many years now, and there is no reason to believe progress is stopping here. If you project out just one year, even assuming progress stops after that, the implications are staggering.
- nozzlegear 9mo agoEvery S-curve looks like an exponential until you hit the bend.
- NitpickLawyer 9mo agoWe've been hearing this for 3 years now. And especially 25 was full of "they've hit a wall, no more data, running out of data, plateau this, saturated that". And yet, here we are. Models keep on getting better, at more broad tasks, and more useful by the month.
- nozzlegear 9mo ago> We've been hearing this for 3 years now Not from me you haven't! > "they've hit a wall, no more data, running out of data, plateau this, saturated that" Everyone thought Moore's Law was infallible too, right until they hit that bend. What hubris to think these AI models are different! But you've probably been hearing that for 3 years too (though not from me). > Models keep on getting better, at more broad tasks, and more useful by the month. If you say so, I'll take your word for it.
- Cyphase 9mo ago25 is 2025.
- nozzlegear 9mo agoOh my bad, the way it was worded made me read it as the name of somebody's model or something.
- torginus 9mo agoExcept for Moore's law, everyone knew decades ahead of what the limits of Dennard scaling are (shrinking geometry through smaller optical feature sizes), and roughly when we would get to the limit. Since then, all improvements came at a tradeoff, and there was a definite flattening of progress.
- geraneum 9mo ago> This was a clean-room implementation This is really pushing it, considering it’s trained on… internet, with all available c compilers. The work is already impressive enough, no need for such misleading statements.
- deleted 9mo ago[deleted]
- antirez 9mo agoThe LLM does not contain a verbatim copy of whatever it saw during the pre-training stage, it may remember certain over-represented parts, otherwise it has a knowledge about a lot of things but such knowledge, while about a huge amount of topics, is similar to the way you could remember things you know very well. And, indeed, if you give it access to internet or the source code of GCC and other compilers, it will implement such a project N times faster.
- halxc 9mo agoWe all saw verbatim copies in the early LLMs. They "fixed" it by implementing filters that trigger rewrites on blatant copyright infringement. It is a research topic for heaven's sake: https://arxiv.org/abs/2504.16046 https://arxiv.org/abs/2504.16046
- RyanCavanaugh 9mo agoThe internet is hundreds of billions of terabytes; a frontier model is maybe half a terabyte. While they are certainly capable of doing some verbatim recitations, this isn't just a matter of teasing out the compressed C compiler written in Rust that's already on the internet (where?) and stored inside the model.
- philipportner 9mo agoThis seems related, it may not be a codebase but they are able to extract "near" verbatim books out of Claude Sonnet. https://arxiv.org/pdf/2601.02671 https://arxiv.org/pdf/2601.02671 > For Claude 3.7 Sonnet, we were able to extract four whole books near-verbatim, including two books under copyright in the U.S.: Harry Potter and the Sorcerer’s Stone and 1984 (Section 4).
- gmueckl 9mo agoThe result is hardly a clean room implementation. It was rather a brute force attempt to decompress fuzzily stored knowledge contained within the network and it required close steering (using a big suite of tests) to get a reasonable approximation to the desired output. The compression and storage happened during the LLM training. Prove this statement wrong.
- NitpickLawyer 9mo ago> Prove this statement wrong. If all it takes is "trained on the Internet" and "decompress stored knowledge", then surely gpt3, 3.5, 4, 4.1, 4o, o1, o3, o4, 5, 5.1, 5.x should have been able to do it, right? Claude 2, 3, 4, 4.1, 4.5? Surely.
- geraneum 9mo agoPerhaps 4.5 could also do it? We don’t know really until we try. I don’t trust the marketing material as much. The fact that the previous version (smaller versions) couldn’t or could do it does not really disprove that claim.
- gmueckl 9mo agoThis comparison is only meaningful with comparable numbers of parameters and context window tokens. And then it would mainly test the efficiency and accuracy of the information encoding. I would argue that this is the main improvement over all model generations.
- hn_acc1 9mo agoAre you really asking for "all the previous versions were implemented so poorly they couldn't even do this simple, basic LLM task"?
- Philpax 9mo agoPlease look at the source code and tell me how this is a "simple, basic LLM task".
- 9mo ago
- dyauspitr 9mo ago> Claude did not have internet access at any point during its development Why is this even desirable? I want my LLM to take into account everything there is out there and give me the best possible output.
- simonw 9mo agoIt's desirable if you're trying to build a C compiler as a demo of coding agent capabilities without all of the Hacker News commenters saying "yeah but it could just copy implementation details from the internet".
- panzi 9mo ago> clean-room implementation Except its trained on all source out there, so I assume on GCC and clang. I wonder how similar the code is to either.
- pertymcpert 9mo agoI'm familiar with both compilers. There's more similarity to LLVM, it even borrows some naming such as mem2reg (which doesn't really exist anymore) and GetElementPtr. But that's pretty much where things end. The rest of it is just common sense.
- kelnos 9mo agoHonestly I don't find it that impressive. I mean, it's objectively impressive that it can be done at all, but it's not impressive from the standpoint of doing stuff that nearly all real-world users will want it to do. The C specification and Linux kernel source code are undoubtedly in its training data, as are texts about compilers from a theoretical/educational perspective. Meanwhile, I'm certain most people will never need it to perform this task. I would be more interested in seeing if it could add support for a new instruction set to LLVM, for example. Or perhaps write a complier for a new language that someone just invented, after writing a first draft of a spec for it.
- simonw 9mo agoAre you a frequent user of coding agents? I ask because, as someone who uses these things every day, the idea that this kind of thing only works because of similar projects in the training data doesn't fit my mental model of how they work at all. I'm wondering if the "it's in the training data" theorists are coding agent practitioners, or if they're mainly people who don't use the tools.
- bdangubic 9mo agoI am all-daily user (multiple claude max accounts). this fits my mental model mostly but not model I had before but developed with daily use. my job revolves around two core things: 1. data analysis / visualization / … 2. “is this possible? can this even be done?” for #1 - I don’t do much anymore, for #2 I mostly do it still all “by hand” not for the lack of serious trying. so “it can do #1 1000x better than me cause it is generally solved problem(s) it is trained on while it can’t effectively do #2 otherwise” fits perfectly
- steveklabnik 9mo ago> Or perhaps write a complier for a new language that someone just invented, after writing a first draft of a spec for it. Hello, this is what I did over my Christmas break. I've been taking some time to do other things, but plan on returning to it. But this absolutely works. Claude has written far more programs in my language than I have. https://rue-lang.dev/ https://rue-lang.dev/ if you want to check it out. Spec and code are both linked there.
- chamomeal 9mo agoWhat's making these models so much better on every iteration? Is it new data? Different training methods? Kinda waiting for them to plateau so I can stop feeling so existential ¯\_(ツ)_/¯
- esafak 9mo agoMore compute (bigger models, and prediction-time scaling), algorithmic advances, and ever more data (including synthetic). Remember that all white collar workers are in your position.
- shubhamjain 9mo agoYeah, I am amazed how people are brushing this off simply because GCC exists. This was far more challenging task than the browser thing, because of how far few open source compilers are there. Add to that no internet access and no dependencies. At this point, it’s hard to deny that AI has become capable of completing extremely difficult tasks, provided it has enough time and tokens.
- bjackman 9mo agoI don't think this is more challenging than the browser thing. The scope is much smaller. The fact that this is "only" 100k lines is evidence for this. But, it's still very impressive. I think this is Anthropic seeing the Cursor guy's bullshit and saying "but, we need to show people that the AI _can actually_ do very impressive shit as long as you pick a more sensible goal"