9 ms·

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ https://clan

I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. https://clangbuiltlinux.github.io/ https://clangbuiltlinux.github.io/ This LLM did it in (checks notes): > Over nearly 2,000 Claude Code sessions and $20,000 in API costs It may build, but does it boot (was also a significant and distinct next milestone)? (Also, will it blend?). Looks like yes! > The 100,000-line compiler can build a bootable Linux 6.9 on x86, ARM, and RISC-V. The next milestone is: Is the generated code correct? The jury is still out on that one for production compilers. And then you have performance of generated code. > The generated code is not very efficient. Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled. Still a really cool project!

phillmv 9mo ago
i mean… your work also went into the training set, so it's not entirely surprising that it spat a version back out!
- GaggiX 9mo ago
  Clang is not written in Rust tho
  - underdeserver 9mo ago
    jinx
- underdeserver 9mo ago
  Anthropic's version is in Rust though, so at least a little different.
  - rwmj 9mo ago
    It's not really important in latent space / conceptually.
    - D-Machine 9mo ago
      This is the proper deep critique / skepticism (or sophisticated goal-post moving, if you prefer) here. Yes, obviously this isn't just reproducing C compiler code in the training set, since this is Rust, but it is much less clear how much of the generated Rust code can (or can not) be accurately seen as being translated from C code in the training set.
  - ndesaulniers 9mo ago
    There's parts of LLVM architecture that are long in the tooth (IMO) (as is the language it's implemented in, IMO). I had hoped one day to re-implement parts of LLVM itself in Rust; in particular, I've been curious if we can concurrently compile C (and parse C in parallel, or lazily) that haven't been explored in LLVM, and I think might be safer to do in Rust. I don't know enough about grammers to know if it's technically impossible, but a healthy dose of ignorance can sometimes lead to breakthroughs. LLVM is pretty well designed for test. I was able to implement a lexer for C in Rust that could lex the Linux kernel, and use clang to cross check my implementation (I would compare my interpretation of the token stream against clang's). Just having a standard module system makes having reusable pieces seems like perhaps a better way to compose a toolchain, but maybe folks with more experience with rustc have scars to disagree?
    - jcranmer 9mo ago
      > I had hoped one day to re-implement parts of LLVM itself in Rust Heh, earlier this day, I was just thinking how crazy a proposal would it actually be to have a Rust dependency (specifically, the egg crate, since one of the things I'm banging my head against right now might be better solved with egraphs).
      - ndesaulniers 9mo ago
        Guess I better add https://github.com/bytecodealliance/rfcs/blob/main/accepted/cranelift-egraph.md#recap-e-graphs https://github.com/bytecodealliance/rfcs/blob/main/accepted/... to my reading list!
  - yoz-y 9mo ago
    One thing LLMs are really good at is translation. I haven’t tried porting projects from one language to another, but it wouldn’t surprise me if they were particularly good at that too.
    - andrekandre 9mo ago
      as someone who has done that in a professional setting, it really does work well, at least for straightforward things like data classes/initializers and average biz logic with if else statements etc... things like code annotations and other more opaque stuff like that can get more unreliable though because there are less 1-1 representations... it would be interesting to train an llm for each encountered new pattern and slowly build up a reliable conversion workflow
shakna 9mo ago
> Opus was unable to implement a 16-bit x86 code generator needed to boot into 16-bit real mode. While the compiler can output correct 16-bit x86 via the 66/67 opcode prefixes, the resulting compiled output is over 60kb, far exceeding the 32k code limit enforced by Linux. Instead, Claude simply cheats here and calls out to GCC for this phase Does it really boot...?
- ndesaulniers 9mo ago
  > Does it really boot...? They don't need 16b x86 support for the RISCV or ARM ports, so yes, but depends on what 'it' we're talking about here. Also, FWIW, GCC doesn't directly assemble to machine code either; it shells out to GAS (GNU Assembler). This blog post calls it "GCC assembler and linker" but to be more precise the author should edit this to "GNU binutils assembler and linker." Even then GNU binutils contains two linkers (BFD and GOLD), or did they excise GOLD already (IIRC, there was some discussion a few years ago about it)?
  - shakna 9mo ago
    Yeah, didn't mention gas or ld, for similar reasons. I agree that a compiler doesn't necessarily "need" those. I don't agree that all the claims are backed up by their own comments, which means that there's probably other places where it falls down. Its... Misrepresentation. Like Chicken is a Scheme compiler. But they're very up front that it depends on a C compiler. Here, they wrote a C compiler that is at least sometimes reliant on having a different C compiler around. So is the project at 50%? 75%? Even if its 99%, thats not the same story as they tried to write. And if they wrote that tale instead, it would be more impressive, rather than "There's some holes. How many?"
    - Philpax 9mo ago
      Their C compiler is not reliant on having another C compiler around. Compiling the 16-bit real mode bootstrap for the Linux kernel on x86(-64) requires another C compiler; you certainly don't need another compiler to compile the kernel for another architecture, or to compile another piece of software not subject to the 32k constraint. The compiler itself is entirely functional; it just can't generate code optimal enough to fit within the constraints for that very specific (tiny!) part of the system, so another compiler is required to do that step.
zaphirplane 9mo ago
What were the challenges out of interest. Some of it is the use of gcc extensions? Which needed an equivalent and porting over to the equivalent
- ndesaulniers 9mo ago
  `asm goto` was the big one. The x86_64 maintainers broke the clang builds very intentionally just after we had gotten x86_64 building (with necessary patches upstreamed) by requiring compiler support for that GNU C extension. This was right around the time of meltdown+spectre, and the x86_64 maintainers didn't want to support fallbacks for older versions of GCC (and ToT Clang at the time) that lacked `asm goto` support for the initial fixes shipped under duress (embargo). `asm goto` requires plumbing throughout the compiler, and I've learned more about register allocation than I particularly care... Fixing some UB in the kernel sources, lots of plumbing to the build system (particularly making it more hermetic). Getting the rest of the LLVM binutils substitutes to work in place of GNU binutils was also challenging. Rewriting a fair amount of 32b ARM assembler to be "unified syntax" in the kernel. Linker bugs are hard to debug. Kernel boot failures are hard to debug (thank god for QEMU+gdb protocol). Lots of people worked on many different parts here, not just me. Evangelism and convincing upstream kernel developers why clang support was worth anyones while. https://github.com/ClangBuiltLinux/linux/issues https://github.com/ClangBuiltLinux/linux/issues for a good historical perspective. https://github.com/ClangBuiltLinux/linux/wiki/Talks,-Presentations,-and-Communications https://github.com/ClangBuiltLinux/linux/wiki/Talks,-Present... for talks on the subject. Keynoting LLVM conf was a personal highlight (https://www.youtube.com/watch?v=6l4DtR5exwo https://www.youtube.com/watch?v=6l4DtR5exwo).
beambot 9mo ago
This is getting close to a Ken Thompson "Trusting Trust" era -- AI could soon embed itself into the compilers themselves.
- bopbopbop7 9mo ago
  A pay to use non-deterministic compiler. Sounds amazing, you should start.
  - ndesaulniers 9mo ago
    Some people care more about compile times than the performance of generated code. Perhaps even the correctness of generated code. Perhaps more so than determinism of the generated code. Different people in different contexts can have different priorities. Trying to make everyone happy can sometimes lead to making no one happy. Thus dichotomies like `-O2` vs `-Os`. EDIT (since HN is preventing me from responding): > Some people care more about compiler speed than the correctness? Yeah, I think plenty of people writing code in languages that have concepts like Undefined Behavior technically don't really care as much about correctness as they may claim otherwise, as it's pretty hard to write large volumes of code without indirectly relying on UB somewhere. What is correct in such case was left up to interpretation of the implementer by ISO WG14.
    - bopbopbop7 9mo ago
      Some people care more about compiler speed than the correctness? I would love to meet these imaginary people that are fine with a compiler that is straight up broken. Emitting working code is the baseline, not some preference slider.
      - fragmede 9mo ago
        Let's pretend, for just a second, that the people who do, having been able to learn how to program, are not absolute fucking morons. Straight up broken is obviously not useful, so maybe the conclusions you've jumped to could use some reexamination.
jbjbjbjb 9mo ago
It’s cool but there’s a good chance it’s just copying someone else’s homework albeit in an elaborate round about way.
- luke5441 9mo ago
  It looks like a much more progressed/complete version of https://github.com/kidoz/smdc-toolchain/tree/master/crates/smdc/src https://github.com/kidoz/smdc-toolchain/tree/master/crates/s... . But that one is only a month old. So a bit confused there. Maybe that was also created via LLM?
  - madmax911 9mo ago
    [dead]
- nomel 9mo ago
  I would claim that LLMs desperately need proprietary code in their training, before we see any big gains in quality. There's some incredible source available code out there. Statistically, I think there's a LOT more not so great source available code out there, because the majority of output of seasoned/high skill developers is proprietary. To me, a surprising portion of Claude 4.5 output definitely looks like student homework answers, because I think that's closer to the mean of the code population.
  - andai 9mo ago
    Let's start with the source code for the Flash IDE :)
  - bhadass 9mo ago
    yeah, but isn't the whole point of claude code to get people to provide preference data/telemetry data to anthropic (unless you opt out?). same w/ other providers. i'm guessing most of the gains we've seen recently are post training rather than pretraining.
    - 9mo ago
deleted 9mo ago
[deleted]
eek2121 9mo ago
Also: a large amount of folks seem to think Claude code is losing a ton of money. I have no idea where the final numbers land, however, if the $20,000 figure is accurate and based on some of the estimates I've seen, they could've hired 8 senior level developers at a quarter million a year for the same amount of money spent internally. Granted, marketing sucks up far too much money for any startup, and again, we don't know the actual numbers in play, however, this is something to keep in mind. (The very same marketing that likely also wrote the blog post, FWIW).
- willsmith72 9mo ago
  this doesn't add up. the 20k is in API costs. people talk about CC losing money because it's way more efficient than the API. I.e. the same work with efficient use of CC might have cost ~$5k. but regardless, hiring is difficult and high-end talent is limited. If the costs were anywhere close to equivalent, the agents are a no-brainer
  - majormajor 9mo ago
    CC hits their APIs, And internally I'm sure Anthropic tracks those calls, which is what they seem to be referencing here. What exactly did Anthropic do in this test to have "inefficient use of CC" vs your proposed "efficient use of CC"? Or do you mean that if an external user replicated this experience they might get billed less than $20k due to CC being sold at lower rates than per-API-call metered billing?
  - NitpickLawyer 9mo ago
    > hiring is difficult and high-end talent is limited. Not only that, but firing talent is also a pain. You can't "hire" 10 devs for 2 weeks, and fire them afterwards. At least you can't keep doing that, people talk and no one would apply.
- GorbachevyChase 9mo ago
  Even if the dollar cost for product created was the same, the flexibility of being able to spin a team up and down with an API call is a major advantage. That AI can write working code at all is still amazing to me.
qarl 9mo ago
> Still a really cool project! Yeah. This test sorta definitely proves that AI is legit. Despite the millions of people still insisting it's a hoax. The fact that the optimizations aren't as good as the 40 year gcc project? Eh - I think people who focus on that are probably still in some serious denial.
- PostOnce 9mo ago
  It's amazing that it "works", but viability is another issue. It cost $20,000 and it worked, but it's also totally possible to spend $20,000 and have Claude shit out a pile of nonsense. You won't know until you've finished spending the money whether it will fail or not. Anthropic doesn't sell a contract that says "We'll only bill you if it works" like you can get from a bunch of humans. Do catastrophic bugs exist in that code? Who knows, it's 100,000 lines, it'll take a while to review. On top of that, Anthropic is losing money on it. All of those things combined, viability remains a serious question.
  - deleted 9mo ago
    [deleted]
  - tumdum_ 9mo ago
    > On top of that, Anthropic is losing money on it. It seems they are *not* losing money on inference: https://bsky.app/profile/steveklabnik.com/post/3mdirf7tj5s2e https://bsky.app/profile/steveklabnik.com/post/3mdirf7tj5s2e
    - deleted 9mo ago
      [deleted]
    - byzantinegene 9mo ago
      no, and that is widely known. the actual problem is that the margins are not sufficient at that scale to make up for the gargantuan training costs to train their SOTA model.
      - 9mo ago
the_jends 9mo ago
Being just a grunt engineer in a product firm I can't imagine being able to spend multiple years on one project. If it's something you're passionate about, that sounds like a dream!
- ndesaulniers 9mo ago
  This work originally wasn't my 100% project, it was my 20% project (or as I prefer to call it, 120% project). I had to move teams twice before a third team was able to say: this work is valuable to us, please come work for us and focus just on that. I had to organize multiple internal teams, then build an external community of contributors to collaborate on this shared common goal. Having carte blanche to contribute to open source projects made this feasible at all; I can see that being a non-starter at many employers, sadly. Having low friction to change teams also helped a lot.
MaskRay 9mo ago
I want to verify the claim that it builds the Linux kernel. It quickly runs into errors, but yeah, still pretty cool! make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc -j30 defconfig all ``` /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:44:184: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) ~0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (~0x80000000); (void)pto_tmp__; } asm ("and" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:49:183: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) 0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (0x80000000); (void)pto_tmp__; } asm ("or" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:61:212: error: expected ';' after expression before 'pao_tmp__' ```
- silver_sun 9mo ago
  They said it builds Linux 6.9, maybe you are trying to compile a newer version there?
TZubiri 9mo ago
>Is the generated code correct? The jury is still out on that one for production compilers. And then you have performance of generated code. It's worth noting that this was developed by compiling Linux and running tests, so at least that is part of the training set and not the testing set. But at least for linux, I'm guessing the tests are very robust and I'm guessing that will work correctly. That said, if any bugs pop up, it will show weak points in the linux tests.
iberator 9mo ago
Claude did not wrote it. you wrote it with PREVIOUS EXPERIENCE with 20.000 long commandshyellihg him exactly what to do. Real usable AI would create it with simple: 'make c compilers c99 faster than GCC'. AI usage should be banned in general. It takes jobs faster than creating new ones ..
- arcanemachiner 9mo ago
  That's actually pretty funny. They're patting it on the back for using, in all likelihood, some significant portions of code that they actually wrote, which was stolen from them without attribution so that it could be used as part of a very expensive parlour trick.
  - whynotminot 9mo ago
    Did you do diffs to confirm the code as stolen or are you just speculating.
- wiseowise 9mo ago
  > It takes jobs faster than creating new ones .. You think compiler engineer from Google gives a single shit about this? They’ll automate millions out of career existence for their amusement while cashing out stock money and retiring early comfortably.
- embedding-shape 9mo ago
  > AI usage should be banned in general. It takes jobs faster than creating new ones .. I don't have an strong opinion about that in either direction, but curious: Do you feel the same about everything, or is just about this specific technology? For example, should the nail gun have been forbidden if it was invented today, as one person with a nail gun could probably replace 3-4 people with normal "manual" hammers? You feel the same about programmers who are automating others out of work without the use of AI too?
- 7thpower 9mo ago
  Businesses do not exist to create jobs; jobs are a byproduct.
9rx 9mo ago
> I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. How much of that time was spent writing the tests that they found to use in this experiment? You (or someone like you) were a major contributor to this. All Opus had to do here was keep brute forcing a solution until the tests passed. It is amazing that it is possible at all, but remains an impossibly without a heavy human hand. One could easily still spend a good part of their career reproducing this if they first had to rewrite all of the tests from scratch.
ip26 9mo ago
I’m excited and waiting for the team that shows with $20k in credits they can substantially speed up the generated code by improving clang!
- byzantinegene 9mo ago
  i'm sorry but that will take another $20 billion in AI capex to train our latest SOTA model so that it will cost $20k to improve the code.
grey-area 9mo ago
Isn't the AI basing what it does heavily on the publicly available source code for compilers in C though? Without that work it would not be able to generate this would it? Or in your opinion is it sufficiently different from the work people like you did to be classed as unique creation? I'm curious on your take on the references the GAI might have used to create such a project and whether this matters.
ur-whale 9mo ago
> This LLM did it You do realize the LLM had access (via his training set) and "reused" (not as is, of course) your own work, right?
tdemin 9mo ago
[dead]
brundolf 9mo ago
One thing people have pointed out is that well-specified (even if huge and tedious) projects are an ideal fit for AI, because the loop can be fully closed and it can test and verify the artifact by itself with certainty. Someone was saying they had it generate a rudimentary JS engine because the available test suite is so comprehensive Not to invalidate this! But it's toward the "well-suited for AI" end of the spectrum
- HarHarVeryFunny 9mo ago
  Yes - the gcc "torture test suite" that is mentioned must have been one of the enablers for this. It's notable that the article says Claude was unable to build a working assembler (& linker), which is nominally a much simpler task than building a compiler. I wonder if this was at least in part due to not having a test suite, although it seems one could be auto generated during bootstrapping with gas (GNU assembler) by creating gas-generated (asm, ELF) pairs as the necessary test suite. It does beg the question of how they got the compiler to point of correctness of generating a valid C -> asm mapping, before tackling the issue of gcc compatibility, since the generated code apparently has no relation to what gcc generates. I wonder which compilers' source code Claude has been trained on, and how closely this compiler's code generation and attempted optimizations compares to those?
  - spullara 9mo ago
    i'm sure claude has been trained on every open source compiler
- deleted 9mo ago
  [deleted]
VladVladikoff 9mo ago
>$20,000 of tokens. >less efficient than existing compilers what is the ecological cost of producing this piece of software that nobody will ever use?
- ryanjshaw 9mo ago
  If you evaluate the cost/benefit in isolation? It’s net negative. If you see this as part of a bigger picture to improve human industrial efficiency and bring us one step closer to the singularity? Most likely net positive.
- thefounder 9mo ago
  With that way of thinking you would just move in a cave.
HarHarVeryFunny 9mo ago
> I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel Did this come down to making Clang 100% gcc compatible (extensions, UDB, bugs and all), or were there any issues that might be considered as specific to the linux kernel? Did you end up building a gcc compatability test suite as a part of this? Did the gcc project themselves have a regression/test suite that you were able to use as a starting point?
- ndesaulniers 9mo ago
  > extensions Some were necessary (asm goto), some were not (nested functions, flexible array members not at the end of structs). > UDB, bugs and all Luckily, the kernel didn't intentionally rely on GCC specifics this way. Where it did unintentionally, we fixed the kernel sources properly with detailed commit messages explaining why. > or were there any issues that might be considered as specific to the linux kernel? Yes, https://github.com/ClangBuiltLinux/linux/issues https://github.com/ClangBuiltLinux/linux/issues is our issue tracker. We use tags extensively to mark if we triage the issue to be kernel-side vs toolchain-side. > Did you end up building a gcc compatability test suite as a part of this? No, but some tricky cases LLVM got wrong were distilled from kernel sources using either: - creduce - cvise (my favorite) - bugpoint - llvm-reduce and then added to LLVM's existing test suite. Many such tests were also simply manually written. > Did the gcc project themselves have a regression/test suite that you were able to use as a starting point? GCC and binutils have their own test suites. Folks in the LLVM community have worked on being able to test clang against GCC's test suite. I personally have never run GCC's test suite or looked at its sources.
m463 9mo ago
> getting Clang to build the linux kernel. wonder if clang source is part of its model :)

phillmv 9mo ago

i mean… your work also went into the training set, so it's not entirely surprising that it spat a version back out!

GaggiX 9mo ago
Clang is not written in Rust tho
- underdeserver 9mo ago
  jinx
underdeserver 9mo ago
Anthropic's version is in Rust though, so at least a little different.
- rwmj 9mo ago
  It's not really important in latent space / conceptually.
  - D-Machine 9mo ago
    This is the proper deep critique / skepticism (or sophisticated goal-post moving, if you prefer) here. Yes, obviously this isn't just reproducing C compiler code in the training set, since this is Rust, but it is much less clear how much of the generated Rust code can (or can not) be accurately seen as being translated from C code in the training set.
- ndesaulniers 9mo ago
  There's parts of LLVM architecture that are long in the tooth (IMO) (as is the language it's implemented in, IMO). I had hoped one day to re-implement parts of LLVM itself in Rust; in particular, I've been curious if we can concurrently compile C (and parse C in parallel, or lazily) that haven't been explored in LLVM, and I think might be safer to do in Rust. I don't know enough about grammers to know if it's technically impossible, but a healthy dose of ignorance can sometimes lead to breakthroughs. LLVM is pretty well designed for test. I was able to implement a lexer for C in Rust that could lex the Linux kernel, and use clang to cross check my implementation (I would compare my interpretation of the token stream against clang's). Just having a standard module system makes having reusable pieces seems like perhaps a better way to compose a toolchain, but maybe folks with more experience with rustc have scars to disagree?
  - jcranmer 9mo ago
    > I had hoped one day to re-implement parts of LLVM itself in Rust Heh, earlier this day, I was just thinking how crazy a proposal would it actually be to have a Rust dependency (specifically, the egg crate, since one of the things I'm banging my head against right now might be better solved with egraphs).
    - ndesaulniers 9mo ago
      Guess I better add https://github.com/bytecodealliance/rfcs/blob/main/accepted/cranelift-egraph.md#recap-e-graphs https://github.com/bytecodealliance/rfcs/blob/main/accepted/... to my reading list!
- yoz-y 9mo ago
  One thing LLMs are really good at is translation. I haven’t tried porting projects from one language to another, but it wouldn’t surprise me if they were particularly good at that too.
  - andrekandre 9mo ago
    as someone who has done that in a professional setting, it really does work well, at least for straightforward things like data classes/initializers and average biz logic with if else statements etc... things like code annotations and other more opaque stuff like that can get more unreliable though because there are less 1-1 representations... it would be interesting to train an llm for each encountered new pattern and slowly build up a reliable conversion workflow

shakna 9mo ago

> Opus was unable to implement a 16-bit x86 code generator needed to boot into 16-bit real mode. While the compiler can output correct 16-bit x86 via the 66/67 opcode prefixes, the resulting compiled output is over 60kb, far exceeding the 32k code limit enforced by Linux. Instead, Claude simply cheats here and calls out to GCC for this phase Does it really boot...?

ndesaulniers 9mo ago
> Does it really boot...? They don't need 16b x86 support for the RISCV or ARM ports, so yes, but depends on what 'it' we're talking about here. Also, FWIW, GCC doesn't directly assemble to machine code either; it shells out to GAS (GNU Assembler). This blog post calls it "GCC assembler and linker" but to be more precise the author should edit this to "GNU binutils assembler and linker." Even then GNU binutils contains two linkers (BFD and GOLD), or did they excise GOLD already (IIRC, there was some discussion a few years ago about it)?
- shakna 9mo ago
  Yeah, didn't mention gas or ld, for similar reasons. I agree that a compiler doesn't necessarily "need" those. I don't agree that all the claims are backed up by their own comments, which means that there's probably other places where it falls down. Its... Misrepresentation. Like Chicken is a Scheme compiler. But they're very up front that it depends on a C compiler. Here, they wrote a C compiler that is at least sometimes reliant on having a different C compiler around. So is the project at 50%? 75%? Even if its 99%, thats not the same story as they tried to write. And if they wrote that tale instead, it would be more impressive, rather than "There's some holes. How many?"
  - Philpax 9mo ago
    Their C compiler is not reliant on having another C compiler around. Compiling the 16-bit real mode bootstrap for the Linux kernel on x86(-64) requires another C compiler; you certainly don't need another compiler to compile the kernel for another architecture, or to compile another piece of software not subject to the 32k constraint. The compiler itself is entirely functional; it just can't generate code optimal enough to fit within the constraints for that very specific (tiny!) part of the system, so another compiler is required to do that step.

zaphirplane 9mo ago

What were the challenges out of interest. Some of it is the use of gcc extensions? Which needed an equivalent and porting over to the equivalent

ndesaulniers 9mo ago
`asm goto` was the big one. The x86_64 maintainers broke the clang builds very intentionally just after we had gotten x86_64 building (with necessary patches upstreamed) by requiring compiler support for that GNU C extension. This was right around the time of meltdown+spectre, and the x86_64 maintainers didn't want to support fallbacks for older versions of GCC (and ToT Clang at the time) that lacked `asm goto` support for the initial fixes shipped under duress (embargo). `asm goto` requires plumbing throughout the compiler, and I've learned more about register allocation than I particularly care... Fixing some UB in the kernel sources, lots of plumbing to the build system (particularly making it more hermetic). Getting the rest of the LLVM binutils substitutes to work in place of GNU binutils was also challenging. Rewriting a fair amount of 32b ARM assembler to be "unified syntax" in the kernel. Linker bugs are hard to debug. Kernel boot failures are hard to debug (thank god for QEMU+gdb protocol). Lots of people worked on many different parts here, not just me. Evangelism and convincing upstream kernel developers why clang support was worth anyones while. https://github.com/ClangBuiltLinux/linux/issues https://github.com/ClangBuiltLinux/linux/issues for a good historical perspective. https://github.com/ClangBuiltLinux/linux/wiki/Talks,-Presentations,-and-Communications https://github.com/ClangBuiltLinux/linux/wiki/Talks,-Present... for talks on the subject. Keynoting LLVM conf was a personal highlight (https://www.youtube.com/watch?v=6l4DtR5exwo https://www.youtube.com/watch?v=6l4DtR5exwo).

beambot 9mo ago

This is getting close to a Ken Thompson "Trusting Trust" era -- AI could soon embed itself into the compilers themselves.

bopbopbop7 9mo ago
A pay to use non-deterministic compiler. Sounds amazing, you should start.
- ndesaulniers 9mo ago
  Some people care more about compile times than the performance of generated code. Perhaps even the correctness of generated code. Perhaps more so than determinism of the generated code. Different people in different contexts can have different priorities. Trying to make everyone happy can sometimes lead to making no one happy. Thus dichotomies like `-O2` vs `-Os`. EDIT (since HN is preventing me from responding): > Some people care more about compiler speed than the correctness? Yeah, I think plenty of people writing code in languages that have concepts like Undefined Behavior technically don't really care as much about correctness as they may claim otherwise, as it's pretty hard to write large volumes of code without indirectly relying on UB somewhere. What is correct in such case was left up to interpretation of the implementer by ISO WG14.
  - bopbopbop7 9mo ago
    Some people care more about compiler speed than the correctness? I would love to meet these imaginary people that are fine with a compiler that is straight up broken. Emitting working code is the baseline, not some preference slider.
    - fragmede 9mo ago
      Let's pretend, for just a second, that the people who do, having been able to learn how to program, are not absolute fucking morons. Straight up broken is obviously not useful, so maybe the conclusions you've jumped to could use some reexamination.

jbjbjbjb 9mo ago

It’s cool but there’s a good chance it’s just copying someone else’s homework albeit in an elaborate round about way.

luke5441 9mo ago
It looks like a much more progressed/complete version of https://github.com/kidoz/smdc-toolchain/tree/master/crates/smdc/src https://github.com/kidoz/smdc-toolchain/tree/master/crates/s... . But that one is only a month old. So a bit confused there. Maybe that was also created via LLM?
- madmax911 9mo ago
  [dead]
nomel 9mo ago
I would claim that LLMs desperately need proprietary code in their training, before we see any big gains in quality. There's some incredible source available code out there. Statistically, I think there's a LOT more not so great source available code out there, because the majority of output of seasoned/high skill developers is proprietary. To me, a surprising portion of Claude 4.5 output definitely looks like student homework answers, because I think that's closer to the mean of the code population.
- andai 9mo ago
  Let's start with the source code for the Flash IDE :)
- bhadass 9mo ago
  yeah, but isn't the whole point of claude code to get people to provide preference data/telemetry data to anthropic (unless you opt out?). same w/ other providers. i'm guessing most of the gains we've seen recently are post training rather than pretraining.
  - 9mo ago

deleted 9mo ago

[deleted]

eek2121 9mo ago

Also: a large amount of folks seem to think Claude code is losing a ton of money. I have no idea where the final numbers land, however, if the $20,000 figure is accurate and based on some of the estimates I've seen, they could've hired 8 senior level developers at a quarter million a year for the same amount of money spent internally. Granted, marketing sucks up far too much money for any startup, and again, we don't know the actual numbers in play, however, this is something to keep in mind. (The very same marketing that likely also wrote the blog post, FWIW).

willsmith72 9mo ago
this doesn't add up. the 20k is in API costs. people talk about CC losing money because it's way more efficient than the API. I.e. the same work with efficient use of CC might have cost ~$5k. but regardless, hiring is difficult and high-end talent is limited. If the costs were anywhere close to equivalent, the agents are a no-brainer
- majormajor 9mo ago
  CC hits their APIs, And internally I'm sure Anthropic tracks those calls, which is what they seem to be referencing here. What exactly did Anthropic do in this test to have "inefficient use of CC" vs your proposed "efficient use of CC"? Or do you mean that if an external user replicated this experience they might get billed less than $20k due to CC being sold at lower rates than per-API-call metered billing?
- NitpickLawyer 9mo ago
  > hiring is difficult and high-end talent is limited. Not only that, but firing talent is also a pain. You can't "hire" 10 devs for 2 weeks, and fire them afterwards. At least you can't keep doing that, people talk and no one would apply.
GorbachevyChase 9mo ago
Even if the dollar cost for product created was the same, the flexibility of being able to spin a team up and down with an API call is a major advantage. That AI can write working code at all is still amazing to me.

qarl 9mo ago

> Still a really cool project! Yeah. This test sorta definitely proves that AI is legit. Despite the millions of people still insisting it's a hoax. The fact that the optimizations aren't as good as the 40 year gcc project? Eh - I think people who focus on that are probably still in some serious denial.

PostOnce 9mo ago
It's amazing that it "works", but viability is another issue. It cost $20,000 and it worked, but it's also totally possible to spend $20,000 and have Claude shit out a pile of nonsense. You won't know until you've finished spending the money whether it will fail or not. Anthropic doesn't sell a contract that says "We'll only bill you if it works" like you can get from a bunch of humans. Do catastrophic bugs exist in that code? Who knows, it's 100,000 lines, it'll take a while to review. On top of that, Anthropic is losing money on it. All of those things combined, viability remains a serious question.
- deleted 9mo ago
  [deleted]
- tumdum_ 9mo ago
  > On top of that, Anthropic is losing money on it. It seems they are *not* losing money on inference: https://bsky.app/profile/steveklabnik.com/post/3mdirf7tj5s2e https://bsky.app/profile/steveklabnik.com/post/3mdirf7tj5s2e
  - deleted 9mo ago
    [deleted]
  - byzantinegene 9mo ago
    no, and that is widely known. the actual problem is that the margins are not sufficient at that scale to make up for the gargantuan training costs to train their SOTA model.
    - 9mo ago

the_jends 9mo ago

Being just a grunt engineer in a product firm I can't imagine being able to spend multiple years on one project. If it's something you're passionate about, that sounds like a dream!

ndesaulniers 9mo ago
This work originally wasn't my 100% project, it was my 20% project (or as I prefer to call it, 120% project). I had to move teams twice before a third team was able to say: this work is valuable to us, please come work for us and focus just on that. I had to organize multiple internal teams, then build an external community of contributors to collaborate on this shared common goal. Having carte blanche to contribute to open source projects made this feasible at all; I can see that being a non-starter at many employers, sadly. Having low friction to change teams also helped a lot.

MaskRay 9mo ago

I want to verify the claim that it builds the Linux kernel. It quickly runs into errors, but yeah, still pretty cool! make O=/tmp/linux/x86 ARCH=x86_64 CC=/tmp/p/claudes-c-compiler/target/release/ccc -j30 defconfig all ``` /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:44:184: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) ~0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (~0x80000000); (void)pto_tmp__; } asm ("and" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:49:183: error: expected ';' after expression before 'pto_tmp__' do { u32 pto_val__ = ((u32)(((unsigned long) 0x80000000) & 0xffffffff)); if (0) { __typeof_unqual__((__preempt_count)) pto_tmp__; pto_tmp__ = (0x80000000); (void)pto_tmp__; } asm ("or" "l " "%[val], " "%" "[var]" : [var] "+m" (((__preempt_count))) : [val] "ri" (pto_val__)); } while (0); ^~~~~~~~~ fix-it hint: insert ';' /home/ray/Dev/linux/arch/x86/include/asm/preempt.h:61:212: error: expected ';' after expression before 'pao_tmp__' ```

silver_sun 9mo ago
They said it builds Linux 6.9, maybe you are trying to compile a newer version there?

TZubiri 9mo ago

>Is the generated code correct? The jury is still out on that one for production compilers. And then you have performance of generated code. It's worth noting that this was developed by compiling Linux and running tests, so at least that is part of the training set and not the testing set. But at least for linux, I'm guessing the tests are very robust and I'm guessing that will work correctly. That said, if any bugs pop up, it will show weak points in the linux tests.

iberator 9mo ago

Claude did not wrote it. you wrote it with PREVIOUS EXPERIENCE with 20.000 long commandshyellihg him exactly what to do. Real usable AI would create it with simple: 'make c compilers c99 faster than GCC'. AI usage should be banned in general. It takes jobs faster than creating new ones ..

arcanemachiner 9mo ago
That's actually pretty funny. They're patting it on the back for using, in all likelihood, some significant portions of code that they actually wrote, which was stolen from them without attribution so that it could be used as part of a very expensive parlour trick.
- whynotminot 9mo ago
  Did you do diffs to confirm the code as stolen or are you just speculating.
wiseowise 9mo ago
> It takes jobs faster than creating new ones .. You think compiler engineer from Google gives a single shit about this? They’ll automate millions out of career existence for their amusement while cashing out stock money and retiring early comfortably.
embedding-shape 9mo ago
> AI usage should be banned in general. It takes jobs faster than creating new ones .. I don't have an strong opinion about that in either direction, but curious: Do you feel the same about everything, or is just about this specific technology? For example, should the nail gun have been forbidden if it was invented today, as one person with a nail gun could probably replace 3-4 people with normal "manual" hammers? You feel the same about programmers who are automating others out of work without the use of AI too?
7thpower 9mo ago
Businesses do not exist to create jobs; jobs are a byproduct.

9rx 9mo ago

> I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel. How much of that time was spent writing the tests that they found to use in this experiment? You (or someone like you) were a major contributor to this. All Opus had to do here was keep brute forcing a solution until the tests passed. It is amazing that it is possible at all, but remains an impossibly without a heavy human hand. One could easily still spend a good part of their career reproducing this if they first had to rewrite all of the tests from scratch.

ip26 9mo ago

I’m excited and waiting for the team that shows with $20k in credits they can substantially speed up the generated code by improving clang!

byzantinegene 9mo ago
i'm sorry but that will take another $20 billion in AI capex to train our latest SOTA model so that it will cost $20k to improve the code.

grey-area 9mo ago

Isn't the AI basing what it does heavily on the publicly available source code for compilers in C though? Without that work it would not be able to generate this would it? Or in your opinion is it sufficiently different from the work people like you did to be classed as unique creation? I'm curious on your take on the references the GAI might have used to create such a project and whether this matters.

ur-whale 9mo ago

> This LLM did it You do realize the LLM had access (via his training set) and "reused" (not as is, of course) your own work, right?

tdemin 9mo ago

[dead]

brundolf 9mo ago

One thing people have pointed out is that well-specified (even if huge and tedious) projects are an ideal fit for AI, because the loop can be fully closed and it can test and verify the artifact by itself with certainty. Someone was saying they had it generate a rudimentary JS engine because the available test suite is so comprehensive Not to invalidate this! But it's toward the "well-suited for AI" end of the spectrum

HarHarVeryFunny 9mo ago
Yes - the gcc "torture test suite" that is mentioned must have been one of the enablers for this. It's notable that the article says Claude was unable to build a working assembler (& linker), which is nominally a much simpler task than building a compiler. I wonder if this was at least in part due to not having a test suite, although it seems one could be auto generated during bootstrapping with gas (GNU assembler) by creating gas-generated (asm, ELF) pairs as the necessary test suite. It does beg the question of how they got the compiler to point of correctness of generating a valid C -> asm mapping, before tackling the issue of gcc compatibility, since the generated code apparently has no relation to what gcc generates. I wonder which compilers' source code Claude has been trained on, and how closely this compiler's code generation and attempted optimizations compares to those?
- spullara 9mo ago
  i'm sure claude has been trained on every open source compiler
deleted 9mo ago
[deleted]

VladVladikoff 9mo ago

>$20,000 of tokens. >less efficient than existing compilers what is the ecological cost of producing this piece of software that nobody will ever use?

ryanjshaw 9mo ago
If you evaluate the cost/benefit in isolation? It’s net negative. If you see this as part of a bigger picture to improve human industrial efficiency and bring us one step closer to the singularity? Most likely net positive.
thefounder 9mo ago
With that way of thinking you would just move in a cave.

HarHarVeryFunny 9mo ago

> I spent a good part of my career (nearly a decade) at Google working on getting Clang to build the linux kernel Did this come down to making Clang 100% gcc compatible (extensions, UDB, bugs and all), or were there any issues that might be considered as specific to the linux kernel? Did you end up building a gcc compatability test suite as a part of this? Did the gcc project themselves have a regression/test suite that you were able to use as a starting point?

ndesaulniers 9mo ago
> extensions Some were necessary (asm goto), some were not (nested functions, flexible array members not at the end of structs). > UDB, bugs and all Luckily, the kernel didn't intentionally rely on GCC specifics this way. Where it did unintentionally, we fixed the kernel sources properly with detailed commit messages explaining why. > or were there any issues that might be considered as specific to the linux kernel? Yes, https://github.com/ClangBuiltLinux/linux/issues https://github.com/ClangBuiltLinux/linux/issues is our issue tracker. We use tags extensively to mark if we triage the issue to be kernel-side vs toolchain-side. > Did you end up building a gcc compatability test suite as a part of this? No, but some tricky cases LLVM got wrong were distilled from kernel sources using either: - creduce - cvise (my favorite) - bugpoint - llvm-reduce and then added to LLVM's existing test suite. Many such tests were also simply manually written. > Did the gcc project themselves have a regression/test suite that you were able to use as a starting point? GCC and binutils have their own test suites. Folks in the LLVM community have worked on being able to test clang against GCC's test suite. I personally have never run GCC's test suite or looked at its sources.

m463 9mo ago

> getting Clang to build the linux kernel. wonder if clang source is part of its model :)