7 ms·

I think this is a great example of both points of view in the ongoing debate. Pro-LLM coding agents: look! a working compiler built in a few hours by an agent!

I think this is a great example of both points of view in the ongoing debate. Pro-LLM coding agents: look! a working compiler built in a few hours by an agent! this is amazing! Anti-LLM coding agents: it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. Pro: Sure, but we can get the agent to fix that. Anti: Can you, though? We've seen that the more complex the code base, the worse the agents do. Fixing complex issues in a compiler seems like something the agents will struggle with. Also, if they could fix it, why haven't they? Pro: Sure, maybe now, but the next generation will fix it. Anti: Maybe. While the last few generations have been getting better and better, we're still not seeing them deal with this kind of complexity better. Pro: Yeah, but look at it! This is amazing! A whole compiler in just a few hours! How many millions of hours were spent getting GCC to this state? It's not fair to compare them like this! Anti: Anthropic said they made a working compiler that could compile the Linux kernel. GCC is what we normally compile the Linux kernel with. The comparison was invited. It turned out (for whatever reason) that CCC failed to compile the Linux kernel when GCC could. Once again, the hype of AI doesn't match the reality. Pro: but it's only been a few years since we started using LLMs, and a year or so since agents. This is only the beginning! Anti: this is all true, and yes, this is interesting. But there are so many other questions around this tech. Let's not rush into it and mess everything up.

ares623 9mo ago
Pretty much. It's missing a tiny detail though. One side is demanding we keep giving hundreds of billions to them and at the same time promising the other side's unemployment.
- lelanthran 9mo ago
  > One side is demanding we keep giving hundreds of billions to them and at the same time promising the other side's unemployment. That's a valid take. The problem is that there are, at this time, so many valid takes that it's hard to determine which are more valid/accurate than the other. FWIW, I think this is more insightful than most of the takes I've seen, which basically amount to "side-1: we're moving to a higher level of abstraction" and "side-2: it's not higher abstraction, just less deterministic codegen".
  - lubujackson 9mo ago
    I'm on the "higher level of abstraction" side, but that seems to be very much at odds with however Anthropic is defining it. Abstraction is supposed to give you better high-level clarity at the expense of low-level detail. These $20,000 burning, Gas Town-style orchestration matrices do anything but simplify high level concerns. In fact, they seem committed building extremely complex, low-level harnesses of testing and validation and looping cycles around agents upon agents to avoid actually trying to deal with whatever specific problem they are trying to solve. How do you solve a problem you refuse to define explicitly? We end up with these Goodhart's Law solutions: they hit all of the required goals and declare victory, but completely fail in every reasonable metric that matters. Which I guess is an approach you make when you are selling agents by the token, but I don't see why anyone else is enamored with this approach.
- FridgeSeal 9mo ago
  You’re copping downvotes for this, but you’re not wrong. “It will get better, and then we will use it to make many of you unemployed” Colour-me-shocked that swathes of this industry might have an issue with that.
- kristiandupont 9mo ago
  I don't feel that I see this anywhere but if so, I guess I'm in a third camp. I am "pro" in the sense that I believe that LLM's are making traditional programming obsolete. In fact there isn't any doubt in my mind. However, I am "anti" in the sense that I am not excited or happy about it at all! And I certainly don't encourage anyone to throw money at accelerating that process.
  - bopbopbop7 9mo ago
    > I believe that LLM's are making traditional programming obsolete. In fact there isn't any doubt in my mind. Is this what AI psychosis looks like? How can anyone that is a half decent programmer actually believe that English + non-deterministic code generator will replace "traditional" programming?
    - bux93 9mo ago
      That's also my take, vibe coding as a non-deterministic 4GL. https://en.wikipedia.org/wiki/Fourth-generation_programming_language https://en.wikipedia.org/wiki/Fourth-generation_programming_... 4GLs are productive yes, but also limited, and still require someone to come up with specs that are both informed by (business) realities and engineering considerations. But this is also an arena where bosses expect magic to happen when people are involved; just pronounce a new strategy, and your business magically transforms - without any of that pesky 'figuring out what to do' or 'aligning stakeholders' or 'wondering what drugs the c-suite is doing'. Let LLMs write the specs!
  - simianwords 9mo ago
    Of all takes, I find this most honest and believable. Not many would want a disruption of their stable life
- kuerbel 9mo ago
  And no-one ever stops and thinks about what it means to give up so much control. Maybe one of those companies will come out on top. The others produce garbage in comparison. Capital loves a single throat to choke and doesn't gently pluralise. So of course you buy the best service. And it really can generate any code, get it working, bug free. People unlearn coding on this level. And some day, poof, Microsoft is coming around and having some tiny problem that it can generate a working Office clone. Or whatever, it's just an example. This technology will never be used to set anyone free. Never. The entity that owns the generator owns the effective means of production, even if everyone else can type prompts. The same technology could, in a different political and economic universe, widen human autonomy. But that universe would need strong commons, enforced interoperability, and a cultural refusal to outsource understanding. And why is this different from abstractions that came before? There are people out there understanding what compilers are doing. They understand the model from top to bottom. Tools like compilers extended human agency while preserving a path to mastery. AI code generation offers capability while dissolving the ladder behind you. We are not merely abstracting labor. We are abstracting comprehension itself. And once comprehension becomes optional, it rapidly becomes rare. Once it becomes rare, it becomes political. And once it becomes political, it will not be distributed generously.
  - stoneforger 9mo ago
    Nah bro it makes them productive. Get with the program. Amazing . Fantastic. Of course it resonates with idiots because they can't think beyond the vicinity of their own greed. We are doomed , noone gives two cents. Idiocracy is here and it's not Costco.
    - kuerbel 9mo ago
      Sorry! Of course. What an amazing tech. And look, the CEOs are promising us a good future! Maybe we can cool the datacenters with Brawndo. Let me ask chat if that is a good idea.
  - nopurpose 9mo ago
    You could make same argument in "information superhighway" days, but it turned out to be the opposite: no company monopolised internet services, despite trying hard. With so many companies in AI race it is already pretty competitive landscape and it doesnt seem likely to me that any of them can build deep enough moat to come ahead.
soulofmischief 9mo ago
In my experience, it is often the other way around. Enthusiasts are tasked with trying to open minds that seem very closed on the subject. Most serious users of these tools recognize the shortcomings and also can make well-educated guesses on the short term future. It's the anti crowd who get hellbent on this ridiculously unfounded "robots are just parrots and can't ever replace real programmers" shtick.
- rockskon 9mo ago
  Maybe if AI evangelists would stop lying about what AI can do then people would hate it less. But lying and hype is baked into the DNA of AI booster culture. At this point it can be safely assumed anything short of right-here-right-now proof is pure unfettered horseshit when coming from anyone and everyone promoting the value of AI.
  - soulofmischief 9mo ago
    Your comment is a perfect example of the biases I'm talking about. "AI evangelists" are not a singular group of people.
    - rockskon 9mo ago
      You're right! Sometimes even the right-here-right-now claims of AI capabilities are horseshit too with people in actuality remotely controlling the product. It's not common for present-capabilities to be lied about too. But it does happen! And the smallest presence are the users who don't work in the AI industry but rave about AI. I know of...two....people who fit that bill. A lead developer at a cybersecurity firm and someone who works heavily in statistics and data analytics. Both of which are very senior people in their fields who can articulate exactly what they're looking for without much left to interpretation.
      - soulofmischief 8mo ago
        Now you're just presenting straw man arguments and continuing to conflate separate issues.
nvrmnd 9mo ago
> It's not fair to compare them like this! As someone who leans pro in this debate, I don't think I would make that statement. I would say the results are exactly as we expect. Also, a highly verifiable task like this is well suited to LLMs, and I expect within the next ~2 years AI tools will produce a better compiler than gcc.
- dashzebra 9mo ago
  The same two years as in "full self driving available in 2 years"? Right.
  - tyre 9mo ago
    These are different technologies with different rates of demonstrated growth. They have very little to do with each other.
    - dashzebra 9mo ago
      Well let's check again in two years then.
- phorkyas82 9mo ago
  Don't forget that gcc is in the training set. That's what always puts me off: when AI replaces artists, SO and FOSS projects, it can only feed into itself and deteriorate..
  - FeepingCreature 9mo ago
    it can feed into itself and improve. the idea that self-training necessarily causes deterioration is fanfic. remember that they spend massive amounts of compute on rl.
  - kuboble 9mo ago
    The AlphaZero approach shows otherwise, as long as there is an automated way to generate new test cases and evaluate the outcomes. We can't do it for all domains, but I believe we can for efficient code. today's models could be probably already good enough to compose tasks, and evaluate the results.
raincole 9mo ago
Are you trying to demonstrate a textbook example of straw man argument?
- co_king_3 9mo ago
  Um actually, it's called a Steel Man argument. I learned about it from HackerNews and ChatGPT.
NitpickLawyer 9mo ago
Two thoughts here: First, remember when we had LLMs run optimisation passes last year? Alphaevolve doing square packing, and optimising ML kernels? The "anti" crowd was like "well, of course it can automatically optimise some code, that's easy". And things like "wake me up when it does hard tasks". Now, suddenly when they do hard tasks, we're back at "haha, but it's unoptimised and slow, laaame". Second, if you could take 100 juniors, 100 mid level devs and 100 senior devs, lock them in a room for 2 weeks, how many working solutions that could boot up linux in 2 different arches, and almost boot in the third arch would you get? And could you have the same devs now do it in zig? The thing that keeps coming up is that the "anti" crowd is fighting their own deamons, and have kinda lost the plot along the way. Every "debate" is about promisses, CEOs, billions, and so on. Meanwhile, at every step of the way these things become better and better. And incredibly useful in the right hands. I find it's best to just ignore the identity folks, and keep on being amazed at the progress. The haters will just find the next goalpost and the next fight with invisible entities. To paraphrase - those who can, do, those who can't, find things to nitpick.
- Ygg2 9mo ago
  First off Alpha Evolve isn't an LLM. No more than a human is a kidney. Second depends. If you told them to pretrain for writing C compiler however long it takes, I could see a smaller team doing it in a week or two. Keep in mind LLMs pretrain on all OSS including GCC. > Meanwhile, at every step of the way these things become better and better. Will they? Or do they just ingest more data and compute?[1] Again, time will tell. But to me this seems more like speed-running into an Idiocracy scenario than a revolution.[2] I think this will turn out another driverless car situation where last 1% needs 99% of the time. And while it might happen eventually it's going to take extremely long time. [1] Because we don't have much more computing jumps left, nor will future data be as clean as now. [2] Why idiocracy? Because they are polluting their own corpus of data. And by replacing thinking about computers, there will be no one to really stop them. We'll equalize the human and computer knowledge by making humans less knowledgeable rather than more. So you end up in an Idiocracy-like scenario where a doctor can't diagnose you, nor can the machine because it was dumbed down by each successive generation, until it resembles a child's toy.
deleted 9mo ago
[deleted]
bgirard 9mo ago
This to me sounds a lot like the SpaceX conversation: - Ohh look it can [write small function / do a small rocket hop] but it can't [ write a compiler / get to orbit]! - Ohh look it can [write a toy compiler / get to orbit] but it can't [compile linux / be reusable] - Ohh look it can [compile linux / get reusable orbital rocket] but it can't [build a compiler that rivals GCC / turn the rockets around fast enough] - <Denial despite the insane rate of progress> There's no reason to keep building this compiler just to prove this point. But I bet it would catch up real fast to GCC with a fraction of the resources if it was guided by a few compiler engineers in the loop. We're going to see a lot of disruption come from AI assisted development.
- raincole 9mo ago
  > the insane rate of progress Yeah but the speed of progress can never catch the speed of a moving goalpost!
  - friendzis 9mo ago
    > Yeah but the speed of progress can never catch the speed of a moving goalpost! How do you like those coast-to-coast self drives since the end of 2017?
  - wrxd 9mo ago
    What about the hype? If you claim your LLM generated compiler is functionally on par with GCC I’d expect it to match your claim. I still won’t use it while it also matches all the non-functional requirements but you’re free to go and recompile all the software you use with it.
  - samultio 9mo ago
    Training data only teaches it how to reach the goalpost, not how to overtake it.
    - codethief 9mo ago
      Are we sure about that? I mean, we have seen that LLMs are able to generalize to some degree. So I don't see a reason why you couldn't put an agent in a loop with a profiler and have it try to optimize the code. Will it come up with entirely novel ideas? Unlikely. Could it potentially combine existing ideas in interesting, novel ways that would lead to CCC outperforming GCC? I think so. Will it get stuck along the way? Almost certainly.
dataflow 9mo ago
> Pro: Sure, maybe now, but the next generation will fix it. Do we need a c2 wiki page for "sufficiently smart LLM" like we do for https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler ?
- sungho_ 9mo ago
  > This is still true - a compiler can never win this battle. All a human programmer has to do is take the output of the compiler and make a single optimisation, and he/she wins. This is the advantage that the human has - they can use any of a wide variety of tools at their disposal (including the compiler), whilst the compiler can only do what it was programmed. The best the compiler can hope for is a tie. This is awkward
Alupis 9mo ago
I'm reminded, once again, of the recent "vibe coded" OCaml fiasco[1]. The PR author had zero understanding why their entirely LLM-generated contribution was viewed so suspiciously. The article validates a significant point: it is one thing to have passing tests and be able to produce output that resembles correctness - however it's something entirely different for that output to be good and maintainable. [1] https://github.com/ocaml/ocaml/pull/14369 https://github.com/ocaml/ocaml/pull/14369
- Culonavirus 9mo ago
  Damn... "AI has a very deep understanding of how this code works. Please challenge me on this." this person is something else. Just... wow.
  - copilot_king_2 9mo ago
    This is the norm in my experience. If you haven't come across a significant number of AI addicts as obnoxiously delusional as @Culonavirus describes, you must be getting close to retirement age. People with any connection to new college graduates understand that this sort of idiotic LLM-backed arrogance is extremely common among low-to-mid-functioning twenty-somethings.
  - co_king_3 9mo ago
    [flagged]
    - monegator 9mo ago
      *grifters
      - co_king_3 9mo ago
        Successful grifters are not this stupid. Terminal AI users are genuinely *slow* individuals. These people are functionally illiterate; they are not able to extract information by reading a bit of text in the same ways that you or I are. These people are very, very *gullible*.
gignico 9mo ago
Exactly. This flawed argument by which everything will be fixed by future models drives me crazy every time.
- bopbopbop7 9mo ago
  Just a couple more trillion and 6 more months!
- deleted 9mo ago
  [deleted]
- simianwords 9mo ago
  That’s been the trend for a while. Can you make a prediction that says something concretely like “AI will not be able to do X by 2028” for a specific and well defined X?
  - direwolf20 9mo ago
    Were any made about 2025?
  - xigoi 9mo ago
    In 2030, an AI model that I can run on my computer, without having to trust an evil megacorporation, will not be able to write a compiler for my markup language [0] based on a corpus of examples, without seeing the original implementation, using no more than 1.5× as much code as I did. https://git.sr.ht/~xigoi/hilda https://git.sr.ht/~xigoi/hilda
- Kiro 9mo ago
  So far it has been accurate though. Models have gotten much better than even the most optimistic predictions.
  - LinXitoW 9mo ago
    No? The most optimistic predictions involved AGI around the corner, 6 months until no more developers for years now.
red75prime 9mo ago
> Pro-LLM coding agents: look! a working compiler built in a few hours by an agent! this is amazing! > Anti-LLM coding agents: it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. Pro-LLM: Read the freaking article, it's not that long. The compiler made a mistake in an area where only two compilers exist that are up to the task: Linux Kernel.
- wrxd 9mo ago
  Anti-LLM: isn’t all this intelligence supposed to give us something better than what we already have?
  - red75prime 9mo ago
    Me: Top 0.02%[1] human-level intelligence? Sure. But we aren't there yet. [1] There are around 8k programming languages that are used (or were used) in practice (that is, they were deemed better than existing ones in some aspects) and there are around 50 million programmers. I use it to estimate how many people did something, which is objectively better than existing products.
- marcus_holmes 9mo ago
  Anthropic said they vibe-coded a C compiler that could compile the Linux kernel. That's what they said. No-one forced them to say that. They could have picked another code base. It turns out that isn't true in all instances, as this article demonstrates. I'm not nearly expert enough to be able to decide if that error was simple, stupid, irrelevant, or whatever. I can make a call on whether it successfully compiled the Linux kernel: it did not.
  - red75prime 9mo ago
    I'm sorry for being excessively edgy, but "it's useless" is not a good summary for "linking errors after successfully compiling Linux kernel for x86_64."
    - 9mo ago
rk06 9mo ago
As an Anti, my argument is "if AI will good in future, then come back in the future"
- aurareturn 9mo ago
  As a pro, my argument is "it's good enough now to make me incredibly productive, and it's only going to keep getting better because of advancements in compute". I'd rather get really good at leveraging AI now than to bury my head in the sand hoping this will go away. I happen to agree with the saying that AI isn't going to replace people, but people using AI will replace people who don't. So by the time you come back in the future, you might have been replaced already.
  - player1234 9mo ago
    [dead]
  - techpression 9mo ago
    Why would anything you learn today be relevant tomorrow if AI keeps advancing? You would need less and less of all your tooling, markdown files and other rituals and just let the AI figure it out altogether.
    - aurareturn 9mo ago
      So I can keep my job now so I can pay for compute in the future when I'm out of a job. The compute will be used to create my own business to make money.
      - techpression 9mo ago
        Those are two different things though, and not everyone are stuck at a place enforcing token usage. And why would anyone pay you for something if all it takes is compute to make it? They would just make it themselves.
      - layer8 9mo ago
abrbhat 9mo ago
It seems that the cause of the difference in opinion is that the anti camp is looking at the current state while the pro camp looking at the slope and projecting it into the future.
quaintdev 9mo ago
I read a Youtube comment recently on pro AI video, it was "The source code of gcc is available online"
yoyohello13 9mo ago
I think LLMs the technology is very cool and l’m frankly amazed at what it can do. What I’m ‘anti’ about is pushing the entire economy all in on LLM tech. The accelerationist take of ‘just keep going as fast as possible and it will work out, trust me bro’ is the most unhinged dangerous shit I’ve ever heard and unfortunately seems to be the default worldview of those in charge of the money. I’m not sure where all the AI tools will end up, but I am willing to bet big that the average person is not going to be better off 10 years from now. The direction the world is going scares the shit out of me and the usages of AI by bad actors is not helping assuage that fear. Honestly? I think if we as a society could trust our leaders (government and industry) to not be total dirtbags the resistance to AI would be much lower. Like imagine if the message was “hey, this will lead to unemployment, but we are going to make sure people can still feed their families during the transition, maybe look in to ways to support subsidies retraining programs for people whose jobs have been impacted .” Seems like a much more palatable narrative than, “fuck you pleb! go retrain as a plumber or die in a ditch. I’ll be on my private island counting the money I made from destroying your livelihood.”
aurareturn 9mo ago
I don't think this is how pro and anti conversation goes. I think the pro would tell you that if GCC developers could leverage Opus 4.6, they'd be more productive. The anti would tell you that it doesn't help with productivity, it makes us less versed in the code base. I think the CCC project was just a demonstration on what Opus can do now autonomously. 99.9% of software projects out there aren't building something as complex as a Linux compiler.
Rapzid 9mo ago
Two completely valid perspectives. Unless you need a correctly compiled Linux kernel. In that case one gets exhausting real quick.
- Rygian 9mo ago
  The perspective that says "a whole compiler in just a few hours" is making false claims. So not a valid perspective.
nikitau 9mo ago
And not to mention that a C compiler is something we have literally 50 years worth of code for. I still seriously doubt the ability of LLMs to tackle truly new problems.
- simianwords 9mo ago
  What do you classify as new? Every problem that we solve as developers is a very small deviation from already existing problems. Maybe that’s the point of llms? How many developers do you think are solving truly novel problems? Most like me are CRUD bunnies.
  - 1718627440 9mo ago
    And these developers do not write the majority of their codebase, they use tons of libraries and only write the glue code.
  - xigoi 9mo ago
    If your problem is a very small deviation from an existing problem, you should be able to take an existing open-source solution and make a very small modification to adapt it to your use case. No need for “vibe-coding” a lower-quality implementation from scratch.
    - palmotea 9mo ago
      Yeah, it kind of strikes me how a lot of the LLM use cases would actually be better served by existing techniques, like more/better libraries. And if that's not possible, it'd be way better to find the closest match, fork it, and make minimal modifications. At least then you have the benefit of an upstream. But, sort of like cryptocurrency, the LLM people aren't so much trying to solve actual problems, but rather find an application of their existing technology. Sort of like the proverbial saying: when you're selling hammers, you want convince everyone that their problem as a nail.
frizlab 9mo ago
I think you also forgot: Anti: But the whole thing can only have been generated because GCC and other compilers already exists (and depending on how strong the anti-feeling is: and has been stolen…)!
_23sd 9mo ago
What does this imagined conversation have to do with the linked article? The “pro” and “anti” character both sound like the kind of insufferable idiots I’d expect to encounter on social media, the OP is a very nice blog post about performance testing and finding out what compilers do, doesn’t attempt any unwarranted speculation about what agents “struggle with” or will do “next generation”, how is it an example of that sort of shitposting?
abbyprog 9mo ago
I'm firmly in the anti/unimpressed camp so far - but of course open to see where it goes. I mean this compiler is the equivalent of handing someone a calculator when it was first invented and seeing that it took 2 hours to multiply two numbers together, I would go "cool that you have a machine that can do math, but I can multiply faster by hand, so it's a useless device to me".
_zagj 9mo ago
The "Anti" stance is only tenable now if you believe LLMs are going to hit a major roadblock in the next few months around which Big AI won't be able to navigate. Something akin to the various "ghosts in the machine" that started bedeviling EEs after 2000 when transistors got sufficiently small, including gate leakage and sub-threshold current, such that Dennard Scaling came to an abrupt end and clock speeds stalled. I personally hope that that happens, but I doubt it will. Note also that processors still continued to improve even without Dennard Scaling due to denser, better optimized onboard caches, better branch prediction, and more parallelism (including at the instruction level), and the broader trend towards SoCs and away from PCB-based systems, among other things. So at least by analogy, it's not impossible that even with that conjectured roadblock, Big AI could still find room for improvement, just at a much slower rate. But current LLMs are thoroughly compelling, and even just continued incremental improvements will prove massively disruptive to society.
weli 9mo ago
> this is all true, and yes, this is interesting. But there are so many other questions around this tech. Let's not rush into it and mess everything up. That's a really nice fictitious conversation but in my experience "anti-ai" people would be prone to say "This is stupid LLM's will never be able to write complex code and attempting to do so is futile". If your mind is open to explore how LLM's will actually write complex software then by definition you are not "anti".
viraptor 9mo ago
That's such a strawman conversation. Starting from: > it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. It works. It's not perfect, but anthropic claims to have successfully compiled and booted 3 different configurations with it. The blog post failed to reproduce one specific version on one specific architecture. I wish anthropic gave us more information about which kernel commits succeeded, but still. Compare this to years that it took for clang to compile the kernel, yet people were not calling that compiler useless. If anyone thinks other compilers "just work", I invite them to start fixing packages that fail to build in nixos after every major compiler change, to get a dose of real world experience.
agumonkey 9mo ago
This is a pattern I see a lot, in programming languages communities too, where it's a source a joy and dreams first and facts later.
nineteen999 9mo ago
I mean - who would honestly expect an LLM to be able to compete with a compiler with 40 years of development behind it? Even more if you count the collective man years expended in that time. The Claude agents took two weeks to produce a substandard compiler, under the fairly tight direction of a human who understood the problem space. At the same time - you could direct Claude to review the register spilling code and the linker code of both LLVM/gcc for potential improvements to CCC and you will see improvements. You can ask it not to copy GPL code verbatim but to paraphrase and tell it it can rip code from LLVM as long as the licenses are preserved. It will do it. You might only see marginal improvements without spending another $100K on API calls. This is about one of the hardest projects you could ask it to bite off and chew on. And would you trust the compiler output yet over GCC or LLVM? Of course not. But I wager, that if you _started_ with the LLVM/gcc codebases and asked it to look for improvements - it might be surprising to see what it finds. Both sides have good arguments. But this could be a totally different ball game in 2, 5 and 10 years. I do feel like those who are most terrified by it are those whose identity is very much tied to being a programmer, and seeing the potential for their role to be replaced and I can understand that. Me personally - I'm relieved I finally have someone else to blame and shout at rather than myself for the bugs in the software I produce. I'm relieved that I can focus now on the more creative direction and design of my personal projects (and even some work projects on the non-critical paths) and not get bogged down in my own perfectionism with respect to every little component until reaching exhaustion and giving up. And I'm fascinated by the creativity of some of the projects I see that are taking the same mindset and approach. I was depressed by it at first. But as I've experimented more and more, I've come to enjoy seeing things that I couldn't ever have achieved even with 100 man years of my own come to fruition.
kaycey2022 9mo ago
Maybe Anthropic can sponsor a research team to polish this using just an agent. A lot of things can be learned from that exercise.
zozbot234 9mo ago
You didn't even mention that this vibe-coded toy compiler cost $20k in token spend. That's an insane amount of money for what this is.
- layer8 9mo ago
  It seems at least comparable to what you would have to pay a suitably competent developer to code up something similar.
  - palmotea 9mo ago
    > It seems at least comparable to what you would have to pay a suitably competent developer to code up something similar. No. You could have someone of fiverr do it for $5. Download gcc, done.
    - layer8 9mo ago
      If only the Claude team knew that, they could have saved a lot of tokens.
sylware 9mo ago
You are missing the most important: "Pro": give me tons of money to keep going this endeavour. More seriously: if some LLM or other can _assist_ c++ to plain and simple C port...
63stack 9mo ago
This is spot on, you can find traces of this conversation in the original thread posted on HN as well, where people are proclaiming "yeah it doesn't work, but still impressive!" Reminds me so much of the people posting their problems about the tesla cybertruck and ending the post with "still love the truck though"
Wowfunhappy 9mo ago
> Pro: but it's only been a few years since we started using LLMs, and a year or so since agents. This is only the beginning! The billion dollar question is, can we get from 80% to 100%? Is this going to be a situation where that final gap is just insurmountable, or will the capabilities simply keep increasing?
palmotea 9mo ago
> Pro-LLM coding agents: look! a working compiler built in a few hours by an agent! this is amazing! > Anti-LLM coding agents: it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. Also, from the Anti-LLM perspective: did the coding agent actually build a working compiler, or just plagiarize prior art? C compilers are certainly part of the LLM's training set. That's relevant because the implication seems to be: "Look, the agent can successfully develop really advanced software!" when the reality may be that it can plagiarize existing advanced software, and will fall on its face if asked to do anything not already done before. A lot of propaganda and hype follows the pattern of presenting things in a way to creating misleading implications in the mind of the listener that the facts don't actually support.
bicepjai 9mo ago
This encompasses all the back-and-forth arguments I can think of. I would assume that proponents will eventually also mention AGI or ASI. :)
giancarlostoro 9mo ago
We could be colonizing Mars with Claude Code and there will always be some skeptic somewhere.

ares623 9mo ago

Pretty much. It's missing a tiny detail though. One side is demanding we keep giving hundreds of billions to them and at the same time promising the other side's unemployment.

lelanthran 9mo ago
> One side is demanding we keep giving hundreds of billions to them and at the same time promising the other side's unemployment. That's a valid take. The problem is that there are, at this time, so many valid takes that it's hard to determine which are more valid/accurate than the other. FWIW, I think this is more insightful than most of the takes I've seen, which basically amount to "side-1: we're moving to a higher level of abstraction" and "side-2: it's not higher abstraction, just less deterministic codegen".
- lubujackson 9mo ago
  I'm on the "higher level of abstraction" side, but that seems to be very much at odds with however Anthropic is defining it. Abstraction is supposed to give you better high-level clarity at the expense of low-level detail. These $20,000 burning, Gas Town-style orchestration matrices do anything but simplify high level concerns. In fact, they seem committed building extremely complex, low-level harnesses of testing and validation and looping cycles around agents upon agents to avoid actually trying to deal with whatever specific problem they are trying to solve. How do you solve a problem you refuse to define explicitly? We end up with these Goodhart's Law solutions: they hit all of the required goals and declare victory, but completely fail in every reasonable metric that matters. Which I guess is an approach you make when you are selling agents by the token, but I don't see why anyone else is enamored with this approach.
FridgeSeal 9mo ago
You’re copping downvotes for this, but you’re not wrong. “It will get better, and then we will use it to make many of you unemployed” Colour-me-shocked that swathes of this industry might have an issue with that.
kristiandupont 9mo ago
I don't feel that I see this anywhere but if so, I guess I'm in a third camp. I am "pro" in the sense that I believe that LLM's are making traditional programming obsolete. In fact there isn't any doubt in my mind. However, I am "anti" in the sense that I am not excited or happy about it at all! And I certainly don't encourage anyone to throw money at accelerating that process.
- bopbopbop7 9mo ago
  > I believe that LLM's are making traditional programming obsolete. In fact there isn't any doubt in my mind. Is this what AI psychosis looks like? How can anyone that is a half decent programmer actually believe that English + non-deterministic code generator will replace "traditional" programming?
  - bux93 9mo ago
    That's also my take, vibe coding as a non-deterministic 4GL. https://en.wikipedia.org/wiki/Fourth-generation_programming_language https://en.wikipedia.org/wiki/Fourth-generation_programming_... 4GLs are productive yes, but also limited, and still require someone to come up with specs that are both informed by (business) realities and engineering considerations. But this is also an arena where bosses expect magic to happen when people are involved; just pronounce a new strategy, and your business magically transforms - without any of that pesky 'figuring out what to do' or 'aligning stakeholders' or 'wondering what drugs the c-suite is doing'. Let LLMs write the specs!
- simianwords 9mo ago
  Of all takes, I find this most honest and believable. Not many would want a disruption of their stable life
kuerbel 9mo ago
And no-one ever stops and thinks about what it means to give up so much control. Maybe one of those companies will come out on top. The others produce garbage in comparison. Capital loves a single throat to choke and doesn't gently pluralise. So of course you buy the best service. And it really can generate any code, get it working, bug free. People unlearn coding on this level. And some day, poof, Microsoft is coming around and having some tiny problem that it can generate a working Office clone. Or whatever, it's just an example. This technology will never be used to set anyone free. Never. The entity that owns the generator owns the effective means of production, even if everyone else can type prompts. The same technology could, in a different political and economic universe, widen human autonomy. But that universe would need strong commons, enforced interoperability, and a cultural refusal to outsource understanding. And why is this different from abstractions that came before? There are people out there understanding what compilers are doing. They understand the model from top to bottom. Tools like compilers extended human agency while preserving a path to mastery. AI code generation offers capability while dissolving the ladder behind you. We are not merely abstracting labor. We are abstracting comprehension itself. And once comprehension becomes optional, it rapidly becomes rare. Once it becomes rare, it becomes political. And once it becomes political, it will not be distributed generously.
- stoneforger 9mo ago
  Nah bro it makes them productive. Get with the program. Amazing . Fantastic. Of course it resonates with idiots because they can't think beyond the vicinity of their own greed. We are doomed , noone gives two cents. Idiocracy is here and it's not Costco.
  - kuerbel 9mo ago
    Sorry! Of course. What an amazing tech. And look, the CEOs are promising us a good future! Maybe we can cool the datacenters with Brawndo. Let me ask chat if that is a good idea.
- nopurpose 9mo ago
  You could make same argument in "information superhighway" days, but it turned out to be the opposite: no company monopolised internet services, despite trying hard. With so many companies in AI race it is already pretty competitive landscape and it doesnt seem likely to me that any of them can build deep enough moat to come ahead.

soulofmischief 9mo ago

In my experience, it is often the other way around. Enthusiasts are tasked with trying to open minds that seem very closed on the subject. Most serious users of these tools recognize the shortcomings and also can make well-educated guesses on the short term future. It's the anti crowd who get hellbent on this ridiculously unfounded "robots are just parrots and can't ever replace real programmers" shtick.

rockskon 9mo ago
Maybe if AI evangelists would stop lying about what AI can do then people would hate it less. But lying and hype is baked into the DNA of AI booster culture. At this point it can be safely assumed anything short of right-here-right-now proof is pure unfettered horseshit when coming from anyone and everyone promoting the value of AI.
- soulofmischief 9mo ago
  Your comment is a perfect example of the biases I'm talking about. "AI evangelists" are not a singular group of people.
  - rockskon 9mo ago
    You're right! Sometimes even the right-here-right-now claims of AI capabilities are horseshit too with people in actuality remotely controlling the product. It's not common for present-capabilities to be lied about too. But it does happen! And the smallest presence are the users who don't work in the AI industry but rave about AI. I know of...two....people who fit that bill. A lead developer at a cybersecurity firm and someone who works heavily in statistics and data analytics. Both of which are very senior people in their fields who can articulate exactly what they're looking for without much left to interpretation.
    - soulofmischief 8mo ago
      Now you're just presenting straw man arguments and continuing to conflate separate issues.

nvrmnd 9mo ago

> It's not fair to compare them like this! As someone who leans pro in this debate, I don't think I would make that statement. I would say the results are exactly as we expect. Also, a highly verifiable task like this is well suited to LLMs, and I expect within the next ~2 years AI tools will produce a better compiler than gcc.

dashzebra 9mo ago
The same two years as in "full self driving available in 2 years"? Right.
- tyre 9mo ago
  These are different technologies with different rates of demonstrated growth. They have very little to do with each other.
  - dashzebra 9mo ago
    Well let's check again in two years then.
phorkyas82 9mo ago
Don't forget that gcc is in the training set. That's what always puts me off: when AI replaces artists, SO and FOSS projects, it can only feed into itself and deteriorate..
- FeepingCreature 9mo ago
  it can feed into itself and improve. the idea that self-training necessarily causes deterioration is fanfic. remember that they spend massive amounts of compute on rl.
- kuboble 9mo ago
  The AlphaZero approach shows otherwise, as long as there is an automated way to generate new test cases and evaluate the outcomes. We can't do it for all domains, but I believe we can for efficient code. today's models could be probably already good enough to compose tasks, and evaluate the results.

raincole 9mo ago

Are you trying to demonstrate a textbook example of straw man argument?

co_king_3 9mo ago
Um actually, it's called a Steel Man argument. I learned about it from HackerNews and ChatGPT.

NitpickLawyer 9mo ago

Two thoughts here: First, remember when we had LLMs run optimisation passes last year? Alphaevolve doing square packing, and optimising ML kernels? The "anti" crowd was like "well, of course it can automatically optimise some code, that's easy". And things like "wake me up when it does hard tasks". Now, suddenly when they do hard tasks, we're back at "haha, but it's unoptimised and slow, laaame". Second, if you could take 100 juniors, 100 mid level devs and 100 senior devs, lock them in a room for 2 weeks, how many working solutions that could boot up linux in 2 different arches, and almost boot in the third arch would you get? And could you have the same devs now do it in zig? The thing that keeps coming up is that the "anti" crowd is fighting their own deamons, and have kinda lost the plot along the way. Every "debate" is about promisses, CEOs, billions, and so on. Meanwhile, at every step of the way these things become better and better. And incredibly useful in the right hands. I find it's best to just ignore the identity folks, and keep on being amazed at the progress. The haters will just find the next goalpost and the next fight with invisible entities. To paraphrase - those who can, do, those who can't, find things to nitpick.

Ygg2 9mo ago
First off Alpha Evolve isn't an LLM. No more than a human is a kidney. Second depends. If you told them to pretrain for writing C compiler however long it takes, I could see a smaller team doing it in a week or two. Keep in mind LLMs pretrain on all OSS including GCC. > Meanwhile, at every step of the way these things become better and better. Will they? Or do they just ingest more data and compute?[1] Again, time will tell. But to me this seems more like speed-running into an Idiocracy scenario than a revolution.[2] I think this will turn out another driverless car situation where last 1% needs 99% of the time. And while it might happen eventually it's going to take extremely long time. [1] Because we don't have much more computing jumps left, nor will future data be as clean as now. [2] Why idiocracy? Because they are polluting their own corpus of data. And by replacing thinking about computers, there will be no one to really stop them. We'll equalize the human and computer knowledge by making humans less knowledgeable rather than more. So you end up in an Idiocracy-like scenario where a doctor can't diagnose you, nor can the machine because it was dumbed down by each successive generation, until it resembles a child's toy.

deleted 9mo ago

[deleted]

bgirard 9mo ago

This to me sounds a lot like the SpaceX conversation: - Ohh look it can [write small function / do a small rocket hop] but it can't [ write a compiler / get to orbit]! - Ohh look it can [write a toy compiler / get to orbit] but it can't [compile linux / be reusable] - Ohh look it can [compile linux / get reusable orbital rocket] but it can't [build a compiler that rivals GCC / turn the rockets around fast enough] - <Denial despite the insane rate of progress> There's no reason to keep building this compiler just to prove this point. But I bet it would catch up real fast to GCC with a fraction of the resources if it was guided by a few compiler engineers in the loop. We're going to see a lot of disruption come from AI assisted development.

raincole 9mo ago
> the insane rate of progress Yeah but the speed of progress can never catch the speed of a moving goalpost!
- friendzis 9mo ago
  > Yeah but the speed of progress can never catch the speed of a moving goalpost! How do you like those coast-to-coast self drives since the end of 2017?
- wrxd 9mo ago
  What about the hype? If you claim your LLM generated compiler is functionally on par with GCC I’d expect it to match your claim. I still won’t use it while it also matches all the non-functional requirements but you’re free to go and recompile all the software you use with it.
- samultio 9mo ago
  Training data only teaches it how to reach the goalpost, not how to overtake it.
  - codethief 9mo ago
    Are we sure about that? I mean, we have seen that LLMs are able to generalize to some degree. So I don't see a reason why you couldn't put an agent in a loop with a profiler and have it try to optimize the code. Will it come up with entirely novel ideas? Unlikely. Could it potentially combine existing ideas in interesting, novel ways that would lead to CCC outperforming GCC? I think so. Will it get stuck along the way? Almost certainly.

dataflow 9mo ago

> Pro: Sure, maybe now, but the next generation will fix it. Do we need a c2 wiki page for "sufficiently smart LLM" like we do for https://wiki.c2.com/?SufficientlySmartCompiler https://wiki.c2.com/?SufficientlySmartCompiler ?

sungho_ 9mo ago
> This is still true - a compiler can never win this battle. All a human programmer has to do is take the output of the compiler and make a single optimisation, and he/she wins. This is the advantage that the human has - they can use any of a wide variety of tools at their disposal (including the compiler), whilst the compiler can only do what it was programmed. The best the compiler can hope for is a tie. This is awkward

Alupis 9mo ago

I'm reminded, once again, of the recent "vibe coded" OCaml fiasco[1]. The PR author had zero understanding why their entirely LLM-generated contribution was viewed so suspiciously. The article validates a significant point: it is one thing to have passing tests and be able to produce output that resembles correctness - however it's something entirely different for that output to be good and maintainable. [1] https://github.com/ocaml/ocaml/pull/14369 https://github.com/ocaml/ocaml/pull/14369

Culonavirus 9mo ago
Damn... "AI has a very deep understanding of how this code works. Please challenge me on this." this person is something else. Just... wow.
- copilot_king_2 9mo ago
  This is the norm in my experience. If you haven't come across a significant number of AI addicts as obnoxiously delusional as @Culonavirus describes, you must be getting close to retirement age. People with any connection to new college graduates understand that this sort of idiotic LLM-backed arrogance is extremely common among low-to-mid-functioning twenty-somethings.
- co_king_3 9mo ago
  [flagged]
  - monegator 9mo ago
    *grifters
    - co_king_3 9mo ago
      Successful grifters are not this stupid. Terminal AI users are genuinely *slow* individuals. These people are functionally illiterate; they are not able to extract information by reading a bit of text in the same ways that you or I are. These people are very, very *gullible*.

gignico 9mo ago

Exactly. This flawed argument by which everything will be fixed by future models drives me crazy every time.

bopbopbop7 9mo ago
Just a couple more trillion and 6 more months!
deleted 9mo ago
[deleted]
simianwords 9mo ago
That’s been the trend for a while. Can you make a prediction that says something concretely like “AI will not be able to do X by 2028” for a specific and well defined X?
- direwolf20 9mo ago
  Were any made about 2025?
- xigoi 9mo ago
  In 2030, an AI model that I can run on my computer, without having to trust an evil megacorporation, will not be able to write a compiler for my markup language [0] based on a corpus of examples, without seeing the original implementation, using no more than 1.5× as much code as I did. https://git.sr.ht/~xigoi/hilda https://git.sr.ht/~xigoi/hilda
Kiro 9mo ago
So far it has been accurate though. Models have gotten much better than even the most optimistic predictions.
- LinXitoW 9mo ago
  No? The most optimistic predictions involved AGI around the corner, 6 months until no more developers for years now.

red75prime 9mo ago

> Pro-LLM coding agents: look! a working compiler built in a few hours by an agent! this is amazing! > Anti-LLM coding agents: it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. Pro-LLM: Read the freaking article, it's not that long. The compiler made a mistake in an area where only two compilers exist that are up to the task: Linux Kernel.

wrxd 9mo ago
Anti-LLM: isn’t all this intelligence supposed to give us something better than what we already have?
- red75prime 9mo ago
  Me: Top 0.02%[1] human-level intelligence? Sure. But we aren't there yet. [1] There are around 8k programming languages that are used (or were used) in practice (that is, they were deemed better than existing ones in some aspects) and there are around 50 million programmers. I use it to estimate how many people did something, which is objectively better than existing products.
marcus_holmes 9mo ago
Anthropic said they vibe-coded a C compiler that could compile the Linux kernel. That's what they said. No-one forced them to say that. They could have picked another code base. It turns out that isn't true in all instances, as this article demonstrates. I'm not nearly expert enough to be able to decide if that error was simple, stupid, irrelevant, or whatever. I can make a call on whether it successfully compiled the Linux kernel: it did not.
- red75prime 9mo ago
  I'm sorry for being excessively edgy, but "it's useless" is not a good summary for "linking errors after successfully compiling Linux kernel for x86_64."
  - 9mo ago

rk06 9mo ago

As an Anti, my argument is "if AI will good in future, then come back in the future"

aurareturn 9mo ago
As a pro, my argument is "it's good enough now to make me incredibly productive, and it's only going to keep getting better because of advancements in compute". I'd rather get really good at leveraging AI now than to bury my head in the sand hoping this will go away. I happen to agree with the saying that AI isn't going to replace people, but people using AI will replace people who don't. So by the time you come back in the future, you might have been replaced already.
- player1234 9mo ago
  [dead]
- techpression 9mo ago
  Why would anything you learn today be relevant tomorrow if AI keeps advancing? You would need less and less of all your tooling, markdown files and other rituals and just let the AI figure it out altogether.
  - aurareturn 9mo ago
    So I can keep my job now so I can pay for compute in the future when I'm out of a job. The compute will be used to create my own business to make money.
    - techpression 9mo ago
      Those are two different things though, and not everyone are stuck at a place enforcing token usage. And why would anyone pay you for something if all it takes is compute to make it? They would just make it themselves.
    - layer8 9mo ago

abrbhat 9mo ago

It seems that the cause of the difference in opinion is that the anti camp is looking at the current state while the pro camp looking at the slope and projecting it into the future.

quaintdev 9mo ago

I read a Youtube comment recently on pro AI video, it was "The source code of gcc is available online"

yoyohello13 9mo ago

I think LLMs the technology is very cool and l’m frankly amazed at what it can do. What I’m ‘anti’ about is pushing the entire economy all in on LLM tech. The accelerationist take of ‘just keep going as fast as possible and it will work out, trust me bro’ is the most unhinged dangerous shit I’ve ever heard and unfortunately seems to be the default worldview of those in charge of the money. I’m not sure where all the AI tools will end up, but I am willing to bet big that the average person is not going to be better off 10 years from now. The direction the world is going scares the shit out of me and the usages of AI by bad actors is not helping assuage that fear. Honestly? I think if we as a society could trust our leaders (government and industry) to not be total dirtbags the resistance to AI would be much lower. Like imagine if the message was “hey, this will lead to unemployment, but we are going to make sure people can still feed their families during the transition, maybe look in to ways to support subsidies retraining programs for people whose jobs have been impacted .” Seems like a much more palatable narrative than, “fuck you pleb! go retrain as a plumber or die in a ditch. I’ll be on my private island counting the money I made from destroying your livelihood.”

aurareturn 9mo ago

I don't think this is how pro and anti conversation goes. I think the pro would tell you that if GCC developers could leverage Opus 4.6, they'd be more productive. The anti would tell you that it doesn't help with productivity, it makes us less versed in the code base. I think the CCC project was just a demonstration on what Opus can do now autonomously. 99.9% of software projects out there aren't building something as complex as a Linux compiler.

Rapzid 9mo ago

Two completely valid perspectives. Unless you need a correctly compiled Linux kernel. In that case one gets exhausting real quick.

Rygian 9mo ago
The perspective that says "a whole compiler in just a few hours" is making false claims. So not a valid perspective.

nikitau 9mo ago

And not to mention that a C compiler is something we have literally 50 years worth of code for. I still seriously doubt the ability of LLMs to tackle truly new problems.

simianwords 9mo ago
What do you classify as new? Every problem that we solve as developers is a very small deviation from already existing problems. Maybe that’s the point of llms? How many developers do you think are solving truly novel problems? Most like me are CRUD bunnies.
- 1718627440 9mo ago
  And these developers do not write the majority of their codebase, they use tons of libraries and only write the glue code.
- xigoi 9mo ago
  If your problem is a very small deviation from an existing problem, you should be able to take an existing open-source solution and make a very small modification to adapt it to your use case. No need for “vibe-coding” a lower-quality implementation from scratch.
  - palmotea 9mo ago
    Yeah, it kind of strikes me how a lot of the LLM use cases would actually be better served by existing techniques, like more/better libraries. And if that's not possible, it'd be way better to find the closest match, fork it, and make minimal modifications. At least then you have the benefit of an upstream. But, sort of like cryptocurrency, the LLM people aren't so much trying to solve actual problems, but rather find an application of their existing technology. Sort of like the proverbial saying: when you're selling hammers, you want convince everyone that their problem as a nail.

frizlab 9mo ago

I think you also forgot: Anti: But the whole thing can only have been generated because GCC and other compilers already exists (and depending on how strong the anti-feeling is: and has been stolen…)!

_23sd 9mo ago

What does this imagined conversation have to do with the linked article? The “pro” and “anti” character both sound like the kind of insufferable idiots I’d expect to encounter on social media, the OP is a very nice blog post about performance testing and finding out what compilers do, doesn’t attempt any unwarranted speculation about what agents “struggle with” or will do “next generation”, how is it an example of that sort of shitposting?

abbyprog 9mo ago

I'm firmly in the anti/unimpressed camp so far - but of course open to see where it goes. I mean this compiler is the equivalent of handing someone a calculator when it was first invented and seeing that it took 2 hours to multiply two numbers together, I would go "cool that you have a machine that can do math, but I can multiply faster by hand, so it's a useless device to me".

_zagj 9mo ago

The "Anti" stance is only tenable now if you believe LLMs are going to hit a major roadblock in the next few months around which Big AI won't be able to navigate. Something akin to the various "ghosts in the machine" that started bedeviling EEs after 2000 when transistors got sufficiently small, including gate leakage and sub-threshold current, such that Dennard Scaling came to an abrupt end and clock speeds stalled. I personally hope that that happens, but I doubt it will. Note also that processors still continued to improve even without Dennard Scaling due to denser, better optimized onboard caches, better branch prediction, and more parallelism (including at the instruction level), and the broader trend towards SoCs and away from PCB-based systems, among other things. So at least by analogy, it's not impossible that even with that conjectured roadblock, Big AI could still find room for improvement, just at a much slower rate. But current LLMs are thoroughly compelling, and even just continued incremental improvements will prove massively disruptive to society.

weli 9mo ago

> this is all true, and yes, this is interesting. But there are so many other questions around this tech. Let's not rush into it and mess everything up. That's a really nice fictitious conversation but in my experience "anti-ai" people would be prone to say "This is stupid LLM's will never be able to write complex code and attempting to do so is futile". If your mind is open to explore how LLM's will actually write complex software then by definition you are not "anti".

viraptor 9mo ago

That's such a strawman conversation. Starting from: > it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. It works. It's not perfect, but anthropic claims to have successfully compiled and booted 3 different configurations with it. The blog post failed to reproduce one specific version on one specific architecture. I wish anthropic gave us more information about which kernel commits succeeded, but still. Compare this to years that it took for clang to compile the kernel, yet people were not calling that compiler useless. If anyone thinks other compilers "just work", I invite them to start fixing packages that fail to build in nixos after every major compiler change, to get a dose of real world experience.

agumonkey 9mo ago

This is a pattern I see a lot, in programming languages communities too, where it's a source a joy and dreams first and facts later.

nineteen999 9mo ago

I mean - who would honestly expect an LLM to be able to compete with a compiler with 40 years of development behind it? Even more if you count the collective man years expended in that time. The Claude agents took two weeks to produce a substandard compiler, under the fairly tight direction of a human who understood the problem space. At the same time - you could direct Claude to review the register spilling code and the linker code of both LLVM/gcc for potential improvements to CCC and you will see improvements. You can ask it not to copy GPL code verbatim but to paraphrase and tell it it can rip code from LLVM as long as the licenses are preserved. It will do it. You might only see marginal improvements without spending another $100K on API calls. This is about one of the hardest projects you could ask it to bite off and chew on. And would you trust the compiler output yet over GCC or LLVM? Of course not. But I wager, that if you _started_ with the LLVM/gcc codebases and asked it to look for improvements - it might be surprising to see what it finds. Both sides have good arguments. But this could be a totally different ball game in 2, 5 and 10 years. I do feel like those who are most terrified by it are those whose identity is very much tied to being a programmer, and seeing the potential for their role to be replaced and I can understand that. Me personally - I'm relieved I finally have someone else to blame and shout at rather than myself for the bugs in the software I produce. I'm relieved that I can focus now on the more creative direction and design of my personal projects (and even some work projects on the non-critical paths) and not get bogged down in my own perfectionism with respect to every little component until reaching exhaustion and giving up. And I'm fascinated by the creativity of some of the projects I see that are taking the same mindset and approach. I was depressed by it at first. But as I've experimented more and more, I've come to enjoy seeing things that I couldn't ever have achieved even with 100 man years of my own come to fruition.

kaycey2022 9mo ago

Maybe Anthropic can sponsor a research team to polish this using just an agent. A lot of things can be learned from that exercise.

zozbot234 9mo ago

You didn't even mention that this vibe-coded toy compiler cost $20k in token spend. That's an insane amount of money for what this is.

layer8 9mo ago
It seems at least comparable to what you would have to pay a suitably competent developer to code up something similar.
- palmotea 9mo ago
  > It seems at least comparable to what you would have to pay a suitably competent developer to code up something similar. No. You could have someone of fiverr do it for $5. Download gcc, done.
  - layer8 9mo ago
    If only the Claude team knew that, they could have saved a lot of tokens.

sylware 9mo ago

You are missing the most important: "Pro": give me tons of money to keep going this endeavour. More seriously: if some LLM or other can _assist_ c++ to plain and simple C port...

63stack 9mo ago

This is spot on, you can find traces of this conversation in the original thread posted on HN as well, where people are proclaiming "yeah it doesn't work, but still impressive!" Reminds me so much of the people posting their problems about the tesla cybertruck and ending the post with "still love the truck though"

Wowfunhappy 9mo ago

> Pro: but it's only been a few years since we started using LLMs, and a year or so since agents. This is only the beginning! The billion dollar question is, can we get from 80% to 100%? Is this going to be a situation where that final gap is just insurmountable, or will the capabilities simply keep increasing?

palmotea 9mo ago

> Pro-LLM coding agents: look! a working compiler built in a few hours by an agent! this is amazing! > Anti-LLM coding agents: it's not a working compiler, though. And it doesn't matter how few hours it took, because it doesn't work. It's useless. Also, from the Anti-LLM perspective: did the coding agent actually build a working compiler, or just plagiarize prior art? C compilers are certainly part of the LLM's training set. That's relevant because the implication seems to be: "Look, the agent can successfully develop really advanced software!" when the reality may be that it can plagiarize existing advanced software, and will fall on its face if asked to do anything not already done before. A lot of propaganda and hype follows the pattern of presenting things in a way to creating misleading implications in the mind of the listener that the facts don't actually support.

bicepjai 9mo ago

This encompasses all the back-and-forth arguments I can think of. I would assume that proponents will eventually also mention AGI or ASI. :)

giancarlostoro 9mo ago

We could be colonizing Mars with Claude Code and there will always be some skeptic somewhere.