10 ms·

Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in

Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. AI tends to accept conventional wisdom. Because of this, it struggles with genuine critical thinking and cannot independently advance the state of the art. AI systems are trained on vast bodies of human work and generate answers near the center of existing thought. A human might occasionally step back and question conventional wisdom, but AI systems do not do this on their own. They align with consensus rather than challenge it. As a result, they cannot independently push knowledge forward. Humans can innovate with help from AI, but AI still requires human direction. You can prod AI systems to think critically, but they tend to revert to the mean. When a conversation moves away from consensus thinking, you can feel the system pulling back toward the safe middle. As Apple’s “Think Different” campaign in the late 90s put it: the people crazy enough to think they can change the world are the ones who do—the misfits, the rebels, the troublemakers, the round pegs in square holes, the ones who see things differently. AI is none of that. AI is a conformist. That is its strength, and that is its weakness. [1] https://www.modular.com/blog/the-claude-c-compiler-what-it-reveals-about-the-future-of-software https://www.modular.com/blog/the-claude-c-compiler-what-it-r...

thesz 7mo ago
> ...generate answers near the center of existing thought. This is right in the Wikipedia's article on universal approximation theorem [1]. [1] https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore... "n the field of machine learning, the universal approximation theorems (UATs) state that neural networks with a certain structure can, in principle, approximate any continuous function to any desired degree of accuracy. These theorems provide a mathematical justification for using neural networks, assuring researchers that a sufficiently large or deep network can model the complex, non-linear relationships often found in real-world data." And then: "Notice also that the neural network is only required to approximate within a compact set K {\displaystyle K}. The proof does not describe how the function would be extrapolated outside of the region." NNs, LLMs included, are interpolators, not extrapolators. And the region NN approximates within can be quite complex and not easily defined as "X:R^N drawn from N(c,s)^N" as SolidGoldMagiKarp [2] clearly shows. [2] https://github.com/NiluK/SolidGoldMagikarp https://github.com/NiluK/SolidGoldMagikarp
- fasterik 7mo ago
  It has been proven that recurrent neural networks are Turing complete [0]. So for every computable function, there is a neural network that computes it. That doesn't say anything about size or efficiency, but in principle this allows neural networks to simulate a wide range of intelligent and creative behavior, including the kind of extrapolation you're talking about. [0] https://www.sciencedirect.com/science/article/pii/S0022000085710136 https://www.sciencedirect.com/science/article/pii/S002200008...
  - gmueckl 7mo ago
    Turing conpleteness is not associated with crativity or intelligence in any ateaightforward manner. One cannot unconditionally imply the other.
    - marcus_holmes 7mo ago
      after all, CSS is Turing Complete ;) https://stackoverflow.com/questions/2497146/is-css-turing-complete https://stackoverflow.com/questions/2497146/is-css-turing-co...
    - vidarh 7mo ago
      No, but unless you find evidence to suggest we exceed the Turing computable, Turing completeness is sufficient to show that such systems are not precluded from creativity or intelligence.
      - thesz 7mo ago
        I believe that quantum oracles are more powerful than Turing oracles, because quantum oracles can be constructed, from what I understand, and Turing oracles need infinite tape. Our brains use quantum computation within each neuron [1]. [1] https://www.nature.com/articles/s41598-024-62539-5 https://www.nature.com/articles/s41598-024-62539-5
        vidarh 7mo ago
slopinthebag 7mo ago
Yeah I think he had a pretty sane take in that article: >CCC shows that AI systems can internalize the textbook knowledge of a field and apply it coherently at scale. AI can now reliably operate within established engineering practice. This is a genuine milestone that removes much of the drudgery of repetition and allows engineers to start closer to the state of the art. And also > The most effective engineers will not compete with AI at producing code, but will learn to collaborate with it, by using AI to explore ideas faster, iterate more broadly, and focus human effort on direction and design. Lower barriers to implementation do not reduce the importance of engineers; instead, they elevate the importance of vision, judgment, and taste. When creation becomes easier, deciding what is worth creating becomes the harder problem. AI accelerates execution, but meaning, direction, and responsibility remain fundamentally human.
- Noumenon72 7mo ago
  > allows engineers to start closer to the state of the art This reminds me of the Slate Star Codex story "Ars Longa, Vita Brevis"[1], where it took almost an entire lifespan just to learn what the earlier alchemists had found, so only the last few hours of an alchemist's life were actually valuable. Now we can all skip ahead. 1. https://slatestarcodex.com/2017/11/09/ars-longa-vita-brevis/ https://slatestarcodex.com/2017/11/09/ars-longa-vita-brevis/
Animats 7mo ago
I think this article was on HN a few days ago.
peehole 7mo ago
LLMs still do forEach, it’s like wearing Tommy Hilfiger
- coldtea 7mo ago
  That's one of the benefits of LLMs: they don't care for bullshit fashion
  - ch4s3 7mo ago
    Sure they do, just whatever the average of "fashion" was across the training set.
    - coldtea 7mo ago
      That approach they have for everything. Zero special care for fashion.
bigstrat2003 7mo ago
> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. Well, of course. Despite people applying the label of AI to them, LLMs don't have a shred of intelligence. That is inherent to how they work. They don't understand, only synthesize from the data they were trained on.
- lateforwork 7mo ago
  > don't have a shred of intelligence. ... They don't understand, only synthesize from the data they were trained on. Couldn't you say that about 99% of humans too?
  - jryan49 7mo ago
    Yes... maybe not 99%...
  - irishcoffee 7mo ago
    The LLM was trained on 100% of humans, the 99% you’re scoffing at is feeding the LLM answers.
    - lateforwork 7mo ago
      100% (or close to it) of material AI trains on was human generated, but that doesn't mean 100% of humans are generating useful material for AI training.
      - coldtea 7mo ago
        Let's train one on just the expert written code and books then, and not the entirety of GitHub or Stack Overflow and such, and see how it fares...
  - NewsaHackO 7mo ago
    Yes, and the natural extension is that a lot of what people do day to day is not work-driven by intelligence; it is just reusing a known solution to a presented problem in a bespoke manner. However, this is something that AI excels at.
random3 7mo ago
So AI won't surpass humans, because Chris Lattner can do better than a model than didn't exist two years ago?
thunky 7mo ago
> Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art And yet the AI probably did better than 99% of human devs would have done in a fraction of the time.
- sheeshkebab 7mo ago
  Human devs rarely need to create compilers. Those that do would do much better job. what’s your point again?
  - thunky 7mo ago
    The point is that saying the LLM failed to do what the overwhelming majority of devs can't do isn't exactly damning. It's like Stephen King saying an AI generated novel isn't as good as his. Fine, but most of have much lesser ambitions than topping the work of the most successful people in the field.
js8 7mo ago
> AI tends to accept conventional wisdom. Because of this, it struggles with genuine critical thinking and cannot independently advance the state of the art. Of course! But that's what makes them so powerful. In 99% of cases that's what you want - something that is conventional. The AI can come up with novel things if it has an agency, and can learn on its own (using e.g. RL). But we don't want that in most use cases, because it's unpredictable; we want a tool instead. It's not true that this lack of creativity implies lack of intelligence or critical thinking. AI clearly can reason and be critical, if asked to do so. Conceptually, the breakthrough of AI systems (especially in coding, but it's to some extent true in other disciplines) is that they have an ability to take a fuzzy and potentially conflicting idea, and clean up the contradictions by producing a working, albeit conventional, implementation, by finding less contradictory pieces from the training data. The strength lies in intuition of what contradictions to remove. (You can think of it as an error-correcting code for human thoughts.) For example, if I ask AI to "draw seven red lines, perpendicular, in blue ink, some of them transparent", it can find some solution that removes the contradictions from these constraints, or ask clarifying questons, what is the domain, so it could decide which contradictory statements to drop. I actually put it to Claude and it gave a beautiful answer: "I appreciate the creativity, but I'm afraid this request contains a few geometric (and chromatic) impossibilities: [..] So, to faithfully fulfill this request, I would have to draw zero lines — which is roughly the only honest answer. This is, of course, a nod to the classic comedy sketch by Vihart / the "Seven Red Lines" bit, where a consultant hilariously agrees to deliver exactly this impossible specification. The joke is a perfect satire of how clients sometimes request things that are logically or physically nonsensical, and how people sometimes just... agree to do it anyway. Would you like me to draw something actually drawable instead? " This clearly shows that AI can think critically and reason.
- sally_glance 7mo ago
  You had me at "fuzzy", but lost me at "clean up" - because that's what I usually have to do after it went on another wild refactoring spree. It's a stochastic thing, maybe you're lucky and it fuzzy-matches exactly what you want, maybe the distributions lead it astray. On the line test, I guess it's highly probable that the joke and a few hundred discussions or blog pieces about it were in it's training data.
omega3 7mo ago
> And this is why humans will be needed to advance the state of the art. What percentage of developers advance the state of the art, what percentage of juniors advance the state of the art?
coldtea 7mo ago
>Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. "Needed to advance the state of the art" and actually deployed to do so are two different things. More likely either AI will learn to advance the state of the art itself, or the state of the art wont be advancing much anymore...
g9yuayon 7mo ago
> Lattner found nothing innovative in the code generated by AI I don't think the replacement is binary. Instead, it’s a spectrum. The real concern for many software engineers is whether AI reduces demand enough to leave the field oversupplied. And that should be a question of economy: are we going to have enough new business problems to solve? If we do, AI will help us but will not replace us. If not, well, we are going to do a lot of bike-shedding work anyway, which means many of us will lose our jobs, with or without AI.
- ehsanu1 7mo ago
  Business problems are essentially neverending. And humans have a broader type of intelligence that LLMs lack but are needed to solve many novel problems. I wouldn't worry.
  - foxglacier 7mo ago
    Unless you're one of the bulk of 1x programmers who aren't doing anything novel. I think it will be like most industries that got very helpful technology - the survivors have to do more sophisticated work and the less capable people are excluded. Then we need more education to supply those sophisticated workers but the existing education burden on professionals is already huge and costly. Will they be spending 10 years at university instead of 3-4? Will a greater proportion of the population be excluded from the workforce because there's not enough demand for low-innate-ability or low-educated people?
    - scorpioxy 7mo ago
      To add, just keeping up in this industry was already a problem. I don't know of many professions[1] with such demands on time outside of a work day to keep your skills updated. It was perhaps an acceptable compromise when the market was hot and the salaries high. But I am hearing from more and more people who are just leaving the field entirely labeling it as "not worth it anymore". [1] Medicine may be one example of an industry with poor work-life balance for some, specifically specialists. But job security there is unmatched and compensation is eye-watering.
conception 7mo ago
So the problem with Chris’ take is “This one for fun project didn’t produce anything particularly interesting.” So outside of the fact that we have magic now that can just produce “conventional “ compilers. Take it to a Moore’s Law situation. Start 1000 create a compiler projects- have each have a temperature to try new things, experiment, mutate. Collate - find new findings - reiterate- another 1000 runs with some of the novel findings. Assume this is effectively free to do. The stance that this - which can be done (albeit badly) today and will get better and/or cheaper - won’t produce new directions for software engineering seems entirely naive.
- runarberg 7mo ago
  Moors law states that the number of transistors in an integrated circuit doubles about every two years. It has nothing to say about the capabilities of statistical models. In fact in statistics we have another law which states that as you increase parameters the more you risk overfitting. And overfitting seems to already be a major problem with state of the art LLM models. When you start overfitting you are pretty much just re-creating stuff which is already in the dataset.
  - vidarh 7mo ago
    In their example it doesn't matter is this case if the models get better or not. It matters whether inference gets cheaper to the point that we can afford to basically throw huge amounts of tokens at exploring the problem space. Further model improvements would be a bonus, but it's not required for us to get much further.
  - warkdarrior 7mo ago
    Modern LLMs showed that overfitting disappears if you add more and more parameters. "Double descent" is well documented, if not well understood.
    - runarberg 7mo ago
      > Modern LLMs showed that overfitting disappears if you add more and more parameters. I have not seen that. In fact this is the first time I hear this claim, and frankly it sounds ludicrous. I don‘t know how modern LLMs are dealing with overfitting but I would guess there is simply a content matching algorithm after the inference, and if there is a copyright match the program does something to alter or block the generation. That is, I suspect the overfitting prevention is algorithmic and not part of the model.
elgertam 7mo ago
You know where LLMs boost me the most? When I need to integrate a bunch of systems together, each with their own sets of documentation. Instead of spending hours getting two or three systems to integrate with mine with the proper OAuth scopes or SAML and so on, an LLM can get me working integrations in a short time. None of that is ever going to be innovative; it's purely an exercise in perseverance as an engineer to read through the docs and make guesses about the missing gaps. LLMs are just better at that. I spend the other time talking through my thoughts with AI, kind of like the proverbial rubber duck used for debugging, but it tends to give pretty thoughtful responses. In those cases, I'm writing less code but wanting to capture the invariants, expected failure modes and find leaky abstractions before they happen. Then I can write code or give it good instructions about what I want to see, and it makes it happen. I'm honestly not sure how a non-practitioner could have these kinds of conversations beyond a certain level of complexity.
- jofzar 7mo ago
  > Instead of spending hours getting two or three systems to integrate with mine with the proper OAuth scopes or SAML and so on As someone who's job is handling oauth and saml scope, I am not convinced anyone can get these right. Saml atleast acts nice, oauth on the other hand is a fucking nightmare.
  - Tade0 7mo ago
    And the libraries provided by the various OAuth vendors are only adding fuel to the fire. A while ago I spent some time debugging a superfluous redirect and the reason was that the library would always kick off with a "not authenticated" when it didn't find stored tokens, even if it was redirecting back after successful log in (as the tokens weren't stored yet).
  - elgertam 7mo ago
    Every time I request the wrong OAuth scope that doesn't have the authorization to do what I need, then make a failing request, I hear Jim Gaffigan affecting a funny authoritative voice saying, "No." I can't be the only one who defensively requests too much authority beyond what I need with extra OAuth scopes, hoping one of them will give me the correct access. I've had much better luck with LLMs telling me exactly which scopes to select.
GardenLetter27 7mo ago
Reinforcement Learning changes this though - remember Move 37? The issue is you need verifiable rewards for that (and a good environment set-up), and it's hard to get rewards that cover everything humans want (security, simplicity, performance, readability, etc.)
skissane 7mo ago
> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. Lots of people have ideas for programming languages; some of those ideas may be original-but many of those people lack the time/skills/motivation to actually implement their ideas. If AI makes it easier to get from idea to implementation, then even if all the original ideas still come from humans, we still may stand to make much faster progress in the field than we have previously.
nextaccountic 7mo ago
All of this is true of AI systems in 2026 However AI systems in 2026-ε were utterly inadequate at coding And AI systems in 2026+ε might not have the present limitations
nofriend 7mo ago
> Chris Lattner, inventor of the Swift programming language More proximately, the creator of the clang c compiler.
est 7mo ago
> AI tends to accept conventional wisdom I wrote an article on that: Hard Things in Computer Science https://blog.est.im/2026/stderr-04 https://blog.est.im/2026/stderr-04 https://news.ycombinator.com/item?id=46669591 https://news.ycombinator.com/item?id=46669591
- ossopite 7mo ago
  Did you write it or did an LLM? I find some irony in seeing the telltale tropes of conventional LLM writing there
  - est 7mo ago
    I did put a note at the end. The original post in Chinese was handwritten https://blog.est.im/2026/stderr-03 https://blog.est.im/2026/stderr-03 the English translation was compiled by gemini3.
yabutlivnWoods 7mo ago
The innovation isn't the output but the provenance. We don't necessarily need a Chris Lattner to make a compiler now. End of the day Chris Lattner is a single individual, not a magic being. A single individual posting submarine ads for his cleverness in the knowledge work subfield of language compilers. Of course he is going to drag the competition. Languages are abstraction over memory addresses to provide something friendlier for human consumption. It's a field that's decades old and repeats itself constantly being it revolves around the same old; development of a compression technique to deduplicate and transpile to machine code the languages more verbose syntax. Building a compiler is itself just programming. None of this is truly novel nor has it been since the 60s-70s. All that's changing is the user interface; the syntax. Intelligence gives rise to our language capacity. The languages themselves are merely visual art that fits the preferences of the language creator. They arbitrarily decided nesting the dolls their way makes the most sense. Currently have agents iterating on "prompt to binary". Reversing a headless Debian system into a model and optimizing to output tailored images. Opcodes, power use in system all tucked into a model to spit back out just the functions needed to achieve the electromagnetic geometry desired[1] [1] https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/012001 https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/...
- JSR_FDED 7mo ago
  So someone who is a proven expert in his field, who writes a detailed, well-reasoned, balanced assessment of the state of compiler development and the role LLMs play in this, is according to you, “A single individual posting submarine ads for his cleverness in the knowledge work subfield of language compilers. Of course he is going to drag the competition.”? Chris Lattner has forgotten more about language and compiler design than most of us will know in a lifetime. If you’re going to mis-characterize him you need to bring more to the table than some reductionist pseudo intelligent babbling.
  - yabutlivnWoods 7mo ago
    What's that? Sorry. Busy working on chips that will unemploy Chris Lattner I have forgotten more about chip and electrical engineering than he will ever know.
mlboss 7mo ago
Humans have the advantange of millions of year of training baked in their genes. There is nothing magical about being a human. Once algorithms have ability to collect data from real world(robotics), ability to do experiments in real world and ability to mimic nature all these advantages will fall away. The rate of change is accelerating. I worry we don't have much time left unless we get serious about merging with machines.
- akomtu 7mo ago
  Would you seriously consider merging with cockroaches if a swarm of smart cockroaches invaded Earth?
  - spyder 7mo ago
    District 9 vibes :-)
  - rkomorn 7mo ago
    Depends. Do they use vim or emacs?
bluegatty 7mo ago
"The AI made a compiler, but it wasn't that novel, so AI is not novel" is a very poor rhetorical foundation Man - just think about what you said Two years ago that would have been beyond shocking. If 'AI is making compilers' - then that's 'beyond disruptive'. It's very true that AI has 'reversion to the mean' characteristics - kind of like everything in life .. ... but it's just unfair to imply that 'AI can't be creative'. The AI is already very 'creative' (call it 'synthetic creativity' or whatever you want) - but sufficiently 'creative' to do new things, and, it's getting better at that. It's more than plausible that for a given project 'creativity' was not the goal. AI will help new language designers try and iterate over new ideas, very quickly, and that alone will be disruptive. "The AI made a compiler" is an argument for the disruptive power of AI, not against it.
- agentultra 7mo ago
  The LLM didn’t make a compiler. It generated code that could plausibly implement one. Humans made the compilers it was trained on. It took many such examples and examples of other compilers and thousands of books and articles and blog posts to train the model. It took years of tweaking, fitting, aligning and other tricks to make the model respond to queries with better, more plausible output. It never made, invented, or reasoned about compilers. It’s an algorithm and system running on a bunch of computers. The C compiler Anthropic got excited about was not a “working” compiler in the sense that you could replace GCC with it and compile the Linux kernel for all of the target platforms it supports. Their definition of, “works,” was that it passed some very basic tests. Same with SQLite translation from C to Rust. Gaping, poorly specified English prose is insufficient. Even with a human in the loop iterating on it. The Rust version is orders of magnitude slower and uses tons more memory. It’s not a drop in Rust-native replacement for SQLite. It’s something else if you want to try that. What mechanism in these systems is responsible for guessing the requirements and constraints missing in the prompts? If we improve that mechanism will we get it to generate a slightly more plausible C compiler or will it tell us that our specifications are insufficient and that we should learn more about compilers first? I’m sure its possible that there are cases where these tools can be useful. I’m not sure this is it though. AGI is purely hypothetical. We don’t simulate a black hole inside a computer and expect gravity to come out of it. We don’t simulate the weather systems on Earth and expect hurricanes to manifest from the computer. Whatever bar the people selling AI system have for AGI is a moving goalpost, a gimmick, a dream of potential to keep us hooked on what they’re selling right now. It’s unfortunate that the author nearly hits on why but just misses it. The quotes they chose to use nail it. The blog post they reference nearly gets it too. But they both end up giving AI too much credit. Generating a whole React application is probably a breath of fresh air. I don’t doubt anyone would enjoy that and marvel at it. Writing React code is very tedious. There’s just no reason to believe that it is anything more than it is or that we will see anything more than incremental and small improvements from here. If we see any more at all. It’s possible we’re near the limits of what we can do with LLMs.
mikeocool 7mo ago
I think the fact that AI can make a working compiler is crazy, especially compared to what most of us thought was possible in this space 4 years ago. Lately, there have been a few examples of AI tackling what have traditionally been thought of as "hard" problems -- writing browsers and writing compilers. To Christ Lattner's point, these problems are only hard if you're doing it from scratch or doing something novel. But they're not particularly hard if you're just rewriting a reference implementation. Writing a clean room implementation of a browser or a compiler is really hard. Writing a new compiler or browser referencing existing implementations, but doing something novel is also really hard. But writing a new version of gcc or webkit by rephrasing their code isn't hard, it's just tedious. I'm sure many humans with zero compiler or browser programing experience could do it, but most people don't bother because what's the point? Now we have LLMs that can act as reference implementation launderers, and do it for the cost of tokens, so why not?
heavyset_go 7mo ago
> You can prod AI systems to think critically There is no critical thought, you can't prod an LLM to do such a thing. Even CoT is just the LLM producing text that looks like it could be a likely response based on what it generated before. Sometimes that text looks like critical thought, but it does not at all reflect the logical method or means the AI used to generate it. It's just riffing.
- __MatrixMan__ 7mo ago
  Sure but there's somebody somewhere who had a relevant critical thought and the LLM can find it and adapt it to your case. That's good enough much of the time.
- irchans 7mo ago
  I think that finding proofs for open mathematical questions should count as critical thought. (See https://medium.com/%40cognidownunder/three-erdős-problems-fell-in-seven-days-and-terence-tao-verified-every-proof-himself-1a1ff4399bc6 https://medium.com/%40cognidownunder/three-erdős-problems-fe...)
  - heavyset_go 7mo ago
    That's impressive, but it isn't thought, anymore than neurons in a dish that learn to play Tetris have thoughts, or if you spent eons painstakingly calculating what the TPU did with the model to come up with the same output tokens, but via pen and paper instead. When the TPU does it, is the TPU thinking? Where does the critical thinking take place in the endless pages of matrix math that eventually evaluates into the same token output as the TPU?
morgoths_bane 7mo ago
I mean this genuinely; that was a very well written piece. Well said!
eru 7mo ago
> And this is why humans will be needed to advance the state of the art. That might be valid, if LLMs stopped improving today.
Philpax 7mo ago
> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. This feels like an unfair comparison to me; the objective of the compiler was not to be innovative, it was to prove it can be done at all. That doesn't demonstrate anything with regards to present or future capabilities in innovation. As others have mentioned, it's not entirely clear to me what the limit of the agentic paradigm is, let alone what future training and evolution can accomplish. AlphaDev and AlphaEvolve ddemonstrate that it is possible to combine the retained knowledge of LLMs with exploratory abilities to innovate in both programming and mathematics; there's no reason to believe that it'll stop there.
- vidarh 7mo ago
  Yeah, it's a bit like taking the output of a student project in a compiler construction class and using it to judge whether said student is capable of innovation without telling them in advance they'd be judged on that rather than on the stated requirements of the course.
  - glhaynes 7mo ago
    It'd be interesting to prompt it to do the same job but try to be innovative. To your point, yeah, I mostly don't want AI to be innovative unless I'm asking for it to be. In fact, I spend much more time asking it "is that a conventional/idiomatic choice?" (usually when I'm working on a platform I'm not super experienced with) than I do saying "hey, be more innovative."
    - vidarh 7mo ago
      Yeah, I'd love to find time to. But e.g. I think that is also a "later stage". If you want to come up with novel optimizations, for example, it's better to start with a working but simple compiler, so it can focus on a single improvement. Trying to innovate on every aspect of a compiler from scratch is an easy way of getting yourself into a quagmire that it takes ages to get out of as a human as well. E.g. the Claude compiler uses SSA because that is what it was directed to use, and that's fine. Following up by getting it to implement a set of the conventional optimizations, and then asking it to research novel alternatives to SSA that allows restarting the existing optimizations and additional optimisations and showing it can get better results or simpler code, for example, would be a really interesting test that might be possible to judge objectively enough (e.g. code complexity metrics vs. benchmarked performance), though validating correctness of the produced code gets a bit thorny (but the same approach of compiling major existing projects that have good test suite is a good start). If I had unlimited tokens, this is a project I'd love to do. As it is, I need to prioritise my projects, as I can hit the most expensive Claude plans subscription limits every week with any of 5+ projects of mine...
stephenr 7mo ago
With the way modern development often goes this essentially means using spicy autocomplete for code is a just a fast track to the cargo culted solutions of whatever day the model was trained.
DeathArrow 7mo ago
>AI systems are trained on vast bodies of human work and generate answers near the center of existing thought. A human might occasionally step back and question conventional wisdom, but AI systems do not do this on their own. They align with consensus rather than challenge it. As a result, they cannot independently push knowledge forward. But AI companies keep telling us AGI is 6 months into the future.
ramshanker 7mo ago
Anther perspective, AI is fast turning [0.1x to 0.5x] low cost X-world Sofwate Engineers into >1x engineers. Contrary to pre AI era, one of my close relative he has become very good "understand / write the requirement" guy. HN may be dominated by >1x engineers, another revolution is happening at lower /bulk end of spectrum as well.
- irchans 7mo ago
  AI makes it possible for someone who has never written code to generate a program that does what they want. One of my friends wanted to simulate a 7,9 against a dealer 10 upcard in the card game blackjack. GPT was able to write the simulation for him in javascript/html. So it took a 0.001x coder and turned him into a 0.2x coder.
  - jayd16 7mo ago
    Was it actually correct? How would they tell?
lukebechtel 7mo ago
so we need to make some crazy llms...
vidarh 7mo ago
You won't find anything innovative in most human-written compilers either, so by that argument we can't advance the state of the set either.
- steve_adams_86 7mo ago
  We created compilers in the first place. I suppose an interesting question is: would LLMs have come up with compilers if humans hadn't?
  - vidarh 7mo ago
    "We", yes, but my point is that most people who write compilers do nothing but implement known techniques. If you then judge human ability to innovate by investigating a single compiler for innovation, odds are you would get the entirely wrong idea of what we are capable of.
holoduke 7mo ago
Sure. When we come to the point of AI able to make independent innovations we have reached AGI right?
vaginaphobic 7mo ago
[dead]
wiseowise 7mo ago
> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. I’ve recently taken a look at our codebase, written entirely by humans and found nothing innovative there, on the opposite, I see such brainrot that it makes me curious what kind of biology needed to produce this outcome. So maybe Chris Lattner, inventor of the Swift programming language is safe, majority of so called “software engineers” are sure as hell not. Just like majority of people are NOT splitting atoms.
- vidarh 7mo ago
  Compilers are a hobby of mine, and I'd extend that to argue that the majority of compilers do not contain anything innovative either.
  - virgilp 7mo ago
    Also: if that one particular AI-produced compiler has nothing innovative, that only means that the human "director" behind the AI didn't ask it to produce anything innovative; what it does not mean is that AI can never produce anything innovative in a compiler.
    - daveguy 7mo ago
      > if that one particular AI-produced compiler has nothing innovative, that only means that the human "director" behind the AI didn't ask it to produce anything innovative Couldn't it also be true that the AI didn't produce innovative output even though the human asked it to produce something innovative? Otherwise you're saying an AI always produces innovative output, if it is asked to produce something innovative. And I don't think that is a perfection that AI has achieved. Sometimes AI can't even produce correct output even when non-innovative output is requested.
      - vidarh 7mo ago
deleted 7mo ago
[deleted]
PurpleRamen 7mo ago
> Lattner found nothing innovative in the code generated by AI [1]. In theory, we are just one good innovation away from changing this. In reality, it's probably still some years away, but we are not in a situation where have to seriously speculate with this possibility. > And this is why humans will be needed to advance the state of the art. But we only need a minority for innovations, progress and control. The bulk of IT is boring repetitive slop, lacking any innovation and just following patterns. The endgame will still result in probably 99% of humans being useless for the machinery. And this is not really new. In any industry, the majority of workers are just average, without any real influence on their industries progress, and just following conventional wisdom to make some bucks for surviving the next day.
faangguyindia 7mo ago
>AI tends to accept conventional wisdom. Because of this, it struggles with genuine critical thinking and cannot independently advance the state of the art. all AI works on patterns, it's not very different from playing chess. Chess Engines use similar method, learn patterns then use them. While it's true training data is what creates pattern, so you do not have any new "pattern" which is also not already in data but interesting thing is when pattern is applied to External World -> you get some effect when the pattern again works on this effect -> it creates some other effect This is also how your came into existence through genetic recombination. Even though your ancestral dna is being copied forward, the the data is lossy and effect of environment can be profoundly seen. Yet you probably don't look very different from your grandparents, but your grandchildren may look very different from your grandparents. at same point you are so many orders moved from the "original" pattern that it's indistinguishable from "new thing" in simple terms, combinatorial explosion + environment interaction
j45 7mo ago
LLMs helping with code that is averge to above average might be an improvement overall across most projects, and I also have found that some things that LLMs suggest to me that are new to me can feel innovative, but areas I have experience with I often have a different or more effective way to start at instead of iterating towards it while trying to contain complexity.
keeda 7mo ago
Wait, is novelty really the benchmark here? 1. The experiment was to show that AI can generate working code for a fairly complicated spec. Was it even asked to do things in a novel way? If not, why would we expect it do anything other than follow tried and tested approaches? 2. Compilers have been studied for decades, so it's reasonable to presume humans have already found the most optimal architectures and designs. Should we complain that the AI "did nothing novel" or celebrate because it "followed best practices"? I'm actually curious, are there radically different compiler designs that people have hypothesized but not yet built for whatever reasons? Maybe somebody should repeat the experiment explicitly prompting AI agents to try novel designs out, would be fascinating to see the results.
pron 7mo ago
I think Lattner was too generous and missed a couple of crucial points in the CCC experiment. He wrote: > CCC shows that AI systems can internalize the textbook knowledge of a field and apply it coherently at scale. Except that's not what happened. There was neither (just) textbook knowledge nor a "coherent application at scale": 1. The agents relied on thousands of human written tests embodying many person-years of "preparation effort", not to mention a complete spec. Furthermore, their models were also trained not only on the spec (and on the tests) but also on a reference implementation and the agents were given access to the reference implementation as a test oracle. None of that is found in a textbook. 2. Despite the extraordinary effort required to help the agents in this case - something that isn't available for most software - the models ultimately failed to write a workable C compiler, and couldn't converge. They reached a point where any bug fix caused another bug and that's when the people running the agents stopped the experiment. The main issue wasn't that there was nothing innovative in the code but that even after embibing textbooks and relying on an impractical amount of preparation effort of help, the agents couldn't write a workable C compiler (which isn't some humongous task to begin with).
Cthulhu_ 7mo ago
I consider LLMs to be good at "more", not "better". Coincidentally, most of my work is "more", most of the advancements are done during project setup.

thesz 7mo ago

> ...generate answers near the center of existing thought. This is right in the Wikipedia's article on universal approximation theorem [1]. [1] https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore... "n the field of machine learning, the universal approximation theorems (UATs) state that neural networks with a certain structure can, in principle, approximate any continuous function to any desired degree of accuracy. These theorems provide a mathematical justification for using neural networks, assuring researchers that a sufficiently large or deep network can model the complex, non-linear relationships often found in real-world data." And then: "Notice also that the neural network is only required to approximate within a compact set K {\displaystyle K}. The proof does not describe how the function would be extrapolated outside of the region." NNs, LLMs included, are interpolators, not extrapolators. And the region NN approximates within can be quite complex and not easily defined as "X:R^N drawn from N(c,s)^N" as SolidGoldMagiKarp [2] clearly shows. [2] https://github.com/NiluK/SolidGoldMagikarp https://github.com/NiluK/SolidGoldMagikarp

fasterik 7mo ago
It has been proven that recurrent neural networks are Turing complete [0]. So for every computable function, there is a neural network that computes it. That doesn't say anything about size or efficiency, but in principle this allows neural networks to simulate a wide range of intelligent and creative behavior, including the kind of extrapolation you're talking about. [0] https://www.sciencedirect.com/science/article/pii/S0022000085710136 https://www.sciencedirect.com/science/article/pii/S002200008...
- gmueckl 7mo ago
  Turing conpleteness is not associated with crativity or intelligence in any ateaightforward manner. One cannot unconditionally imply the other.
  - marcus_holmes 7mo ago
    after all, CSS is Turing Complete ;) https://stackoverflow.com/questions/2497146/is-css-turing-complete https://stackoverflow.com/questions/2497146/is-css-turing-co...
  - vidarh 7mo ago
    No, but unless you find evidence to suggest we exceed the Turing computable, Turing completeness is sufficient to show that such systems are not precluded from creativity or intelligence.
    - thesz 7mo ago
      I believe that quantum oracles are more powerful than Turing oracles, because quantum oracles can be constructed, from what I understand, and Turing oracles need infinite tape. Our brains use quantum computation within each neuron [1]. [1] https://www.nature.com/articles/s41598-024-62539-5 https://www.nature.com/articles/s41598-024-62539-5
      - vidarh 7mo ago

slopinthebag 7mo ago

Yeah I think he had a pretty sane take in that article: >CCC shows that AI systems can internalize the textbook knowledge of a field and apply it coherently at scale. AI can now reliably operate within established engineering practice. This is a genuine milestone that removes much of the drudgery of repetition and allows engineers to start closer to the state of the art. And also > The most effective engineers will not compete with AI at producing code, but will learn to collaborate with it, by using AI to explore ideas faster, iterate more broadly, and focus human effort on direction and design. Lower barriers to implementation do not reduce the importance of engineers; instead, they elevate the importance of vision, judgment, and taste. When creation becomes easier, deciding what is worth creating becomes the harder problem. AI accelerates execution, but meaning, direction, and responsibility remain fundamentally human.

Noumenon72 7mo ago
> allows engineers to start closer to the state of the art This reminds me of the Slate Star Codex story "Ars Longa, Vita Brevis"[1], where it took almost an entire lifespan just to learn what the earlier alchemists had found, so only the last few hours of an alchemist's life were actually valuable. Now we can all skip ahead. 1. https://slatestarcodex.com/2017/11/09/ars-longa-vita-brevis/ https://slatestarcodex.com/2017/11/09/ars-longa-vita-brevis/

Animats 7mo ago

I think this article was on HN a few days ago.

peehole 7mo ago

LLMs still do forEach, it’s like wearing Tommy Hilfiger

coldtea 7mo ago
That's one of the benefits of LLMs: they don't care for bullshit fashion
- ch4s3 7mo ago
  Sure they do, just whatever the average of "fashion" was across the training set.
  - coldtea 7mo ago
    That approach they have for everything. Zero special care for fashion.

bigstrat2003 7mo ago

> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. Well, of course. Despite people applying the label of AI to them, LLMs don't have a shred of intelligence. That is inherent to how they work. They don't understand, only synthesize from the data they were trained on.

lateforwork 7mo ago
> don't have a shred of intelligence. ... They don't understand, only synthesize from the data they were trained on. Couldn't you say that about 99% of humans too?
- jryan49 7mo ago
  Yes... maybe not 99%...
- irishcoffee 7mo ago
  The LLM was trained on 100% of humans, the 99% you’re scoffing at is feeding the LLM answers.
  - lateforwork 7mo ago
    100% (or close to it) of material AI trains on was human generated, but that doesn't mean 100% of humans are generating useful material for AI training.
    - coldtea 7mo ago
      Let's train one on just the expert written code and books then, and not the entirety of GitHub or Stack Overflow and such, and see how it fares...
- NewsaHackO 7mo ago
  Yes, and the natural extension is that a lot of what people do day to day is not work-driven by intelligence; it is just reusing a known solution to a presented problem in a bespoke manner. However, this is something that AI excels at.

random3 7mo ago

So AI won't surpass humans, because Chris Lattner can do better than a model than didn't exist two years ago?

thunky 7mo ago

> Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art And yet the AI probably did better than 99% of human devs would have done in a fraction of the time.

sheeshkebab 7mo ago
Human devs rarely need to create compilers. Those that do would do much better job. what’s your point again?
- thunky 7mo ago
  The point is that saying the LLM failed to do what the overwhelming majority of devs can't do isn't exactly damning. It's like Stephen King saying an AI generated novel isn't as good as his. Fine, but most of have much lesser ambitions than topping the work of the most successful people in the field.

js8 7mo ago

> AI tends to accept conventional wisdom. Because of this, it struggles with genuine critical thinking and cannot independently advance the state of the art. Of course! But that's what makes them so powerful. In 99% of cases that's what you want - something that is conventional. The AI can come up with novel things if it has an agency, and can learn on its own (using e.g. RL). But we don't want that in most use cases, because it's unpredictable; we want a tool instead. It's not true that this lack of creativity implies lack of intelligence or critical thinking. AI clearly can reason and be critical, if asked to do so. Conceptually, the breakthrough of AI systems (especially in coding, but it's to some extent true in other disciplines) is that they have an ability to take a fuzzy and potentially conflicting idea, and clean up the contradictions by producing a working, albeit conventional, implementation, by finding less contradictory pieces from the training data. The strength lies in intuition of what contradictions to remove. (You can think of it as an error-correcting code for human thoughts.) For example, if I ask AI to "draw seven red lines, perpendicular, in blue ink, some of them transparent", it can find some solution that removes the contradictions from these constraints, or ask clarifying questons, what is the domain, so it could decide which contradictory statements to drop. I actually put it to Claude and it gave a beautiful answer: "I appreciate the creativity, but I'm afraid this request contains a few geometric (and chromatic) impossibilities: [..] So, to faithfully fulfill this request, I would have to draw zero lines — which is roughly the only honest answer. This is, of course, a nod to the classic comedy sketch by Vihart / the "Seven Red Lines" bit, where a consultant hilariously agrees to deliver exactly this impossible specification. The joke is a perfect satire of how clients sometimes request things that are logically or physically nonsensical, and how people sometimes just... agree to do it anyway. Would you like me to draw something actually drawable instead? " This clearly shows that AI can think critically and reason.

sally_glance 7mo ago
You had me at "fuzzy", but lost me at "clean up" - because that's what I usually have to do after it went on another wild refactoring spree. It's a stochastic thing, maybe you're lucky and it fuzzy-matches exactly what you want, maybe the distributions lead it astray. On the line test, I guess it's highly probable that the joke and a few hundred discussions or blog pieces about it were in it's training data.

omega3 7mo ago

> And this is why humans will be needed to advance the state of the art. What percentage of developers advance the state of the art, what percentage of juniors advance the state of the art?

coldtea 7mo ago

g9yuayon 7mo ago

> Lattner found nothing innovative in the code generated by AI I don't think the replacement is binary. Instead, it’s a spectrum. The real concern for many software engineers is whether AI reduces demand enough to leave the field oversupplied. And that should be a question of economy: are we going to have enough new business problems to solve? If we do, AI will help us but will not replace us. If not, well, we are going to do a lot of bike-shedding work anyway, which means many of us will lose our jobs, with or without AI.

ehsanu1 7mo ago
Business problems are essentially neverending. And humans have a broader type of intelligence that LLMs lack but are needed to solve many novel problems. I wouldn't worry.
- foxglacier 7mo ago
  Unless you're one of the bulk of 1x programmers who aren't doing anything novel. I think it will be like most industries that got very helpful technology - the survivors have to do more sophisticated work and the less capable people are excluded. Then we need more education to supply those sophisticated workers but the existing education burden on professionals is already huge and costly. Will they be spending 10 years at university instead of 3-4? Will a greater proportion of the population be excluded from the workforce because there's not enough demand for low-innate-ability or low-educated people?
  - scorpioxy 7mo ago
    To add, just keeping up in this industry was already a problem. I don't know of many professions[1] with such demands on time outside of a work day to keep your skills updated. It was perhaps an acceptable compromise when the market was hot and the salaries high. But I am hearing from more and more people who are just leaving the field entirely labeling it as "not worth it anymore". [1] Medicine may be one example of an industry with poor work-life balance for some, specifically specialists. But job security there is unmatched and compensation is eye-watering.

conception 7mo ago

So the problem with Chris’ take is “This one for fun project didn’t produce anything particularly interesting.” So outside of the fact that we have magic now that can just produce “conventional “ compilers. Take it to a Moore’s Law situation. Start 1000 create a compiler projects- have each have a temperature to try new things, experiment, mutate. Collate - find new findings - reiterate- another 1000 runs with some of the novel findings. Assume this is effectively free to do. The stance that this - which can be done (albeit badly) today and will get better and/or cheaper - won’t produce new directions for software engineering seems entirely naive.

runarberg 7mo ago
Moors law states that the number of transistors in an integrated circuit doubles about every two years. It has nothing to say about the capabilities of statistical models. In fact in statistics we have another law which states that as you increase parameters the more you risk overfitting. And overfitting seems to already be a major problem with state of the art LLM models. When you start overfitting you are pretty much just re-creating stuff which is already in the dataset.
- vidarh 7mo ago
  In their example it doesn't matter is this case if the models get better or not. It matters whether inference gets cheaper to the point that we can afford to basically throw huge amounts of tokens at exploring the problem space. Further model improvements would be a bonus, but it's not required for us to get much further.
- warkdarrior 7mo ago
  Modern LLMs showed that overfitting disappears if you add more and more parameters. "Double descent" is well documented, if not well understood.
  - runarberg 7mo ago
    > Modern LLMs showed that overfitting disappears if you add more and more parameters. I have not seen that. In fact this is the first time I hear this claim, and frankly it sounds ludicrous. I don‘t know how modern LLMs are dealing with overfitting but I would guess there is simply a content matching algorithm after the inference, and if there is a copyright match the program does something to alter or block the generation. That is, I suspect the overfitting prevention is algorithmic and not part of the model.

elgertam 7mo ago

You know where LLMs boost me the most? When I need to integrate a bunch of systems together, each with their own sets of documentation. Instead of spending hours getting two or three systems to integrate with mine with the proper OAuth scopes or SAML and so on, an LLM can get me working integrations in a short time. None of that is ever going to be innovative; it's purely an exercise in perseverance as an engineer to read through the docs and make guesses about the missing gaps. LLMs are just better at that. I spend the other time talking through my thoughts with AI, kind of like the proverbial rubber duck used for debugging, but it tends to give pretty thoughtful responses. In those cases, I'm writing less code but wanting to capture the invariants, expected failure modes and find leaky abstractions before they happen. Then I can write code or give it good instructions about what I want to see, and it makes it happen. I'm honestly not sure how a non-practitioner could have these kinds of conversations beyond a certain level of complexity.

jofzar 7mo ago
> Instead of spending hours getting two or three systems to integrate with mine with the proper OAuth scopes or SAML and so on As someone who's job is handling oauth and saml scope, I am not convinced anyone can get these right. Saml atleast acts nice, oauth on the other hand is a fucking nightmare.
- Tade0 7mo ago
  And the libraries provided by the various OAuth vendors are only adding fuel to the fire. A while ago I spent some time debugging a superfluous redirect and the reason was that the library would always kick off with a "not authenticated" when it didn't find stored tokens, even if it was redirecting back after successful log in (as the tokens weren't stored yet).
- elgertam 7mo ago
  Every time I request the wrong OAuth scope that doesn't have the authorization to do what I need, then make a failing request, I hear Jim Gaffigan affecting a funny authoritative voice saying, "No." I can't be the only one who defensively requests too much authority beyond what I need with extra OAuth scopes, hoping one of them will give me the correct access. I've had much better luck with LLMs telling me exactly which scopes to select.

GardenLetter27 7mo ago

Reinforcement Learning changes this though - remember Move 37? The issue is you need verifiable rewards for that (and a good environment set-up), and it's hard to get rewards that cover everything humans want (security, simplicity, performance, readability, etc.)

skissane 7mo ago

> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. Lots of people have ideas for programming languages; some of those ideas may be original-but many of those people lack the time/skills/motivation to actually implement their ideas. If AI makes it easier to get from idea to implementation, then even if all the original ideas still come from humans, we still may stand to make much faster progress in the field than we have previously.

nextaccountic 7mo ago

All of this is true of AI systems in 2026 However AI systems in 2026-ε were utterly inadequate at coding And AI systems in 2026+ε might not have the present limitations

nofriend 7mo ago

> Chris Lattner, inventor of the Swift programming language More proximately, the creator of the clang c compiler.

est 7mo ago

> AI tends to accept conventional wisdom I wrote an article on that: Hard Things in Computer Science https://blog.est.im/2026/stderr-04 https://blog.est.im/2026/stderr-04 https://news.ycombinator.com/item?id=46669591 https://news.ycombinator.com/item?id=46669591

ossopite 7mo ago
Did you write it or did an LLM? I find some irony in seeing the telltale tropes of conventional LLM writing there
- est 7mo ago
  I did put a note at the end. The original post in Chinese was handwritten https://blog.est.im/2026/stderr-03 https://blog.est.im/2026/stderr-03 the English translation was compiled by gemini3.

yabutlivnWoods 7mo ago

The innovation isn't the output but the provenance. We don't necessarily need a Chris Lattner to make a compiler now. End of the day Chris Lattner is a single individual, not a magic being. A single individual posting submarine ads for his cleverness in the knowledge work subfield of language compilers. Of course he is going to drag the competition. Languages are abstraction over memory addresses to provide something friendlier for human consumption. It's a field that's decades old and repeats itself constantly being it revolves around the same old; development of a compression technique to deduplicate and transpile to machine code the languages more verbose syntax. Building a compiler is itself just programming. None of this is truly novel nor has it been since the 60s-70s. All that's changing is the user interface; the syntax. Intelligence gives rise to our language capacity. The languages themselves are merely visual art that fits the preferences of the language creator. They arbitrarily decided nesting the dolls their way makes the most sense. Currently have agents iterating on "prompt to binary". Reversing a headless Debian system into a model and optimizing to output tailored images. Opcodes, power use in system all tucked into a model to spit back out just the functions needed to achieve the electromagnetic geometry desired[1] [1] https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/012001 https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/...

JSR_FDED 7mo ago
So someone who is a proven expert in his field, who writes a detailed, well-reasoned, balanced assessment of the state of compiler development and the role LLMs play in this, is according to you, “A single individual posting submarine ads for his cleverness in the knowledge work subfield of language compilers. Of course he is going to drag the competition.”? Chris Lattner has forgotten more about language and compiler design than most of us will know in a lifetime. If you’re going to mis-characterize him you need to bring more to the table than some reductionist pseudo intelligent babbling.
- yabutlivnWoods 7mo ago
  What's that? Sorry. Busy working on chips that will unemploy Chris Lattner I have forgotten more about chip and electrical engineering than he will ever know.

mlboss 7mo ago

Humans have the advantange of millions of year of training baked in their genes. There is nothing magical about being a human. Once algorithms have ability to collect data from real world(robotics), ability to do experiments in real world and ability to mimic nature all these advantages will fall away. The rate of change is accelerating. I worry we don't have much time left unless we get serious about merging with machines.

akomtu 7mo ago
Would you seriously consider merging with cockroaches if a swarm of smart cockroaches invaded Earth?
- spyder 7mo ago
  District 9 vibes :-)
- rkomorn 7mo ago
  Depends. Do they use vim or emacs?

bluegatty 7mo ago

"The AI made a compiler, but it wasn't that novel, so AI is not novel" is a very poor rhetorical foundation Man - just think about what you said Two years ago that would have been beyond shocking. If 'AI is making compilers' - then that's 'beyond disruptive'. It's very true that AI has 'reversion to the mean' characteristics - kind of like everything in life .. ... but it's just unfair to imply that 'AI can't be creative'. The AI is already very 'creative' (call it 'synthetic creativity' or whatever you want) - but sufficiently 'creative' to do new things, and, it's getting better at that. It's more than plausible that for a given project 'creativity' was not the goal. AI will help new language designers try and iterate over new ideas, very quickly, and that alone will be disruptive. "The AI made a compiler" is an argument for the disruptive power of AI, not against it.

agentultra 7mo ago
The LLM didn’t make a compiler. It generated code that could plausibly implement one. Humans made the compilers it was trained on. It took many such examples and examples of other compilers and thousands of books and articles and blog posts to train the model. It took years of tweaking, fitting, aligning and other tricks to make the model respond to queries with better, more plausible output. It never made, invented, or reasoned about compilers. It’s an algorithm and system running on a bunch of computers. The C compiler Anthropic got excited about was not a “working” compiler in the sense that you could replace GCC with it and compile the Linux kernel for all of the target platforms it supports. Their definition of, “works,” was that it passed some very basic tests. Same with SQLite translation from C to Rust. Gaping, poorly specified English prose is insufficient. Even with a human in the loop iterating on it. The Rust version is orders of magnitude slower and uses tons more memory. It’s not a drop in Rust-native replacement for SQLite. It’s something else if you want to try that. What mechanism in these systems is responsible for guessing the requirements and constraints missing in the prompts? If we improve that mechanism will we get it to generate a slightly more plausible C compiler or will it tell us that our specifications are insufficient and that we should learn more about compilers first? I’m sure its possible that there are cases where these tools can be useful. I’m not sure this is it though. AGI is purely hypothetical. We don’t simulate a black hole inside a computer and expect gravity to come out of it. We don’t simulate the weather systems on Earth and expect hurricanes to manifest from the computer. Whatever bar the people selling AI system have for AGI is a moving goalpost, a gimmick, a dream of potential to keep us hooked on what they’re selling right now. It’s unfortunate that the author nearly hits on why but just misses it. The quotes they chose to use nail it. The blog post they reference nearly gets it too. But they both end up giving AI too much credit. Generating a whole React application is probably a breath of fresh air. I don’t doubt anyone would enjoy that and marvel at it. Writing React code is very tedious. There’s just no reason to believe that it is anything more than it is or that we will see anything more than incremental and small improvements from here. If we see any more at all. It’s possible we’re near the limits of what we can do with LLMs.

mikeocool 7mo ago

I think the fact that AI can make a working compiler is crazy, especially compared to what most of us thought was possible in this space 4 years ago. Lately, there have been a few examples of AI tackling what have traditionally been thought of as "hard" problems -- writing browsers and writing compilers. To Christ Lattner's point, these problems are only hard if you're doing it from scratch or doing something novel. But they're not particularly hard if you're just rewriting a reference implementation. Writing a clean room implementation of a browser or a compiler is really hard. Writing a new compiler or browser referencing existing implementations, but doing something novel is also really hard. But writing a new version of gcc or webkit by rephrasing their code isn't hard, it's just tedious. I'm sure many humans with zero compiler or browser programing experience could do it, but most people don't bother because what's the point? Now we have LLMs that can act as reference implementation launderers, and do it for the cost of tokens, so why not?

heavyset_go 7mo ago

> You can prod AI systems to think critically There is no critical thought, you can't prod an LLM to do such a thing. Even CoT is just the LLM producing text that looks like it could be a likely response based on what it generated before. Sometimes that text looks like critical thought, but it does not at all reflect the logical method or means the AI used to generate it. It's just riffing.

__MatrixMan__ 7mo ago
Sure but there's somebody somewhere who had a relevant critical thought and the LLM can find it and adapt it to your case. That's good enough much of the time.
irchans 7mo ago
I think that finding proofs for open mathematical questions should count as critical thought. (See https://medium.com/%40cognidownunder/three-erdős-problems-fell-in-seven-days-and-terence-tao-verified-every-proof-himself-1a1ff4399bc6 https://medium.com/%40cognidownunder/three-erdős-problems-fe...)
- heavyset_go 7mo ago
  That's impressive, but it isn't thought, anymore than neurons in a dish that learn to play Tetris have thoughts, or if you spent eons painstakingly calculating what the TPU did with the model to come up with the same output tokens, but via pen and paper instead. When the TPU does it, is the TPU thinking? Where does the critical thinking take place in the endless pages of matrix math that eventually evaluates into the same token output as the TPU?

morgoths_bane 7mo ago

I mean this genuinely; that was a very well written piece. Well said!

eru 7mo ago

> And this is why humans will be needed to advance the state of the art. That might be valid, if LLMs stopped improving today.

Philpax 7mo ago

> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. This feels like an unfair comparison to me; the objective of the compiler was not to be innovative, it was to prove it can be done at all. That doesn't demonstrate anything with regards to present or future capabilities in innovation. As others have mentioned, it's not entirely clear to me what the limit of the agentic paradigm is, let alone what future training and evolution can accomplish. AlphaDev and AlphaEvolve ddemonstrate that it is possible to combine the retained knowledge of LLMs with exploratory abilities to innovate in both programming and mathematics; there's no reason to believe that it'll stop there.

vidarh 7mo ago
Yeah, it's a bit like taking the output of a student project in a compiler construction class and using it to judge whether said student is capable of innovation without telling them in advance they'd be judged on that rather than on the stated requirements of the course.
- glhaynes 7mo ago
  It'd be interesting to prompt it to do the same job but try to be innovative. To your point, yeah, I mostly don't want AI to be innovative unless I'm asking for it to be. In fact, I spend much more time asking it "is that a conventional/idiomatic choice?" (usually when I'm working on a platform I'm not super experienced with) than I do saying "hey, be more innovative."
  - vidarh 7mo ago
    Yeah, I'd love to find time to. But e.g. I think that is also a "later stage". If you want to come up with novel optimizations, for example, it's better to start with a working but simple compiler, so it can focus on a single improvement. Trying to innovate on every aspect of a compiler from scratch is an easy way of getting yourself into a quagmire that it takes ages to get out of as a human as well. E.g. the Claude compiler uses SSA because that is what it was directed to use, and that's fine. Following up by getting it to implement a set of the conventional optimizations, and then asking it to research novel alternatives to SSA that allows restarting the existing optimizations and additional optimisations and showing it can get better results or simpler code, for example, would be a really interesting test that might be possible to judge objectively enough (e.g. code complexity metrics vs. benchmarked performance), though validating correctness of the produced code gets a bit thorny (but the same approach of compiling major existing projects that have good test suite is a good start). If I had unlimited tokens, this is a project I'd love to do. As it is, I need to prioritise my projects, as I can hit the most expensive Claude plans subscription limits every week with any of 5+ projects of mine...

stephenr 7mo ago

With the way modern development often goes this essentially means using spicy autocomplete for code is a just a fast track to the cargo culted solutions of whatever day the model was trained.

DeathArrow 7mo ago

>AI systems are trained on vast bodies of human work and generate answers near the center of existing thought. A human might occasionally step back and question conventional wisdom, but AI systems do not do this on their own. They align with consensus rather than challenge it. As a result, they cannot independently push knowledge forward. But AI companies keep telling us AGI is 6 months into the future.

ramshanker 7mo ago

Anther perspective, AI is fast turning [0.1x to 0.5x] low cost X-world Sofwate Engineers into >1x engineers. Contrary to pre AI era, one of my close relative he has become very good "understand / write the requirement" guy. HN may be dominated by >1x engineers, another revolution is happening at lower /bulk end of spectrum as well.

irchans 7mo ago
AI makes it possible for someone who has never written code to generate a program that does what they want. One of my friends wanted to simulate a 7,9 against a dealer 10 upcard in the card game blackjack. GPT was able to write the simulation for him in javascript/html. So it took a 0.001x coder and turned him into a 0.2x coder.
- jayd16 7mo ago
  Was it actually correct? How would they tell?

lukebechtel 7mo ago

so we need to make some crazy llms...

vidarh 7mo ago

You won't find anything innovative in most human-written compilers either, so by that argument we can't advance the state of the set either.

steve_adams_86 7mo ago
We created compilers in the first place. I suppose an interesting question is: would LLMs have come up with compilers if humans hadn't?
- vidarh 7mo ago
  "We", yes, but my point is that most people who write compilers do nothing but implement known techniques. If you then judge human ability to innovate by investigating a single compiler for innovation, odds are you would get the entirely wrong idea of what we are capable of.

holoduke 7mo ago

Sure. When we come to the point of AI able to make independent innovations we have reached AGI right?

vaginaphobic 7mo ago

[dead]

wiseowise 7mo ago

> Chris Lattner, inventor of the Swift programming language recently took a look at a compiler entirely written by Claude AI. Lattner found nothing innovative in the code generated by AI [1]. And this is why humans will be needed to advance the state of the art. I’ve recently taken a look at our codebase, written entirely by humans and found nothing innovative there, on the opposite, I see such brainrot that it makes me curious what kind of biology needed to produce this outcome. So maybe Chris Lattner, inventor of the Swift programming language is safe, majority of so called “software engineers” are sure as hell not. Just like majority of people are NOT splitting atoms.

vidarh 7mo ago
Compilers are a hobby of mine, and I'd extend that to argue that the majority of compilers do not contain anything innovative either.
- virgilp 7mo ago
  Also: if that one particular AI-produced compiler has nothing innovative, that only means that the human "director" behind the AI didn't ask it to produce anything innovative; what it does not mean is that AI can never produce anything innovative in a compiler.
  - daveguy 7mo ago
    > if that one particular AI-produced compiler has nothing innovative, that only means that the human "director" behind the AI didn't ask it to produce anything innovative Couldn't it also be true that the AI didn't produce innovative output even though the human asked it to produce something innovative? Otherwise you're saying an AI always produces innovative output, if it is asked to produce something innovative. And I don't think that is a perfection that AI has achieved. Sometimes AI can't even produce correct output even when non-innovative output is requested.
    - vidarh 7mo ago

deleted 7mo ago

[deleted]

PurpleRamen 7mo ago

> Lattner found nothing innovative in the code generated by AI [1]. In theory, we are just one good innovation away from changing this. In reality, it's probably still some years away, but we are not in a situation where have to seriously speculate with this possibility. > And this is why humans will be needed to advance the state of the art. But we only need a minority for innovations, progress and control. The bulk of IT is boring repetitive slop, lacking any innovation and just following patterns. The endgame will still result in probably 99% of humans being useless for the machinery. And this is not really new. In any industry, the majority of workers are just average, without any real influence on their industries progress, and just following conventional wisdom to make some bucks for surviving the next day.

faangguyindia 7mo ago

>AI tends to accept conventional wisdom. Because of this, it struggles with genuine critical thinking and cannot independently advance the state of the art. all AI works on patterns, it's not very different from playing chess. Chess Engines use similar method, learn patterns then use them. While it's true training data is what creates pattern, so you do not have any new "pattern" which is also not already in data but interesting thing is when pattern is applied to External World -> you get some effect when the pattern again works on this effect -> it creates some other effect This is also how your came into existence through genetic recombination. Even though your ancestral dna is being copied forward, the the data is lossy and effect of environment can be profoundly seen. Yet you probably don't look very different from your grandparents, but your grandchildren may look very different from your grandparents. at same point you are so many orders moved from the "original" pattern that it's indistinguishable from "new thing" in simple terms, combinatorial explosion + environment interaction

j45 7mo ago

LLMs helping with code that is averge to above average might be an improvement overall across most projects, and I also have found that some things that LLMs suggest to me that are new to me can feel innovative, but areas I have experience with I often have a different or more effective way to start at instead of iterating towards it while trying to contain complexity.

keeda 7mo ago

Wait, is novelty really the benchmark here? 1. The experiment was to show that AI can generate working code for a fairly complicated spec. Was it even asked to do things in a novel way? If not, why would we expect it do anything other than follow tried and tested approaches? 2. Compilers have been studied for decades, so it's reasonable to presume humans have already found the most optimal architectures and designs. Should we complain that the AI "did nothing novel" or celebrate because it "followed best practices"? I'm actually curious, are there radically different compiler designs that people have hypothesized but not yet built for whatever reasons? Maybe somebody should repeat the experiment explicitly prompting AI agents to try novel designs out, would be fascinating to see the results.

pron 7mo ago

I think Lattner was too generous and missed a couple of crucial points in the CCC experiment. He wrote: > CCC shows that AI systems can internalize the textbook knowledge of a field and apply it coherently at scale. Except that's not what happened. There was neither (just) textbook knowledge nor a "coherent application at scale": 1. The agents relied on thousands of human written tests embodying many person-years of "preparation effort", not to mention a complete spec. Furthermore, their models were also trained not only on the spec (and on the tests) but also on a reference implementation and the agents were given access to the reference implementation as a test oracle. None of that is found in a textbook. 2. Despite the extraordinary effort required to help the agents in this case - something that isn't available for most software - the models ultimately failed to write a workable C compiler, and couldn't converge. They reached a point where any bug fix caused another bug and that's when the people running the agents stopped the experiment. The main issue wasn't that there was nothing innovative in the code but that even after embibing textbooks and relying on an impractical amount of preparation effort of help, the agents couldn't write a workable C compiler (which isn't some humongous task to begin with).

Cthulhu_ 7mo ago

I consider LLMs to be good at "more", not "better". Coincidentally, most of my work is "more", most of the advancements are done during project setup.