8 ms·

AI should only run as fast as we can catch up

198 points by yuedongze 10mo ago

rogerkirkness 10mo ago
Appealing, but this is coming from someone smart/thoughtful. No offence to 'rest of world', but I think that most people have felt this way for years. And realistically in a year, there won't be any people who can keep up.
- airstrike 10mo ago
  > And realistically in a year, there won't be any people who can keep up. Bold claim. They said the same thing at the start of this year.
  - adventured 10mo ago
    You're all arguing over how many single digit years it'll take at this point. It doesn't matter if it takes another 12 or 36 months to make that claim true. It doesn't matter if it takes five years. Is AI coming for most of the software jobs? Yes it is. It's moving very quickly, and nothing can stop it. The progress has been particularly exceptionally clear (early GPT to Gemini 3 / Opus 4.5 / Codex).
    - bdangubic 10mo ago
      > Is AI coming for most of the software jobs? be cool to start with one before we move to most…
      - esafak 10mo ago
        https://news.ycombinator.com/item?id=46124063 https://news.ycombinator.com/item?id=46124063
- yuedongze 10mo ago
  im hoping this can introduce a framework to help people visualize the problem and figure out a way to close that gap. image generation is something every one can verify, but code generation is perhaps not. but if we can make verifying code as effortless as verifying images (not saying it's possible), then our productivity can enter the next level...
  - drlobster 10mo ago
    I think you underestimating how good these image generators are at the moment.
    - yuedongze 10mo ago
      oh i mean the other direction! checking if a generated image is "good" that no one will tell something is off and it look naturally, rather than checking if they are fake.
- dontlikeyoueith 10mo ago
  > And realistically in a year, there won't be any people who can keep up. I've heard the same claim every year since GPT-3. It's still just as irrational as it was then.
  - adventured 10mo ago
    You're rather dramatically demonstrating how remarkable the progress has been: GPT-3 was horrible at coding. Claude Opus 4.5 is good at it. They're already far faster than anybody on HN could ever be. Whether it takes another five years or ten, in that span of time nobody on HN will be able to keep up with the top tier models. It's not irrational, it's guaranteed. The progress has been extraordinary and obvious, the direction is certain, the outcome is certain. All that is left is to debate whether it's a couple of years or closer to a decade.
    - Arainach 10mo ago
      People claimed GPT-3 was great at coding when it launched. Those who said otherwise were dismissed. That has continued to be the case in every generation.
      - dwaltrip 10mo ago
        A bit reductive.
      - stale2002 10mo ago
        > People claimed GPT-3 was great at coding when it launched. Ok and they were wrong, but now people are right that it is great at coding. > That has continued to be the case in every generation. If something gets better over time, it is definitionally true that it was bad for every case in the past until it becomes good. But then it is good. Thats how that works. For everything. You are talking in tautologies while not understanding the implication of your arguments and how it applies to very general things like "A thing that improves over time".
      - esafak 10mo ago
gradus_ad 10mo ago
The proliferation of nondeterministically generated code is here to stay. Part of our response must be more dynamic, more comprehensive and more realistic workload simulation and testing frameworks.
- yuedongze 10mo ago
  i've seen a lot of startups that use AI to QA human work. how about the idea of use humans to QA AI work? a lot of interesting things might follow
  - Aldipower 10mo ago
    Sounds inhuman.
    - A4ET8a8uTh0_v2 10mo ago
      Nah, sounds like management, but I am repeating myself. In all seriousness, I have found myself having to carefully rein some of similar decisions in. I don't want to get into details, but there are times I wonder if they understand how things really work or if people need some 'floor' level exposure before they just decree stuff.
    - quantummagic 10mo ago
      As an industry, we've been doing the same thing to people in almost every other sector of the workforce, since we began. Automation is just starting to come for us now, and a lot of us are really pissed off about it. All of a sudden, we're humanitarians.
      - Terr_ 10mo ago
        > Automation is just starting to come for us now This argument is common and facile: Software development has always been about "automating ourselves out of a job", whether in the broad sense of creating compilers and IDEs, or in the individual sense that you write some code and say: "Hey, I don't want to rewrite this again later, not even if I was being paid for my time, I'll make it into a reusable library." > the same thing The reverse: What pisses me off is how what's coming is not the same thing. Customers are being sold a snake-oil product, and its adoption may well ruin things we've spent careers de-crappifying by making them consistent and repeatable and understandable. In the aftermath, some portion of my (continued) career will be diverted to cleaning up the lingering damage from it.
CGMthrowaway 10mo ago
> AI should only run as fast as we can catch up Good principle. This is exactly why we research vaccines and bioweapons side by side in the labs, for example.
yannyu 10mo ago
I think there's a lot of utility to current AI tools, but it's also clear we're in a very unsettled phase of this technology. We likely won't see for years where the technology lands in terms of capability or the changes that will be made to society and industry to accommodate. Somewhat unfortunately, the sheer amount of money being poured into AI means that it's being forced upon many of us, even if we didn't want it. Which results in a stark, vast gap like the author is describing, where things are moving so fast that it can feel like we may never have time to catch up. And what's even worse, because of this industry and individuals are now trying to have the tool correct and moderate itself, which intuitively seems wrong from both a technical and societal standpoint.
cons0le 10mo ago
I directly asked gemini how to get world peace. It said the world should prioritize addressing climate change, inequality, and discrimination. Yeah - we're not gonna do any of that shit. So I don't know what the point of "superintelligent" AI is if we aren't going to even listen to it for the basic big picture stuff. Any sort of "utopia" that people imagine AI bringing is doomed to fail because we already can't cooperate without AI
- PunchyHamster 10mo ago
  I dunno, many people have that weird, unfounded trust in what AI says, more than in actual human experts it seems
  - bilbo0s 10mo ago
    Because AI, or rather, an LLM, is the consensus of many human experts as encoded in its embedding. So it is better, but only for those who are already expert in what they're asking. The problem is, you have to know enough about the subject on which you're asking a question to land in the right place in the embedding. If you don't, you'll just get bunk. (I know it's popular to call AI bunk "hallucinations" these days, but really if it was being spouted by a half wit human we'd just call it "bunk".) So you really have to be an expert in order to maximize your use of an LLM. And even then, you'll only be able to maximize your use of that LLM in the field in which your expertise lies. A programmer, for instance, will likely never be able to ask a coherent enough question about economics or oncology for an LLM to give a reliable answer. Similarly, an oncologist will never be able to give a coherent enough software specification for an LLM to write an application for him or her. That's the achilles heel of AI today as implemented by LLMs.
    - jackblemming 10mo ago
      > is the consensus of many human experts as encoded in its embedding That’s not true.
      - ASalazarMX 10mo ago
blauditore 10mo ago
All these engineers who claim to write most code through AI - I wonder what kind of codebase that is. I keep on trying, but it always ends up producing superficially okay-looking code, but getting nuances wrong. Also fails to fix them (just changes random stuff) if pointed to said nuances. I work on a large product with two decades of accumulated legacy, maybe that's the problem. I can see though how generating and editing a simple greenfield web frontend project could work much better, as long as actual complexity is low.
- cogman10 10mo ago
  Honestly, if you've ever looked at a claude.md file, it seems like absolute madness. I feel like I'm reading affirmations from AA.
  - manmal 10mo ago
    It’s magical incantations that might or might not protect you from bad behavior Claude learned from underqualified RL instructors. A classic instruction I have in CLAUDE.md is „Never delete a test. You are only allowed to replace with a test that covers the same branches.“ and another one „Never mention Claude in a commit message“. Of course those sometimes fail, so I do have a message hook that enforces a certain style of git messages.
    - Havoc 10mo ago
      > Never mention Claude in a commit message“. Of course those sometimes fail, It’s hardcoded into the system prompt which is why your CLAUDE.md approach fails. Ended up intercepting it out via proxy
      - manmal 10mo ago
        Thanks for this idea!
    - HWR_14 10mo ago
      Why would it be bad to mention Claude in a commit message?
jascha_eng 10mo ago
Verification is key, and the issue is that almost all AI generated code looks plausible so just reading the code is usually not enough. You need to build extremely good testing systems and actually run through the scenarios that you want to ensure work to be confident in the results. This can be preview deployments or other AI generated end to end tests that produce video output that you can watch or just a very good test suite with guard rails. Without such automation and guard rails, AI generated code eventually becomes a burden on your team because you simply can't manually verify every scenario.
- yuedongze 10mo ago
  indeed, i see verification debt outweighing tradition tech debt very very soon...
- catigula 10mo ago
  I can automatically generate suites of plausible tests using Claude Code. If you can make as a rule "no AI for tests", then you can simply make the rule "no AI" or just learn to cope with it.
- bigbuppo 10mo ago
  And with any luck, they don't vibe code their tests that ultimately just return true;
- jopsen 10mo ago
  I would rather write the code and have AI write the tests :) And I have on occasion found it useful.
trjordan 10mo ago
The verification asymmetry framing is good, but I think it undersells the organizational piece. Daniel works because someone built the regime he operates in. Platform teams standardized the patterns and defined what "correct" looks like and built test infrastructure that makes spot-checking meaningful and and and .... that's not free. Product teams are about to pour a lot more slop into your codebase. That's good! Shipping fast and messy is how products get built. But someone has to build the container that makes slop safe, and have levers to tighten things when context changes. The hard part is you don't know ahead of time which slop will hurt you. Nobody cares if product teams use deprecated React patterns. Until you're doing a migration and those patterns are blocking 200 files. Then you care a lot. You (or rather, platform teams) need a way to say "this matters now" and make it real. There's a lot of verification that's broadly true everywhere, but there's also a lot of company-scoped or even team-scoped definitions of "correct." (Disclosure: we're working on this at tern.sh, with migrations as the forcing function. There's a lot of surprises in migrations, so we're starting there, but eventually, this notion of "organizational validation" is a big piece of what we're driving at.)
wasmainiac 10mo ago
It’s called TDD, ya write a bunch a little tests to make sure your code is doing what it needs to do and not what it’s not. In short, little blocks of easily verifiable code to verify your code. But seriously, what is this article even? It feels like we are reinventing the wheel or maybe just humble AI hype?
awesome_dude 10mo ago
It's like a buffered queue, if the producer (AI) is too fast for the consumer (dev's brain) then the producer needs to block/stop/slow down other wise data will be lost (in this analogy the data loss is the consumer no longer having a clear understanding of what the code is doing) One day, when AI becomes reliable (which is still a while off because AI doesn't yet understand what it's doing) then the AI will replace the consumer (IMO). FTR - AI is still at the "text matches another pattern of text" stage, and not the "understand what concepts are being conveyed" stage, as demonstrated by AI's failure to do basic arithmetic
kristjank 10mo ago
This feeling of verification >> generation anxiety bears a resemblance to that moment when you're learning a foreign language, you speak a well-prepared sentence, and your correspondent says something back, of which you only understand about a third. In like fashion, when I start thinking of a programming statement (as a bad/rookie programmer) and an assistant completes my train of thought (as is default behaviour in VS Code for example), I get that same feeling that I did not grasp half the stuff I should've, but nevertheless I hit Ctrl-Return because it looks about right to me.
- yuedongze 10mo ago
  > because it looks about right to me this is something one can look in further. it is really probabilistic checkable proofs underneath, and we are naturally looking for places where it needs to look right, and use that as a basis of assuming the work is done right.
yuedongze 10mo ago
It's nice to see a wide array of discussions under this! Glad that I didn't give up on this thought and end up writing it down. I want to stress that the main point of my article is not really about AI coding, it's about letting AI perform any arbitrary tasks reliably. Coding is an interesting one because it seems like it's a place where we can exploit structure and abstraction and approaches (like TDD) to make verification simpler - it's like spot-checking in places with a very low soundness error. I'm encouraging people to look for tasks other than coding to see if we can find similar patterns. The more we can find these cost asymmetry (easier to verify than doing), the more we can harness AI's real potential.
- Yoric 10mo ago
  Note that in the case of coding, there is an entire branch of computer science dedicated to verification. All the type systems (and model-checkers) for Rust, Ada, OCaml, Haskell, TypeScript, Python, C#, Java, ... are based on such research, and these are all rather weak in comparison to what research has created in the last ~30 years (see Rocq, Idris, Lean). This goes beyond that, as some of these mechanisms have been applied to mathematics, but also to some aspects of finance and law (I know of at least mechanisms to prove formally implementations of banking contracts and tax management). So there is lots to do in the domain. Sadly, as every branch of CS other than AI (and in fact pretty much every branch of science other than AI), this branch of computer science is underfunded. But that can change!
  - charcircuit 10mo ago
    Considering how useful I've found AI at finding and fixing bugs proportional to the effort I put in, I question your claim that it's being underfunded. While I have learned things like Idris, in the end I never was able to practically use them to reduce bugs in the software I was writing unlike AI. It's possible that the funding towards these types of languages is actually distracting people from more practical solutions which could actually mean that it is overfunded in regards to program verification.
    - Yoric 10mo ago
deleted 10mo ago
[deleted]
Yoric 10mo ago
> Maybe our future is like the one depicted in Severance - we look at computer screens with wiggly numbers and whatever “feels right” is the right thing to do. We can harvest these effortless low latency “feelings” that nature gives us to make AI do more powerful work. Come to think about it... aren't this exactly what syntax coloring and proper indentation are all about? The ability to quickly pattern-spot errors, or at least smells, based on nothing but aesthetics? I'm sure that there is more research to be done in this direction.
- bitwize 10mo ago
  Our future is going to be like the one in Severance: no creation, no craftsmanship, just grooming the grids of numbers that do the actual work.
diddid 10mo ago
AI can really only be as good as the data it’s trained on. It’s good at images because it’s trained on billions of them. Lines of code, probably 100s of millions, but as you combine those codes into concepts, split by language, framework, formatting etc all you loose the numbers game. It can’t tell you how to make a good enterprise app because almost nobody knows how to make a good enterprise app, just ask Oracle… ba-da-bum!
pglevy 10mo ago
I've been thinking about something like this from a UI perspective. I'm a UX designer working on a product with a fairly legacy codebase. We're vibe coding prototypes and moving towards making it easier for devs to bring in new components. We have a hard enough time verifying the UI quality as it is. And having more devs vibing on frontend code is probably going to make it a lot worse. I'm thinking about something like having agents regularly traversing the code to identify non-approved components (and either fixing or flagging them). Maybe with this we won't fall further behind with verification debt than we already are.
- huflungdung 10mo ago
  [dead]
- cousinbryce 10mo ago
  That’s just static code analysis with extra steps
ambicapter 10mo ago
So, we're giving up on the Singularity, then?...Good.
adxl 10mo ago
I remember a junior dev who thought he was done when his code conpiled without syntax errors.
darylteo 10mo ago
AI: urgh, sick of these escort missions
joeyguerra 10mo ago
Make it do TDD. That'll slow it down.
- vjvjvjvjghv 10mo ago
  Make it do scrum with sprint planning, retrospectives and sprint demos. A then another AI as product owner and scrum master. Ideally this AI has only a vague idea of what the product needs to or the technology but still has decision power. That should really slow it down.
  - theshrike79 10mo ago
    This was done already, they're called "Agent networks" or whatever buzzword the dev decided to give their abomination :)
seanmcdirmid 10mo ago
Make the AI go to lots of meetings. It won’t stand a chance in keeping up its productivity.
- zerosizedweasle 10mo ago
  https://www.reuters.com/graphics/USA-ECONOMY/AI-INVESTMENT/gkvlqbgxkpb/ https://www.reuters.com/graphics/USA-ECONOMY/AI-INVESTMENT/g...
  - HPsquared 10mo ago
    1.6 trillion, why that's almost as much as the F-35 program!
  - timpera 10mo ago
    Great visualization!
deleted 10mo ago
[deleted]
aryehof 10mo ago
> “AI always thinks and learns faster than us, this is undeniable now” No, it neither thinks nor learns. It can give an illusion of thinking, and an AI model itself learns nothing. Instead it can produce a result based on its training data and context. I think it important that we do not ascribe human characteristics where not warranted. I also believe that understanding this can help us better utilize AI.
karlkloss 10mo ago
That has been said about the world in general. Guess what?
gaigalas 10mo ago
Prompt engineering: just basic articulation skills. Context engineering: just basic organization skills. Verification engineering: just basic quality assurance skills. And so on... --- "Eric" will never be able to fully use AI for development because he lacks knowledge about even the most basic aspects of the developer's job. He's a PM after all. I understand that the idea of turning everyone into instant developers is super attractive. However, you can't cheat learning. If you give an edge to non-developers for development tasks, it means you will give an even sharper edge to actual developers.
- booleandilemma 10mo ago
  This is true. I've been anti-ai but I started using it recently as an alternative to stack overflow (because google is shoving it down my mouth via search results). It's pretty effective. It does get things wrong from time to time, but then I just fix it up manually. I can't claim it's making me 100x more productive or anything like that. It's just a nice alternative to scrolling through SO answers and looking for the one with the green checkmark. I still find it sad when people use it for prose though.
  - gaigalas 10mo ago
    If an agent gets things wrong you should stop it and correct it instead. Sometimes the correction will cost more than starting from scratch. In those cases, you start from scratch. You do things manually only when novel work is required (the model is unlikely to be trained with the knowledge). The more novel the thing you're doing, the more manual things you have to do. Identifying "cost of refactoring", and "is this novel?" are also developer skills, so, no formula here. You have to know.
Sabr0 10mo ago
Ai now is becoming hard to keep up with. We gotta make sure to integrate in our daily lives to not fall behind. I literally began to make it a source of income. Make sure to do the same.
delis-thumbs-7e 10mo ago
> A very good example of the first category is image (and video) generation. Drawing/rendering a realistic looking image is a crazily hard task. Have you tried to make a slide look nicer? It will take me literally hours to center the text boxes to make it look “good”. However, you really just need to take a look at the output of Nano Banana and you can tell if it’s a good render or a bad one based on how you feel. The writer could be very accomplished when it comes to developing - I don’t know - but they clearly don’t understand a single thing about visual arts or culture. I probably could center those text boxes after fiddling with them maybe ten seconds - I have studied art since I was a kid. My bf could do it instantly without thinking a second, he is a graphic designer. You might think that you are able to see what « looks good » since, hey you have eyes, but no you can’t. There’s million details you will miss, or maybe feel something is off, but cannot quite say why. This is why you have graphic designers, who are trained to do that to do it. They can also use generative tools to make something genuinely stunning, unlike most of us. Why? Skills. This is the same difference why the guy in the story who can’t code can’t code even with LLM, whereas the guy who cans is able to code even faster with these new tools. If use LLM’s for basically auto-completion (what transformer models really are for) you can work with familiar codebase very quickly I’m sure. I’ve used it to gen SQL call statements, which I can’t be bothered to type myself and it was perfect. If I try to generate something I don’t really understand or know how to do, I’m lost staring at sole horrible gobbledygoo that is never going to work. Why? Skills. There is no verification engineering. There is just people who know how to do things, who have studied their whole life to get those skills. And no, you will not replace a real hardcore professional with an LLM. LLM’s are just tools, nothing else. A tractor replaced a horse in turning the field, bit you still need a farmer to drive it.
- jstanley 10mo ago
  Centering text boxes in competent design software is easy because it has a tool to align things to the centre of other things. For example, Inkscape has this and it is easy to use.
  - delis-thumbs-7e 10mo ago
    I meant just by eye, mate. But it is pretty bad example anyway, obvs it is something that any program can do better than us. Better would be layout or maybe typography. Even professionals mess it up all the time. Point is, even basic visual design is far from intuitive.
kaluga 10mo ago
[dead]
bamboozled 10mo ago
I'm starting to come to the realization that unless there is a bottom to the amount of work people want done, it doesn't really matter about AI or not, there just seems to be a never ending supply of work so yeah, not sure how AI would resolve this.
donatj 10mo ago
I have been a developer for twenty years now. For me to trust code, my want is to understand every single line. I learned long ago working on projects with a team that that becomes impossible for a single person on large projects. I learned to trust that someone understands the code and between blames and Slack I can almost always hunt that person down. More and more often, while doing code review, I find I will not understand something and I will ask, and the "author" will clearly have no idea what it is doing either. I find it quite troubling how little actual human thought is going into things. The AIs context window is not nearly large enough to fully understand the entire scope of any decently sized applications ecosystem. It just takes small peaks at bits and makes decisions based on a tiny slice of the world. It's a powerful tool and as such needs to be guided with care.
- MLgulabio 10mo ago
  Software becomes legacy very fast. I have seen so many projects were people who understood all of it, are just gone. They moved, did something else etc. As soon as this happens, you no longer have anyone 'getting it'. You have to handle so many people adding/changing very thin lines across all components and you can only hope that the original people had enough foresight adding enough unit tests for core decisions. So i really don't mind AI here anymore.
  - rnewme 10mo ago
    Not sure why this is dead, but in nearly all of my consulting gigs sooner or later I ended up having to check on project/service that is effectively abandoned. Last time this morning. Luckily I had claude code and CLI tools to go through few dozen repos and millions LOC to find some obscure endpoints and data structures, since there wasn't even anyone to ask what to look for.
- fragmede 10mo ago
  Can humans though? There's a reason we don't just lump everything into one giant file and singleton class named DoIt(). Who hasn't come back around to some bit of code in a project and wondered what dumbass wrote this, only for the logs to tell you that it was you that wrote it, years ago. If AI is resulting in code that's more modular, in smaller digestible and understandable chunks, I'm not hearing that as a bad thing!
pxc 10mo ago
> AI always thinks and learns faster than us, this is undeniable now. Huh? The LLMs we're using today don't learn at all. I don't even mean that in a philosophical sense— I mean they come "pre-baked" with whatever "knowledge" they have, and that's it.
officerk 10mo ago
A related post on the topic: https://martin.kleppmann.com/2025/12/08/ai-formal-verification.html https://martin.kleppmann.com/2025/12/08/ai-formal-verificati...
geldedus 10mo ago
Sure. You're free to throttle your AI speed or whatever. But don't impose that on me.
nirui 10mo ago
> He would just spot-check the correctness of AI’s work and quickly spin up local deployments to verify it’s indeed working. I'm not really sure how exactly he get the project done, but "spot-check" and "quickly spin up local deployments to verify" is somehow makes me somewhat unconformable. For me, it's either unit-tests that hits at least 100% coverage, or when unit-test is inapplicable, a line-by-line letter-by-letter verification. Otherwise your "spot-check" means no shit to me.
WhyOhWhyQ 10mo ago
"AI always thinks and learns faster than us, this is undeniable now. " Sort of a nitpick, because what's written is true in some contexts (I get it, web development is like the ideal context for AI for a variety of reasons), but this is currently totally false in lots of knowledge domains very much like programming. AI is currently terrible at the math niches I'm interested in. Since there's no economic incentive to improve things and no mountain of literature on those topics, unless AI really becomes self-learning / improves in some real way, I don't see the situation ever changing. AI has consistently gotten effectively a 0% score on my personal benchmarks for those topics. It's just aggravating to see someone write "totally undeniable" when the thing is trivially denied.
- jakeydus 10mo ago
  > It's just aggravating to see someone write "totally undeniable" when the thing is trivially denied. You've described AI hype bros in a nutshell, I think.

198 points by yuedongze 10mo ago

Appealing, but this is coming from someone smart/thoughtful. No offence to 'rest of world', but I think that most people have felt this way for years. And realistically in a year, there won't be any people who can keep up.

airstrike 10mo ago
> And realistically in a year, there won't be any people who can keep up. Bold claim. They said the same thing at the start of this year.
- adventured 10mo ago
  You're all arguing over how many single digit years it'll take at this point. It doesn't matter if it takes another 12 or 36 months to make that claim true. It doesn't matter if it takes five years. Is AI coming for most of the software jobs? Yes it is. It's moving very quickly, and nothing can stop it. The progress has been particularly exceptionally clear (early GPT to Gemini 3 / Opus 4.5 / Codex).
  - bdangubic 10mo ago
    > Is AI coming for most of the software jobs? be cool to start with one before we move to most…
    - esafak 10mo ago
      https://news.ycombinator.com/item?id=46124063 https://news.ycombinator.com/item?id=46124063
yuedongze 10mo ago
im hoping this can introduce a framework to help people visualize the problem and figure out a way to close that gap. image generation is something every one can verify, but code generation is perhaps not. but if we can make verifying code as effortless as verifying images (not saying it's possible), then our productivity can enter the next level...
- drlobster 10mo ago
  I think you underestimating how good these image generators are at the moment.
  - yuedongze 10mo ago
    oh i mean the other direction! checking if a generated image is "good" that no one will tell something is off and it look naturally, rather than checking if they are fake.
dontlikeyoueith 10mo ago
> And realistically in a year, there won't be any people who can keep up. I've heard the same claim every year since GPT-3. It's still just as irrational as it was then.
- adventured 10mo ago
  You're rather dramatically demonstrating how remarkable the progress has been: GPT-3 was horrible at coding. Claude Opus 4.5 is good at it. They're already far faster than anybody on HN could ever be. Whether it takes another five years or ten, in that span of time nobody on HN will be able to keep up with the top tier models. It's not irrational, it's guaranteed. The progress has been extraordinary and obvious, the direction is certain, the outcome is certain. All that is left is to debate whether it's a couple of years or closer to a decade.
  - Arainach 10mo ago
    People claimed GPT-3 was great at coding when it launched. Those who said otherwise were dismissed. That has continued to be the case in every generation.
    - dwaltrip 10mo ago
      A bit reductive.
    - stale2002 10mo ago
      > People claimed GPT-3 was great at coding when it launched. Ok and they were wrong, but now people are right that it is great at coding. > That has continued to be the case in every generation. If something gets better over time, it is definitionally true that it was bad for every case in the past until it becomes good. But then it is good. Thats how that works. For everything. You are talking in tautologies while not understanding the implication of your arguments and how it applies to very general things like "A thing that improves over time".
    - esafak 10mo ago

gradus_ad 10mo ago

The proliferation of nondeterministically generated code is here to stay. Part of our response must be more dynamic, more comprehensive and more realistic workload simulation and testing frameworks.

yuedongze 10mo ago
i've seen a lot of startups that use AI to QA human work. how about the idea of use humans to QA AI work? a lot of interesting things might follow
- Aldipower 10mo ago
  Sounds inhuman.
  - A4ET8a8uTh0_v2 10mo ago
    Nah, sounds like management, but I am repeating myself. In all seriousness, I have found myself having to carefully rein some of similar decisions in. I don't want to get into details, but there are times I wonder if they understand how things really work or if people need some 'floor' level exposure before they just decree stuff.
  - quantummagic 10mo ago
    As an industry, we've been doing the same thing to people in almost every other sector of the workforce, since we began. Automation is just starting to come for us now, and a lot of us are really pissed off about it. All of a sudden, we're humanitarians.
    - Terr_ 10mo ago
      > Automation is just starting to come for us now This argument is common and facile: Software development has always been about "automating ourselves out of a job", whether in the broad sense of creating compilers and IDEs, or in the individual sense that you write some code and say: "Hey, I don't want to rewrite this again later, not even if I was being paid for my time, I'll make it into a reusable library." > the same thing The reverse: What pisses me off is how what's coming is not the same thing. Customers are being sold a snake-oil product, and its adoption may well ruin things we've spent careers de-crappifying by making them consistent and repeatable and understandable. In the aftermath, some portion of my (continued) career will be diverted to cleaning up the lingering damage from it.

CGMthrowaway 10mo ago

> AI should only run as fast as we can catch up Good principle. This is exactly why we research vaccines and bioweapons side by side in the labs, for example.

yannyu 10mo ago

I think there's a lot of utility to current AI tools, but it's also clear we're in a very unsettled phase of this technology. We likely won't see for years where the technology lands in terms of capability or the changes that will be made to society and industry to accommodate. Somewhat unfortunately, the sheer amount of money being poured into AI means that it's being forced upon many of us, even if we didn't want it. Which results in a stark, vast gap like the author is describing, where things are moving so fast that it can feel like we may never have time to catch up. And what's even worse, because of this industry and individuals are now trying to have the tool correct and moderate itself, which intuitively seems wrong from both a technical and societal standpoint.

cons0le 10mo ago

I directly asked gemini how to get world peace. It said the world should prioritize addressing climate change, inequality, and discrimination. Yeah - we're not gonna do any of that shit. So I don't know what the point of "superintelligent" AI is if we aren't going to even listen to it for the basic big picture stuff. Any sort of "utopia" that people imagine AI bringing is doomed to fail because we already can't cooperate without AI

PunchyHamster 10mo ago
I dunno, many people have that weird, unfounded trust in what AI says, more than in actual human experts it seems
- bilbo0s 10mo ago
  Because AI, or rather, an LLM, is the consensus of many human experts as encoded in its embedding. So it is better, but only for those who are already expert in what they're asking. The problem is, you have to know enough about the subject on which you're asking a question to land in the right place in the embedding. If you don't, you'll just get bunk. (I know it's popular to call AI bunk "hallucinations" these days, but really if it was being spouted by a half wit human we'd just call it "bunk".) So you really have to be an expert in order to maximize your use of an LLM. And even then, you'll only be able to maximize your use of that LLM in the field in which your expertise lies. A programmer, for instance, will likely never be able to ask a coherent enough question about economics or oncology for an LLM to give a reliable answer. Similarly, an oncologist will never be able to give a coherent enough software specification for an LLM to write an application for him or her. That's the achilles heel of AI today as implemented by LLMs.
  - jackblemming 10mo ago
    > is the consensus of many human experts as encoded in its embedding That’s not true.
    - ASalazarMX 10mo ago

blauditore 10mo ago

All these engineers who claim to write most code through AI - I wonder what kind of codebase that is. I keep on trying, but it always ends up producing superficially okay-looking code, but getting nuances wrong. Also fails to fix them (just changes random stuff) if pointed to said nuances. I work on a large product with two decades of accumulated legacy, maybe that's the problem. I can see though how generating and editing a simple greenfield web frontend project could work much better, as long as actual complexity is low.

cogman10 10mo ago
Honestly, if you've ever looked at a claude.md file, it seems like absolute madness. I feel like I'm reading affirmations from AA.
- manmal 10mo ago
  It’s magical incantations that might or might not protect you from bad behavior Claude learned from underqualified RL instructors. A classic instruction I have in CLAUDE.md is „Never delete a test. You are only allowed to replace with a test that covers the same branches.“ and another one „Never mention Claude in a commit message“. Of course those sometimes fail, so I do have a message hook that enforces a certain style of git messages.
  - Havoc 10mo ago
    > Never mention Claude in a commit message“. Of course those sometimes fail, It’s hardcoded into the system prompt which is why your CLAUDE.md approach fails. Ended up intercepting it out via proxy
    - manmal 10mo ago
      Thanks for this idea!
  - HWR_14 10mo ago
    Why would it be bad to mention Claude in a commit message?

jascha_eng 10mo ago

Verification is key, and the issue is that almost all AI generated code looks plausible so just reading the code is usually not enough. You need to build extremely good testing systems and actually run through the scenarios that you want to ensure work to be confident in the results. This can be preview deployments or other AI generated end to end tests that produce video output that you can watch or just a very good test suite with guard rails. Without such automation and guard rails, AI generated code eventually becomes a burden on your team because you simply can't manually verify every scenario.

yuedongze 10mo ago
indeed, i see verification debt outweighing tradition tech debt very very soon...
catigula 10mo ago
I can automatically generate suites of plausible tests using Claude Code. If you can make as a rule "no AI for tests", then you can simply make the rule "no AI" or just learn to cope with it.
bigbuppo 10mo ago
And with any luck, they don't vibe code their tests that ultimately just return true;
jopsen 10mo ago
I would rather write the code and have AI write the tests :) And I have on occasion found it useful.

trjordan 10mo ago

The verification asymmetry framing is good, but I think it undersells the organizational piece. Daniel works because someone built the regime he operates in. Platform teams standardized the patterns and defined what "correct" looks like and built test infrastructure that makes spot-checking meaningful and and and .... that's not free. Product teams are about to pour a lot more slop into your codebase. That's good! Shipping fast and messy is how products get built. But someone has to build the container that makes slop safe, and have levers to tighten things when context changes. The hard part is you don't know ahead of time which slop will hurt you. Nobody cares if product teams use deprecated React patterns. Until you're doing a migration and those patterns are blocking 200 files. Then you care a lot. You (or rather, platform teams) need a way to say "this matters now" and make it real. There's a lot of verification that's broadly true everywhere, but there's also a lot of company-scoped or even team-scoped definitions of "correct." (Disclosure: we're working on this at tern.sh, with migrations as the forcing function. There's a lot of surprises in migrations, so we're starting there, but eventually, this notion of "organizational validation" is a big piece of what we're driving at.)

wasmainiac 10mo ago

It’s called TDD, ya write a bunch a little tests to make sure your code is doing what it needs to do and not what it’s not. In short, little blocks of easily verifiable code to verify your code. But seriously, what is this article even? It feels like we are reinventing the wheel or maybe just humble AI hype?

awesome_dude 10mo ago

It's like a buffered queue, if the producer (AI) is too fast for the consumer (dev's brain) then the producer needs to block/stop/slow down other wise data will be lost (in this analogy the data loss is the consumer no longer having a clear understanding of what the code is doing) One day, when AI becomes reliable (which is still a while off because AI doesn't yet understand what it's doing) then the AI will replace the consumer (IMO). FTR - AI is still at the "text matches another pattern of text" stage, and not the "understand what concepts are being conveyed" stage, as demonstrated by AI's failure to do basic arithmetic

kristjank 10mo ago

This feeling of verification >> generation anxiety bears a resemblance to that moment when you're learning a foreign language, you speak a well-prepared sentence, and your correspondent says something back, of which you only understand about a third. In like fashion, when I start thinking of a programming statement (as a bad/rookie programmer) and an assistant completes my train of thought (as is default behaviour in VS Code for example), I get that same feeling that I did not grasp half the stuff I should've, but nevertheless I hit Ctrl-Return because it looks about right to me.

yuedongze 10mo ago
> because it looks about right to me this is something one can look in further. it is really probabilistic checkable proofs underneath, and we are naturally looking for places where it needs to look right, and use that as a basis of assuming the work is done right.

yuedongze 10mo ago

It's nice to see a wide array of discussions under this! Glad that I didn't give up on this thought and end up writing it down. I want to stress that the main point of my article is not really about AI coding, it's about letting AI perform any arbitrary tasks reliably. Coding is an interesting one because it seems like it's a place where we can exploit structure and abstraction and approaches (like TDD) to make verification simpler - it's like spot-checking in places with a very low soundness error. I'm encouraging people to look for tasks other than coding to see if we can find similar patterns. The more we can find these cost asymmetry (easier to verify than doing), the more we can harness AI's real potential.

Yoric 10mo ago
Note that in the case of coding, there is an entire branch of computer science dedicated to verification. All the type systems (and model-checkers) for Rust, Ada, OCaml, Haskell, TypeScript, Python, C#, Java, ... are based on such research, and these are all rather weak in comparison to what research has created in the last ~30 years (see Rocq, Idris, Lean). This goes beyond that, as some of these mechanisms have been applied to mathematics, but also to some aspects of finance and law (I know of at least mechanisms to prove formally implementations of banking contracts and tax management). So there is lots to do in the domain. Sadly, as every branch of CS other than AI (and in fact pretty much every branch of science other than AI), this branch of computer science is underfunded. But that can change!
- charcircuit 10mo ago
  Considering how useful I've found AI at finding and fixing bugs proportional to the effort I put in, I question your claim that it's being underfunded. While I have learned things like Idris, in the end I never was able to practically use them to reduce bugs in the software I was writing unlike AI. It's possible that the funding towards these types of languages is actually distracting people from more practical solutions which could actually mean that it is overfunded in regards to program verification.
  - Yoric 10mo ago

deleted 10mo ago

[deleted]

Yoric 10mo ago

> Maybe our future is like the one depicted in Severance - we look at computer screens with wiggly numbers and whatever “feels right” is the right thing to do. We can harvest these effortless low latency “feelings” that nature gives us to make AI do more powerful work. Come to think about it... aren't this exactly what syntax coloring and proper indentation are all about? The ability to quickly pattern-spot errors, or at least smells, based on nothing but aesthetics? I'm sure that there is more research to be done in this direction.

bitwize 10mo ago
Our future is going to be like the one in Severance: no creation, no craftsmanship, just grooming the grids of numbers that do the actual work.

diddid 10mo ago

AI can really only be as good as the data it’s trained on. It’s good at images because it’s trained on billions of them. Lines of code, probably 100s of millions, but as you combine those codes into concepts, split by language, framework, formatting etc all you loose the numbers game. It can’t tell you how to make a good enterprise app because almost nobody knows how to make a good enterprise app, just ask Oracle… ba-da-bum!

pglevy 10mo ago

I've been thinking about something like this from a UI perspective. I'm a UX designer working on a product with a fairly legacy codebase. We're vibe coding prototypes and moving towards making it easier for devs to bring in new components. We have a hard enough time verifying the UI quality as it is. And having more devs vibing on frontend code is probably going to make it a lot worse. I'm thinking about something like having agents regularly traversing the code to identify non-approved components (and either fixing or flagging them). Maybe with this we won't fall further behind with verification debt than we already are.

huflungdung 10mo ago
[dead]
cousinbryce 10mo ago
That’s just static code analysis with extra steps

ambicapter 10mo ago

So, we're giving up on the Singularity, then?...Good.

adxl 10mo ago

I remember a junior dev who thought he was done when his code conpiled without syntax errors.

darylteo 10mo ago

AI: urgh, sick of these escort missions

joeyguerra 10mo ago

Make it do TDD. That'll slow it down.

vjvjvjvjghv 10mo ago
Make it do scrum with sprint planning, retrospectives and sprint demos. A then another AI as product owner and scrum master. Ideally this AI has only a vague idea of what the product needs to or the technology but still has decision power. That should really slow it down.
- theshrike79 10mo ago
  This was done already, they're called "Agent networks" or whatever buzzword the dev decided to give their abomination :)

seanmcdirmid 10mo ago

Make the AI go to lots of meetings. It won’t stand a chance in keeping up its productivity.

zerosizedweasle 10mo ago
https://www.reuters.com/graphics/USA-ECONOMY/AI-INVESTMENT/gkvlqbgxkpb/ https://www.reuters.com/graphics/USA-ECONOMY/AI-INVESTMENT/g...
- HPsquared 10mo ago
  1.6 trillion, why that's almost as much as the F-35 program!
- timpera 10mo ago
  Great visualization!

deleted 10mo ago

[deleted]

aryehof 10mo ago

> “AI always thinks and learns faster than us, this is undeniable now” No, it neither thinks nor learns. It can give an illusion of thinking, and an AI model itself learns nothing. Instead it can produce a result based on its training data and context. I think it important that we do not ascribe human characteristics where not warranted. I also believe that understanding this can help us better utilize AI.

karlkloss 10mo ago

That has been said about the world in general. Guess what?

gaigalas 10mo ago

Prompt engineering: just basic articulation skills. Context engineering: just basic organization skills. Verification engineering: just basic quality assurance skills. And so on... --- "Eric" will never be able to fully use AI for development because he lacks knowledge about even the most basic aspects of the developer's job. He's a PM after all. I understand that the idea of turning everyone into instant developers is super attractive. However, you can't cheat learning. If you give an edge to non-developers for development tasks, it means you will give an even sharper edge to actual developers.

booleandilemma 10mo ago
This is true. I've been anti-ai but I started using it recently as an alternative to stack overflow (because google is shoving it down my mouth via search results). It's pretty effective. It does get things wrong from time to time, but then I just fix it up manually. I can't claim it's making me 100x more productive or anything like that. It's just a nice alternative to scrolling through SO answers and looking for the one with the green checkmark. I still find it sad when people use it for prose though.
- gaigalas 10mo ago
  If an agent gets things wrong you should stop it and correct it instead. Sometimes the correction will cost more than starting from scratch. In those cases, you start from scratch. You do things manually only when novel work is required (the model is unlikely to be trained with the knowledge). The more novel the thing you're doing, the more manual things you have to do. Identifying "cost of refactoring", and "is this novel?" are also developer skills, so, no formula here. You have to know.

Sabr0 10mo ago

Ai now is becoming hard to keep up with. We gotta make sure to integrate in our daily lives to not fall behind. I literally began to make it a source of income. Make sure to do the same.

delis-thumbs-7e 10mo ago

> A very good example of the first category is image (and video) generation. Drawing/rendering a realistic looking image is a crazily hard task. Have you tried to make a slide look nicer? It will take me literally hours to center the text boxes to make it look “good”. However, you really just need to take a look at the output of Nano Banana and you can tell if it’s a good render or a bad one based on how you feel. The writer could be very accomplished when it comes to developing - I don’t know - but they clearly don’t understand a single thing about visual arts or culture. I probably could center those text boxes after fiddling with them maybe ten seconds - I have studied art since I was a kid. My bf could do it instantly without thinking a second, he is a graphic designer. You might think that you are able to see what « looks good » since, hey you have eyes, but no you can’t. There’s million details you will miss, or maybe feel something is off, but cannot quite say why. This is why you have graphic designers, who are trained to do that to do it. They can also use generative tools to make something genuinely stunning, unlike most of us. Why? Skills. This is the same difference why the guy in the story who can’t code can’t code even with LLM, whereas the guy who cans is able to code even faster with these new tools. If use LLM’s for basically auto-completion (what transformer models really are for) you can work with familiar codebase very quickly I’m sure. I’ve used it to gen SQL call statements, which I can’t be bothered to type myself and it was perfect. If I try to generate something I don’t really understand or know how to do, I’m lost staring at sole horrible gobbledygoo that is never going to work. Why? Skills. There is no verification engineering. There is just people who know how to do things, who have studied their whole life to get those skills. And no, you will not replace a real hardcore professional with an LLM. LLM’s are just tools, nothing else. A tractor replaced a horse in turning the field, bit you still need a farmer to drive it.

jstanley 10mo ago
Centering text boxes in competent design software is easy because it has a tool to align things to the centre of other things. For example, Inkscape has this and it is easy to use.
- delis-thumbs-7e 10mo ago
  I meant just by eye, mate. But it is pretty bad example anyway, obvs it is something that any program can do better than us. Better would be layout or maybe typography. Even professionals mess it up all the time. Point is, even basic visual design is far from intuitive.

kaluga 10mo ago

[dead]

bamboozled 10mo ago

I'm starting to come to the realization that unless there is a bottom to the amount of work people want done, it doesn't really matter about AI or not, there just seems to be a never ending supply of work so yeah, not sure how AI would resolve this.

donatj 10mo ago

I have been a developer for twenty years now. For me to trust code, my want is to understand every single line. I learned long ago working on projects with a team that that becomes impossible for a single person on large projects. I learned to trust that someone understands the code and between blames and Slack I can almost always hunt that person down. More and more often, while doing code review, I find I will not understand something and I will ask, and the "author" will clearly have no idea what it is doing either. I find it quite troubling how little actual human thought is going into things. The AIs context window is not nearly large enough to fully understand the entire scope of any decently sized applications ecosystem. It just takes small peaks at bits and makes decisions based on a tiny slice of the world. It's a powerful tool and as such needs to be guided with care.

MLgulabio 10mo ago
Software becomes legacy very fast. I have seen so many projects were people who understood all of it, are just gone. They moved, did something else etc. As soon as this happens, you no longer have anyone 'getting it'. You have to handle so many people adding/changing very thin lines across all components and you can only hope that the original people had enough foresight adding enough unit tests for core decisions. So i really don't mind AI here anymore.
- rnewme 10mo ago
  Not sure why this is dead, but in nearly all of my consulting gigs sooner or later I ended up having to check on project/service that is effectively abandoned. Last time this morning. Luckily I had claude code and CLI tools to go through few dozen repos and millions LOC to find some obscure endpoints and data structures, since there wasn't even anyone to ask what to look for.
fragmede 10mo ago
Can humans though? There's a reason we don't just lump everything into one giant file and singleton class named DoIt(). Who hasn't come back around to some bit of code in a project and wondered what dumbass wrote this, only for the logs to tell you that it was you that wrote it, years ago. If AI is resulting in code that's more modular, in smaller digestible and understandable chunks, I'm not hearing that as a bad thing!

pxc 10mo ago

> AI always thinks and learns faster than us, this is undeniable now. Huh? The LLMs we're using today don't learn at all. I don't even mean that in a philosophical sense— I mean they come "pre-baked" with whatever "knowledge" they have, and that's it.

officerk 10mo ago

A related post on the topic: https://martin.kleppmann.com/2025/12/08/ai-formal-verification.html https://martin.kleppmann.com/2025/12/08/ai-formal-verificati...

geldedus 10mo ago

Sure. You're free to throttle your AI speed or whatever. But don't impose that on me.

nirui 10mo ago

> He would just spot-check the correctness of AI’s work and quickly spin up local deployments to verify it’s indeed working. I'm not really sure how exactly he get the project done, but "spot-check" and "quickly spin up local deployments to verify" is somehow makes me somewhat unconformable. For me, it's either unit-tests that hits at least 100% coverage, or when unit-test is inapplicable, a line-by-line letter-by-letter verification. Otherwise your "spot-check" means no shit to me.

WhyOhWhyQ 10mo ago

"AI always thinks and learns faster than us, this is undeniable now. " Sort of a nitpick, because what's written is true in some contexts (I get it, web development is like the ideal context for AI for a variety of reasons), but this is currently totally false in lots of knowledge domains very much like programming. AI is currently terrible at the math niches I'm interested in. Since there's no economic incentive to improve things and no mountain of literature on those topics, unless AI really becomes self-learning / improves in some real way, I don't see the situation ever changing. AI has consistently gotten effectively a 0% score on my personal benchmarks for those topics. It's just aggravating to see someone write "totally undeniable" when the thing is trivially denied.

jakeydus 10mo ago
> It's just aggravating to see someone write "totally undeniable" when the thing is trivially denied. You've described AI hype bros in a nutshell, I think.