6 ms·
The great AI delusion is falling apart
- nayshins 1y agoI'm happy to let people think that AI does not yield productivity gains. There is no point engaging on this topic, so I will just outwork/outperform them.
- kubb 1y agoGood luck getting paid more for your improved performance :D
- stronglikedan 1y agoNo one gets paid more to get the same job done, unless you count free time as compensation.
- some_random 1y agoIf they're right in their belief that AI usage leads to significantly more performance, their compensation is that they will keep their job.
- bluefirebrand 1y ago"You get to keep your job" is the worst consolation prize I can imagine
- nayshins 1y agoI get paid a lot already.
- kubb 1y agoThen enjoy demonstrating your AI-supercharged productivity advantage over your inferior peers, I think they'll love that.
- nayshins 1y agoTurns out that I spend a lot of my spare time teaching others how to use them effectively. My goal is to help as many people become experts at this as I can.
- hooverd 1y agoCrazy how productivity gains just lead to more work for you.
- stronglikedan 1y agoCrazy how people would let their managers know they could get more done, instead of getting the same amount done quicker and having more free time.
- gishglish 1y agoIf I was genuinely getting more free time, I’d be more amenable to this line of thought, but RTO put the nail in that coffin.
- bluefirebrand 1y agoThis is actually worth talking about imo There is nothing in it for me, if I am more productive but earn the same and don't get any more time off Why should I bother at that point?
- cableshaft 1y agoRelevant Office Space movie scene: https://youtu.be/to_e1N4xovQ?si=V6EVep8WqJ8zWisG https://youtu.be/to_e1N4xovQ?si=V6EVep8WqJ8zWisG
- bluefirebrand 1y agoYep, I'm ready to be management
- CharlesW 1y agoThe question is what happens when you're half as productive as everyone around you.
- bluefirebrand 1y ago
- troupo 1y ago> I'm happy to let people think that AI does not yield productivity gains. vs. --- start quote --- In a randomised controlled trial – the first of its kind – experienced computer programmers could use AI tools to help them write code. --- end quote --- Your quote is very representative of the magical wishful thinking most people have about AI: https://dmitriid.com/everything-around-llms-is-still-magical-and-wishful-thinking https://dmitriid.com/everything-around-llms-is-still-magical...
- refulgentis 1y ago(not op) Gosh, I was conflicted, then you pulled out that sentence and I was convinced. :) Alternatively: When faced with a contradiction, first, check your premises. I don't want to belabor the point too much, there's little common ground if we're at all or nothing thinking - "the study proved AI is net-negative because of this pull quote" isn't discussion.
- simonw 1y ago"Your quote is very representative of the magical wishful thinking most people have about AI" Your comment here is very representative of how quickly people who are AI skeptics will jump on anything that supports their skepticism.
- ofjcihen 1y agoI don’t think the snark is warranted here. The person you’re responding to is linking an article dealing with that issue and the title is what they mentioned. Edit: SimonW? Really? I didn’t see the name but I didn’t expect you to be like that.
- refulgentis 1y agoNot OP, not sure what you mean but curious. - Snark? - Is "the issue" that anyone who claims any productivity gain is using magical thinking? - How does the linked article "deal with" "the issue"? - What title did they mention? - What did they link to that has that title? > Edit: SimonW? Really? I didn’t see the name but I didn’t expect you to be like that. Like what? I think you're getting a bit emotional & personal here, I don't read anything remotely inappropriate into Simon's comment. Been here 15 years. OP's was odd for HN in that it admits 0 argument: if you think you have productivity gains, it's magical thinking.
- gishglish 1y ago> so I will just outwork/outperform them. At the game of producing garbage slop? Probably yeah.
- quxbar 1y agoI now have the pleasure of giving exercises to candidates where they are explicitly allowed to use any AI or autocomplete that they want, but it's one of those tricky real-world problems where you'll only get yourself into trouble if you only follow the model's suggestions. It really separates the builders from the bureaucrats far more effectively than seeing who can whiteboard or leetcode.
- yomismoaqui 1y agoThat's really interesting... can you give more details about the problem you are using? This sounds like in there will be a race between this kind of booby trap tests and AIs learning them.
- gritzko 1y agoLong-tail problems are not reiterated in the dataset. Making LLM remember that can be difficult.
- elpakal 1y agoSome code challenge platforms allow for seeing how often someone pasted things in. That's been interesting.
- arealaccount 1y agoInteresting, care to elaborate? Or this is a carefully guarded secret?
- raydev 1y agoNot sharing what our coding questions are, but we also allow LLMs now. Interviewees choice to do so. In quite a few interviews in the last year I have come away convinced that they would have performed far better if they had relied on their own knowledge/experience exclusively. Fumbling with windows/tabs, not quite reading what they are copying, if I ask why they chose something, some of them would fold immediately and opt for something way better or more sensible, implying they would have known what to do had they bothered to actually think for a moment. I put down "no hire" for all of them of course.
- Lionga 1y agoIf you are so happy to let people think that AI does not yield productivity gains, why comment here? How exactly did you outperform? Show, don't talk.
- nayshins 1y agoI rolled out a migration to 60+ backends by using Claude code to manage it in the background. Simultaneously, I worked on other features while keeping my usual meeting load. I have more commits and releases per week than I have had in my whole career, which is objectively more productive.
- bluefirebrand 1y agoIf you are producing much lower quality then no, it is not objectively more productive
- nayshins 1y agoIf you find yourself working at the same place as me, feel free to judge my work, but until then, enjoy speculating.
- ajaisjdh 1y ago> I rolled out a migration to 60+ backends How is anyone supposed to understand what this means? Given the ambiguity in your description and lack of actual code it’s hard to take you seriously.
- seszett 1y agoSometimes when I read such meaningless things my first reaction is to feel like I'm too ignorant to understand what the person says. But then when I really think about it usually they're just bullshitting out being purposefully vague, using terms that don't mean anything precise in order to avoid actual criticism.
- dkn 1y ago
- haswell 1y agoThe issue I have with comments like this one is the one-dimensional notion of value described as "productivity gains" for a single person. There are many things in this world that could be fairly described as "more productive" or "faster" than the norm, yet few people would argue that it makes those things a net benefit. You can lie and cheat your way to success, and that tends to be successful too. There are good reasons society frowns on this. To me, focusing only on "I'm more productive" while ignoring the systemic and societal factors impacted by that "productivity" is completely missing the forest for the trees. The fact that you further feel that there isn't even a point in engaging on the topic is disturbing considering those ignored factors.
- masfuerte 1y agoYet here you are, engaging.
- nayshins 1y agothe art of the bait
- skeeter2020 1y ago>> so I will just outwork/outperform them. actually based on your own admission this is not what you're doing...
- throwawayqqq11 1y agoGreen or naturally grown brown field projects? People who boast about AI enhanced productivity seem to always forget to mention.
- thefz 1y agoUntil you will not have access to it and be outperformed by people used to thinking every day.
- archagon 1y agoUh, and who are you, exactly?
- asdev 1y agoI think AI greatly reduces the starting costs for mundane tasks/boilerplate, for a reduction in velocity in implementation. So possibly an illusion that programmers feel more productive. It could be that RPE(Rate of perceived exertion) is lower when using AI for tasks, but raw throughput may be higher if programmers just do the jobs themselves and get into productive/flow state.
- thewebguyd 1y ago> t could be that RPE(Rate of perceived exertion) is lower when using AI for tasks, but raw throughput may be higher if programmers just do the jobs themselves and get into productive/flow state. I think you're onto something with this take, based on my own experience. I definitely agree that my RPE seems lower when I'm using AI for things, whether it actually is making me more productive or not over the long term remains to be seen but things do certainly "feel" easier/less cognitively demanding. Which, tbh, is still a benefit even if it doesn't result in large gains in output. Putting in less cognitive load at work just conserves my energy for things that matter - everything else outside of $dayjob.
- codingwagie 1y agoAI is way underhyped, its just going to take society a long time to integrate this tectonic shift into our existing ways of working
- dr_dshiv 1y agoI agree it is underhyped. Many are like “oh, this is like blockchain, web3, ar/vr, quantum”— but whatever the promise of those technologies, AI is the only one with hundreds of millions of people productively using it everyday.
- compiler-guy 1y agoErr, billions of people use the web everyday in ways that make them vastly more productive. Looking up random docs is now trivial whereas before the web, if you were luck your library might have an old copy. And even then, the web was over-hyped at various points in its life. And it took a terrible crash to sort out what was real from what wasn’t. It’s not that the web wasn’t absolutely transformative (it was!) but it wasn’t what a huge number of proponents thought it would be either.
- elicash 1y agoI'm building things at a level of complexity I wouldn't have even attempted without AI. This piece, however, only focuses on time spent on a task that could be done both ways. Even there, it falls short. Let's assume this study is correct and a specific coding task does take me 19% more time with AI. I can still be more productive because the AI doing some of the work allows me to do other tasks during that time. I do worry about atrophy of my mind outsourcing too many tasks, admittedly. But that's a different issue.
- nromiun 1y agoSo much money is being pumped into this whole thing that we might get a global economic shock if/when it unravels.
- spaceman_2020 1y agoAll AI impact studies and research papers need to be taken with a pinch of salt. The field is moving so fast that by the time you get peer reviewed, you’re already outdated I’ve watched coding change from Cursor-esque IDEs to terminal based agentic tools within months.
- djhn 1y agoWhich also means spending a lot of time learning AI tools gets wasted as the tooling improves. I still suspect the vast silent majority of professional software devs haven’t integrated any, even Cursor-style, AI tools in to their main gig. And I reckon that’s completely rational, for those that have made this choice explicitly. Early adopters of AI tools are making a speculative bet, but so far most of them seem happy with the return.
- simonw 1y agoThat METR paper - discussed here two weeks ago: https://news.ycombinator.com/item?id=44522772 https://news.ycombinator.com/item?id=44522772 - has so much sticking power. It's not a bad paper, but it's also turning into a fantastic illustration of how much thirst there is out there for anything that shows that AI productivity doesn't work. I just learned there was a 4 minute TV news segment about it on CNBC! https://www.youtube.com/watch?v=WP4Ird7jZoA https://www.youtube.com/watch?v=WP4Ird7jZoA
- timr 1y ago> It's not a bad paper, but it's also turning into a fantastic illustration of how much thirst there is out there for anything that shows that AI productivity doesn't work. Maybe. I think it's a fantastic illustration of anyone doing anything to provide something other than hype around the subject. An actual RCT? Too good to be true! The thirst is for fact vs. speculation, influencer blogposts and self-promotion. That this RCT provided evidence opposing the hype, is, of course, irresistible.
- simonw 1y agoThat's what I liked about the paper: the methodology felt better to me than many other studies. I noted that when I blogged about it (second-to-last paragraph https://simonwillison.net/2025/Jul/12/ai-open-source-productivity/ https://simonwillison.net/2025/Jul/12/ai-open-source-product... )
- pydry 1y agoThe other papers ive seen so far all used toy projects. This paper was the first ive seen that deliberately used realistic projects. I think that part is understated, while the part about the "steep learning curve" has been considerably overstated. Ive watched plenty of good developers with hours of cursor practice, using plenty of relevant context giving clear instructions wasting chunks of 5-10 minutes at a time because 4/5 times claude produced crap that needed to be undone. That story doesnt sell though. Nobody will pay me consulting fees to say it. Hype has more $$$ attached.
- 1y ago
- GaggiX 1y agoThis article is based on the study that had a population of 16 people, most of which never used the tool before.
- impure 1y agoI find it funny that the first sentence is: I’m not sure if the whole AI thing is falling apart, but we are definitely in a phase where the hype doesn’t match what we are seeing in the real world. Also I do not think AI is falling apart and I do not think the productivity loss is surprising. It's something I've noticed a lot, you have to check AI output line by line thereby making you slower. AI makes tasks easier, not necessarily faster. And because people are lazy and love making things easier it's still going to be adopted no matter what anyone says.
- some_random 1y agoI'm excited for the AI-cope delusion to fall apart.
- mckngbrd 1y agoI had hardly written any code prior to ChatGPT other than teaching myself some VBA. Since then, using Gen AI tools to learn + write code, I have deployed a functioning full stack application to the web. NextJS on Vercel, with a backend server also deployed running on Python, and a Supabase DB. Is it the best application ever making loads of money? Definitely not. Are there things wrong with it? Absolutely (although I promise I'm not exposing sensitive env vars and API keys to the web). Did the first versions look like absolute ass as I clumsily figured things out and made bad mistakes? You bet. But it's a functioning app that does some useful things and has real users. I would never have imagined doing this in a million years prior to Gen AI. Do some devs see mixed results depending on how they're using the tools? I'm sure. Is Gen AI overhyped broadly speaking? Probably so. But when I see people say it's a delusion, waste of resources, and everyone is wasting their time on it ... for me, it just doesn't line up.
- DamnInteresting 1y ago> I have deployed a functioning full stack application to the web Best hope ChatGPT plagiarized that code from people who properly sanitized user input, otherwise it might be vulnerable to SQL injection, XSS, etc. If such holes exist, it may be tough to resolve them without a professional. > But when I see people say it's a delusion, waste of resources, and everyone is wasting their time on it ... for me, it just doesn't line up. I don't think anyone would say LLMs are useless, but there are people out there comparing the birth of LLMs to the Industrial Revolution, and the advent of the Internet. Such claims are preposterous. I think the widespread use of LLMs will ultimately be undone by a few factors: 1) Due to the way LLMs function, they will never be totally cured of 'hallucinations'. They are accurate a lot of the time, and apparently useful in those times. But you can never really trust its output. That is an exhausting problem, and will burn people out. 2) Prices for LLMs are artificially very low right now, these companies are burning investor money to keep going. The 'cost' is going to have to go way up for these LLM providers to be sustainable. I put cost in quotes because it will be some cost in subscription prices, but also cost in terms of ads that will be inserted everywhere, and other degrading forms of monetization. 3) Models are running out of good training data. They'll either hit a wall, or continue training on content that was itself LLM generated, which will go the way of Habsburg Jaw in a jiffy. Ensloppification is real. I'm sure there will be niches for LLMs in the future, for those who can afford their true cost. But this cram-it-in-everything, available-everywhere frenzy will probably be looked back upon with deserved embarrassment.
- OldGreenYodaGPT 1y agoHonestly, at this point, I'll just work 2 hours a day since I'm already producing 5-10x the output of my coworkers who aren't leveraging Claude Code at the medium-sized startup where I work. It really makes you wonder what's holding people back if they haven't at least doubled their productivity by now
- qsort 1y agoI don't think AI lives up to the current hype but this article is garbage. They're obviously talking about the METR paper, but the main takeaway according to the authors themselves was that self-reporting productivity increases is unreliable, not that you should cancel your subscription. Nothing in that paper said that AI can't speed up software engineering. Why are we responding to hype with nonsense?
- didibus 1y ago> Nothing in that paper said that AI can't speed up software engineering I mean, the paper did provide tangible data that at least in their experiments, AI slowed down software engineering. What they said is that it's not a proof that there's isn't a scenario or a mechanism where AI could result in speeding up software engineering. For that more research would be needed in measuring productivity of AI in more varied contexts. For me at least, their experiment seem to describe the average developer's use of AI. So it's probably telling you that currently on average AI might be slowing things down. Now the question is, can we find good data of outliers, and is it a simple matter of figuring out how to use it effectively, so we can upskill people and get the average to now be faster. Or will the outlier be conditioned on like, only for newbies, only for prototypes, only for the first X weeks on a greenfield code base, etc. Edit: That said, the most fascinating data point of that study is how software engineers are not able to determine if AI makes them faster or slower, because they all thought they were 20% faster but were 19% slower in reality. So now you have to become really skeptical of anyone who claims they found a methodology or a workflow where their use of AI makes them faster. We need better measurement than just "I feel faster".
- ken47 1y agoThe overarching problem is that signal:noise ratio in contemporary discussions of AI is absurdly low. This is thanks in large part to a media complex whose primary goal seems to be the pumping of AI investments and stocks, and the associated cacophony of mendacious exaggerations. If only we could have a reasonable discussion without the "contributions" of the non-technical and/or salespeople...
- Mars008 1y ago> “The results surprised us,” research lab METR reported. “Developers thought they were 20pc faster with AI tools, but they were actually 19pc slower when they had access to AI than when they didn’t.” They forgot about training and practice. Try it with airplanes. "The results surprised us. Given Boeing 777 they thought they will get from London to New York in 7 hours, but they actually couldn't even get off the ground. In fact they couldn't even start the engines."
- ikmckenz 1y ago> They forgot about training and practice. Try it with airplanes. "The results surprised us. Given Boeing 777 they thought they will get from London to New York in 7 hours, but they actually couldn't even get off the ground. In fact they couldn't even start the engines." If your "AI Assistant" is as hard to operate as a trans-Atlantic flight on a Boeing 777, perhaps it's not a very good assistant?
- Mars008 1y agoIt's a tool which is like others. It has limits and needs to get used to. From my experience first attempts are overoptimistic. I tried big chunks at once. Then I learned smaller tasks work better. But the most important thing is it can do simple things in languages that I don't know and don't want to learn (due to natural brain's limitations). Just need to have things done, like simple web front end for application running its own server on some microcontroller.
- deleted 1y ago[deleted]
- mvdtnz 1y ago> This is the question. If AI helps you complete tasks faster, but those tasks generate more work, are you being more productive? What an utterly bizarre question. Yes, by definition being more productive means doing more work.
- mikemac29 1y agoYou're misunderstanding the question. It's not that you get more work done, it's that the AI creates work that wasn't there before that is now required to be done. Exanple - AI generates an email faster than I could type it without question, but now I need to go line-by-line to make sure nothing in it is inaccurate before sending it, versus typing it myself and only putting in information I know is accurate. Which scenario takes more time? That's what I think many of the measures are missing. Too many are looking at how quickly the email is typed and calling it a gain, and ignoring the part where you review it. That might still be more productive, but I've yet to see a single AI sales demo that acknowledges this in a realistic way.
- lucaspauker 1y agoI think the productivity gains greatly depend on the coding task at hand. You could reasonably design a study that shows AI gains and slowdown depending on the task
- elicash 1y agoI agree with this. But also, part of what made the original study interesting is the DISCREPANCY between perceived vs actual productivity, as opposed to which method "won." It implies that even if some tasks are better with AI that it might not be so simple for us to judge which method is better for which task.
- hattmall 1y agoThere is really no argument that AI creates some productivity gains. Even if it's just an improved autocomplete. Because autocomplete does create some productivity gains. Pushing farther is murkier though. When it comes to the bulk of the type of work that AI is proving useful for, one of the main questions is why is there a need for speed. It's not in a sense of fear of automating jobs, it's just that generally we are reaching the bottlenecks more quickly and potentially causing more, but different problems, than we are solving. It's similar to the story of the development of vehicles and how even though we move much faster we spend a greater amount of time in transit. My mom used to lament how annoying it was to have to drive to the grocery store because when she was younger and not everyone had cars the store came to you. Twice a day, in the morning and the evening the "rolling store" would drive through the neighborhood and if they didn't have what you needed right then, they would bring it on the next trip. We are finally coming back full circle with things like Instacart but it's taken a solid ~60 years of largely wasted inefficient travel times.
- kokanator 1y agoAI is NOT going away. Spend serious time to learn how leveraging AI can multiply your output ( quality and quantity ) or get left behind. Practice. Improve. Deliver. Period. You’ve seen the posts and articles: “I’m inherently better than AI, coding faster and smarter.” Sure, maybe for now—especially if you’re half-assing your AI game. But that edge? It’s vanishing fast. Stop being so self indulgent. Become proficient with the tools available to you or get left behind.
- nayshins 1y agoMany have seen the writing on the wall, but many are refusing to believe it.
- NoGravitas 1y agoSounds the same as when NFT bros used to reply to skeptics with "enjoy being poor".
- nostromo 1y ago> Just because you can deal with your inbox more quickly doesn’t mean you’ll spend your afternoon on the beach. The more emails you fire off, the more you’ll receive back, and the never-ending cycle continues. I've seen this firsthand. It's so easy to produce "content" now, like emails, presentations, etc., that more of my time now is just sifting through AI slop looking for signal. I'm on a board for a nonprofit, and we've seen a person that is upset with the organization send dozens of AI-generated quasi-legal demands full of hallucinations of rules and laws that don't actually exist. So now we're forced to pay a real human lawyer to deal with a denial of service attack of ChatGPT slop and are spending lots of time just dealing with the onslaught. Our productivity has crashed as a board due to a single bad actor enabled by ChatGPT. At work, I'm also getting buried in AI sales calls and AI resumes. It's getting more and more difficult to find signal in the noise. It wouldn't surprise me if there's a lot of people who are experiencing lower productivity for similar reasons.
- thedonkeycometh 1y ago[dead]
- deleted 1y ago[deleted]
- returnInfinity 1y agoAI is improving my productivity on simple tasks. It only needs to improve productivity of the world by 1-10% to be worth it 10% would mean a 10 trillion in productivity, 1% is 1 trillion.
- slowmovintarget 1y agoPeople don't want car-washes that consist of camels licking the car? https://sourcegraph.com/blog/the-brute-squad https://sourcegraph.com/blog/the-brute-squad