4 ms·

> Researchers at MIT published a report showing that 95% of the generative AI programs launched by companies failed to do the main thing they were intended for

> Researchers at MIT published a report showing that 95% of the generative AI programs launched by companies failed to do the main thing they were intended for I think everyone had a gut feel for something along those lines, but those numbers are even starker than I would've imagined. Granted, many (most?) people trying to vibe code full apps don't know much about building software, so they're bound to struggle to get it to do what they want. But this quote is about companies and code they've actually put into production. Don't get me wrong, I've vibe coded a bunch of utilities that I now use daily, but 95% is way higher than I would've expected.

torben-friis 1y ago
I’ve heard the story that SQL was originally sold as a language that non tech people could use to query databases. It’s mostly an utter failure at that, yet still immensely popular. I’m expecting a similar future for AI, it will not deliver the “deprecating devs” part but it will still be a useful and ubiquitous tool.
- badgersnake 1y ago
  The hype cycle has priced in a paradigm shift, not gradual efficiency improvements. From that perspective a large correction is on the way.
- rsynnott 1y ago
  Yeah, it was a 4GL. Roughly since the 1950s, every ten years or so someone has come along with “this will allow unskilled people to write programs, and destroy those awful programmers forever” (_COBOL_ was originally basically marketed as this!) SQL is by _far_ the most successful thing to actually come out of this recurrent trend, in that it is actually useful, and unskilled people can actually use it to an extent. Most of the rest of it, 4GLs and 5GLs and drag and drop programming and no-code and… was just kinda useless; at most it made for a good demo, but attempts to make actual workable maintainable software with it broke down fast.
  - antonvs 1y ago
    It's interesting that there's been that constant drumbeat - for at least about half a century at this point - about eliminating software developers, yet somehow we don't get the same messaging about eliminating, say, civil engineers.
    - Disposal8433 1y ago
      > eliminating software developers I must write a "me too" here because I have seen this a lot recently on various sites. Whether it comes from managers or non-coders (I guess astrosurfing managers), it's always about those awful developers gate-keeping software development with their complicated compiled languages. I know it's all fake but it's exhausting, and it's nice to see it acknowledged here on HN.
      - antonvs 1y ago
        Illegitimi non carborundum
    - rsynnott 1y ago
      Don’t give them ideas. Whatever about vibe-coding, we really, really do not want vibe-railways.
- lomase 1y ago
  https://en.wikipedia.org/wiki/Fourth-generation_programming_language https://en.wikipedia.org/wiki/Fourth-generation_programming_... It was all the hype at the time, like LLMs are now. Most of them died because it was a bad idea. And the reason we still use some, like SQL, is not because of the sintaxt.
redbluered 1y ago
I think one issue is time. We're a few years in. It takes time to figure things out and see returns. The web and dot com boom and bust still led to several trillion dollar companies, eventually. AI will transform my industry, but not overnight. My employer is within that 95%... but won't be forever.
- lomase 1y ago
  Mobile phones was what changed society, not the web or the dot com era.
  - Jensson 1y ago
    Almost everyone had a computer in their home before we had smartphones, those computers did shape society in a massive way. You didn't see them on the streets like you do phones but the effects were still just as massive.
    - _DeadFred_ 1y ago
      'Almost everyone' was a very select group even in the 2000s. Look at reddit discourse post cheap postpaid internet phones versus before. The internet connected computer in the home was a productivity tool. Even just gaming required gamers to become pretty PC/OS/tech savvy. Cheap postpaid internet phones are bread and circuses. They two have different effects on society.
    - lomase 1y ago
      At the time I had to read slashdot on my work computer because I had no intenet or a new computer able to connect to it. Not even all geeks had it.
      - Jensson 1y ago
        Almost everyone, not everyone, majority of households in USA had a computer already by year 2000, and that is counting old people without kids who didn't keep up with trends. So by the time smart phones hit almost everyone had a computer at home. If you are talking about the 90s that isn't relevant, the relevant part is how smart phones changed things, and at that time internet was already available to a large majority at home, smart phones just made it portable.
forgotoldacc 1y ago
5% saying it's helping their company is roughly in line with the lizard man constant. [1] There will always be people who will never admit a thing didn't work out as planned and those who just like to answer sarcastically. It's not unreasonable to assume that, if this report is even remotely accurate, it's pretty much 100% of people finding AI fairly disappointing. [1] https://en.m.wiktionary.org/wiki/Lizardman%27s_Constant https://en.m.wiktionary.org/wiki/Lizardman%27s_Constant
- rsynnott 1y ago
  I mean, I assume the blogspam industry is thrilled with it… For the time being, and the foreseeable future, LLM’s sweet spot seems to be low-grade translation, and ultra-low-grade bottom barrel ‘content generation’. Which is… not nothing, but also not what you’d call world-changing. As a number of people said, there probably is an industry here; it’s just that it’s worth on the order of tens of billions, not trillions as the markets currently appear to believe. (Some people will claim it’s a great programming tool. Personally sceptical, but even if it’s the greatest, most amazingest programming tool ever, well, “we might be even more important than Borland and Jetbrains were” is not going to thrill the markets too much. Current valuations are built on mass-market applicability, and if that doesn’t show up soon there will be trouble.)
  - Ekaros 1y ago
    Also image generation. If you just need to fill space on web pages. Generating a few images for free/very cheap is cheaper than paying for stock images. And at that price do you really care if you do not have copy right? Especially if everything else is LLM generated slop as well.
    - rsynnott 1y ago
      Okay, so, “we’re as important as Borland, Jetbrains, AND Getty Images! That’ll be a trillion dollars investment money, pls”. Like, that’s still just silly. On the developer tools thing in particular, I’d note that it is historically extremely difficult to make a sustainable business, nevermind a wildly profitable business, in that space. Borland and Jetbrains probably are the closest that anyone has come.
luckylion 1y ago
> Despite the rush to integrate powerful new models, about 5% of AI pilot programs achieve rapid revenue acceleration; the vast majority stall, delivering little to no measurable impact on P&L. [1] "We wanted to make money with it, but we didn't immediately make a lot of money" feels very different from "the project failed to deliver what it set out to". [1] https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/ https://fortune.com/2025/08/18/mit-report-95-percent-generat...
Traubenfuchs 1y ago
You make „utilities“, those initiatives are about replacing complex processes, business-people-engineer communication and the engineers themselves with AI. e.g. I waste a lot of time with converting business requirements into a proprietary rule language. It should be simple tasks, but the requirements are freaky, the language is limited and I often need to look up internals of systems that produce data the rules act upon. My bosses boss currently wants me to replace my work with AI. It can not work. It‘s setup for failure.
antonvs 1y ago
It's roughly in line with Sturgeon's Law: 90% of everything is crap. Except in this case, where AI can enable people with absolutely no experience in some area to produce something that at least superficially can seem plausibly viable, it's no surprise that the percentage of crap is even higher.
- uncircle 1y ago
  This is an interesting point. As LLMs are not intelligent and trained on what already exists, their output is necessarily mediocre if not bad. We simply have found a way to increase the amount of crap in the digital world, to the point that Sturgeon’s 90% will become a very low estimate.
lomase 1y ago
What I don't really underdstan is how having 10 juniors in a team, all the agentic code stuff, makes a programmer more productive.
- ricardobeat 1y ago
  You don’t understand how having an extra ten programmers in your team can be productive? Junior developers require guidance but are still producing value. And with good guidance, they will do amazing work.
  - linker3000 1y ago
    Here's one possibility. With AI we need fewer programmers, and the juniors will possibly be the first to go, but they might me retrained for other careers (which might eventually get cancelled too because of AI), or out of work. The software they produced did something - it might have been a CRM or a game, but out of work people might have to cut back on their gaming spend. As for the CRM app business, the customers and potential software customers are also cutting back in staff, and the CRM apps will be able to conduct direct B2B negotiations with client CRMs, so there's no job opportunities there, and so more people are out of work. Perhaps the businesses that used the AI-based B2B and B2C CRM and ERP systems won't be needed any more, or not have a viable customer base, too. Other industries are replacing folks with 'AI', so the unemployment pool is getting larger. This means the luxury and non-vital goods manufacturers will have less revenue and they are laying off staff so there's some compensation there, but eventually not enough for survival - which is 'fine' because AI is replacing all this stuff. This snowballs into other industries, leaving just those jobs that can be done more easily by a human, but those jobs will also reduce as AI and surrounding robotics etc improve, so what do all these unemployed people do all day. Some will embrace leisure activities that don't break the bank. Some may volunteer for community work or projects to improve the World, but they still need to eat and pay bills - who's going to help with that? One solution might be a 'Star Trek' economy not based on work for reward, but that's a big cultural shift that people and governments will struggle massively to get their heads around conceptually. There will also be powerful resistance to such a radical rebasing of the planet-wide financial model, especially by those people and organisations that have amassed wealth and don't want to give it up. They'll even fight back with lobbying and arguments against change while they're getting replaced with AI. Or...?
chriskanan 1y ago
Read the paper. The media is not providing a lot of missing context. The paper points out problems like leadership failures for those efforts, lack of employee buy-in (potentially because they use their personal LLM), etc. A huge fraction of people at my work use LLMs, but only a small fraction use the LLM they provided. Almost everyone is using a personal license

torben-friis 1y ago

I’ve heard the story that SQL was originally sold as a language that non tech people could use to query databases. It’s mostly an utter failure at that, yet still immensely popular. I’m expecting a similar future for AI, it will not deliver the “deprecating devs” part but it will still be a useful and ubiquitous tool.

badgersnake 1y ago
The hype cycle has priced in a paradigm shift, not gradual efficiency improvements. From that perspective a large correction is on the way.
rsynnott 1y ago
Yeah, it was a 4GL. Roughly since the 1950s, every ten years or so someone has come along with “this will allow unskilled people to write programs, and destroy those awful programmers forever” (_COBOL_ was originally basically marketed as this!) SQL is by _far_ the most successful thing to actually come out of this recurrent trend, in that it is actually useful, and unskilled people can actually use it to an extent. Most of the rest of it, 4GLs and 5GLs and drag and drop programming and no-code and… was just kinda useless; at most it made for a good demo, but attempts to make actual workable maintainable software with it broke down fast.
- antonvs 1y ago
  It's interesting that there's been that constant drumbeat - for at least about half a century at this point - about eliminating software developers, yet somehow we don't get the same messaging about eliminating, say, civil engineers.
  - Disposal8433 1y ago
    > eliminating software developers I must write a "me too" here because I have seen this a lot recently on various sites. Whether it comes from managers or non-coders (I guess astrosurfing managers), it's always about those awful developers gate-keeping software development with their complicated compiled languages. I know it's all fake but it's exhausting, and it's nice to see it acknowledged here on HN.
    - antonvs 1y ago
      Illegitimi non carborundum
  - rsynnott 1y ago
    Don’t give them ideas. Whatever about vibe-coding, we really, really do not want vibe-railways.
lomase 1y ago
https://en.wikipedia.org/wiki/Fourth-generation_programming_language https://en.wikipedia.org/wiki/Fourth-generation_programming_... It was all the hype at the time, like LLMs are now. Most of them died because it was a bad idea. And the reason we still use some, like SQL, is not because of the sintaxt.

redbluered 1y ago

I think one issue is time. We're a few years in. It takes time to figure things out and see returns. The web and dot com boom and bust still led to several trillion dollar companies, eventually. AI will transform my industry, but not overnight. My employer is within that 95%... but won't be forever.

lomase 1y ago
Mobile phones was what changed society, not the web or the dot com era.
- Jensson 1y ago
  Almost everyone had a computer in their home before we had smartphones, those computers did shape society in a massive way. You didn't see them on the streets like you do phones but the effects were still just as massive.
  - _DeadFred_ 1y ago
    'Almost everyone' was a very select group even in the 2000s. Look at reddit discourse post cheap postpaid internet phones versus before. The internet connected computer in the home was a productivity tool. Even just gaming required gamers to become pretty PC/OS/tech savvy. Cheap postpaid internet phones are bread and circuses. They two have different effects on society.
  - lomase 1y ago
    At the time I had to read slashdot on my work computer because I had no intenet or a new computer able to connect to it. Not even all geeks had it.
    - Jensson 1y ago
      Almost everyone, not everyone, majority of households in USA had a computer already by year 2000, and that is counting old people without kids who didn't keep up with trends. So by the time smart phones hit almost everyone had a computer at home. If you are talking about the 90s that isn't relevant, the relevant part is how smart phones changed things, and at that time internet was already available to a large majority at home, smart phones just made it portable.

forgotoldacc 1y ago

5% saying it's helping their company is roughly in line with the lizard man constant. [1] There will always be people who will never admit a thing didn't work out as planned and those who just like to answer sarcastically. It's not unreasonable to assume that, if this report is even remotely accurate, it's pretty much 100% of people finding AI fairly disappointing. [1] https://en.m.wiktionary.org/wiki/Lizardman%27s_Constant https://en.m.wiktionary.org/wiki/Lizardman%27s_Constant

rsynnott 1y ago
I mean, I assume the blogspam industry is thrilled with it… For the time being, and the foreseeable future, LLM’s sweet spot seems to be low-grade translation, and ultra-low-grade bottom barrel ‘content generation’. Which is… not nothing, but also not what you’d call world-changing. As a number of people said, there probably is an industry here; it’s just that it’s worth on the order of tens of billions, not trillions as the markets currently appear to believe. (Some people will claim it’s a great programming tool. Personally sceptical, but even if it’s the greatest, most amazingest programming tool ever, well, “we might be even more important than Borland and Jetbrains were” is not going to thrill the markets too much. Current valuations are built on mass-market applicability, and if that doesn’t show up soon there will be trouble.)
- Ekaros 1y ago
  Also image generation. If you just need to fill space on web pages. Generating a few images for free/very cheap is cheaper than paying for stock images. And at that price do you really care if you do not have copy right? Especially if everything else is LLM generated slop as well.
  - rsynnott 1y ago
    Okay, so, “we’re as important as Borland, Jetbrains, AND Getty Images! That’ll be a trillion dollars investment money, pls”. Like, that’s still just silly. On the developer tools thing in particular, I’d note that it is historically extremely difficult to make a sustainable business, nevermind a wildly profitable business, in that space. Borland and Jetbrains probably are the closest that anyone has come.

luckylion 1y ago

> Despite the rush to integrate powerful new models, about 5% of AI pilot programs achieve rapid revenue acceleration; the vast majority stall, delivering little to no measurable impact on P&L. [1] "We wanted to make money with it, but we didn't immediately make a lot of money" feels very different from "the project failed to deliver what it set out to". [1] https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/ https://fortune.com/2025/08/18/mit-report-95-percent-generat...

Traubenfuchs 1y ago

You make „utilities“, those initiatives are about replacing complex processes, business-people-engineer communication and the engineers themselves with AI. e.g. I waste a lot of time with converting business requirements into a proprietary rule language. It should be simple tasks, but the requirements are freaky, the language is limited and I often need to look up internals of systems that produce data the rules act upon. My bosses boss currently wants me to replace my work with AI. It can not work. It‘s setup for failure.

antonvs 1y ago

It's roughly in line with Sturgeon's Law: 90% of everything is crap. Except in this case, where AI can enable people with absolutely no experience in some area to produce something that at least superficially can seem plausibly viable, it's no surprise that the percentage of crap is even higher.

uncircle 1y ago
This is an interesting point. As LLMs are not intelligent and trained on what already exists, their output is necessarily mediocre if not bad. We simply have found a way to increase the amount of crap in the digital world, to the point that Sturgeon’s 90% will become a very low estimate.

lomase 1y ago

What I don't really underdstan is how having 10 juniors in a team, all the agentic code stuff, makes a programmer more productive.

ricardobeat 1y ago
You don’t understand how having an extra ten programmers in your team can be productive? Junior developers require guidance but are still producing value. And with good guidance, they will do amazing work.
- linker3000 1y ago
  Here's one possibility. With AI we need fewer programmers, and the juniors will possibly be the first to go, but they might me retrained for other careers (which might eventually get cancelled too because of AI), or out of work. The software they produced did something - it might have been a CRM or a game, but out of work people might have to cut back on their gaming spend. As for the CRM app business, the customers and potential software customers are also cutting back in staff, and the CRM apps will be able to conduct direct B2B negotiations with client CRMs, so there's no job opportunities there, and so more people are out of work. Perhaps the businesses that used the AI-based B2B and B2C CRM and ERP systems won't be needed any more, or not have a viable customer base, too. Other industries are replacing folks with 'AI', so the unemployment pool is getting larger. This means the luxury and non-vital goods manufacturers will have less revenue and they are laying off staff so there's some compensation there, but eventually not enough for survival - which is 'fine' because AI is replacing all this stuff. This snowballs into other industries, leaving just those jobs that can be done more easily by a human, but those jobs will also reduce as AI and surrounding robotics etc improve, so what do all these unemployed people do all day. Some will embrace leisure activities that don't break the bank. Some may volunteer for community work or projects to improve the World, but they still need to eat and pay bills - who's going to help with that? One solution might be a 'Star Trek' economy not based on work for reward, but that's a big cultural shift that people and governments will struggle massively to get their heads around conceptually. There will also be powerful resistance to such a radical rebasing of the planet-wide financial model, especially by those people and organisations that have amassed wealth and don't want to give it up. They'll even fight back with lobbying and arguments against change while they're getting replaced with AI. Or...?

chriskanan 1y ago

Read the paper. The media is not providing a lot of missing context. The paper points out problems like leadership failures for those efforts, lack of employee buy-in (potentially because they use their personal LLM), etc. A huge fraction of people at my work use LLMs, but only a small fraction use the LLM they provided. Almost everyone is using a personal license