13 ms·

A new AI winter is coming?

199 points by voxleone 11mo ago

deadbabe 11mo ago
Modern LLMs could be like the equivalent of those steam powered toys the Romans had in their time. Steam tech went through a long winter before finally being fully utilized for the Industrial Revolution. We’ll probably never see the true AI revolution in our lifetime, only glimpses of what could be, through toys like LLMs. What we should underscore though, is that even if there is a new AI winter, the world isn’t going back to what it was before AI. This is it, forever. Generations ahead will gaslight themselves into thinking this AI world is better, because who wants to grow up knowing they live in a shitty era full of slop? Don’t believe it.
- 7thaccount 11mo ago
  LLMs are useful tools, but certainly have big limitations. I think we'll continue to see anything be automated that can be automated in a way that reduces head count. So you have the dumb AI as a first line of defense and lay off half the customer service you had before. In the meantime, fewer and fewer jobs (especially entry level), a rising poor class as the middle class is eliminated and a greater wealth gap than ever before. The markets are going to also collapse from this AI bubble. It's just a matter of when.
- cardanome 11mo ago
  The development of LLM's required access to huge amounts of decent training data. It could very well that the current generation of AI has poisoned the well for any future endeavors of creating AI. You can't trivially filter out the AI slop and humans are less likely to make their handcrafted content freely available for training. In fact violating GPL code to train models on it might be ruled to be illegal as well generally stricter rules on which data you are allowed to use for training. We might have reached a local optimum that is very difficult to escape from. There might be a long, long AI winter ahead of us, for better or worse. > the world isn’t going back to what it was before AI. This is it, forever. I feel this so much. I though my longing for the pre-smartphone days was bad but damn we have lost so much.
  - esafak 11mo ago
    That would create the impetus to create a training data marketplace.
    - deadbabe 11mo ago
      Maybe the path to redemption for AI stealing jobs would be if people would have to be rehired en masse to write and produce art, about whatever they want, so that it can be used to train more advanced AI.
- rnxrx 11mo ago
  The development of steam technology is a great metaphor. The basic understanding of steam as a thing that could yield some kind of mechanical force almost certainly predated even the Romans. That said, it was the synthesis of other technologies with these basic concepts that started yielding really interesting results. Put another way, the advent of industrialized steam power wasn't so much about steam per se, but rather the intersection of a number of factors (steam itself obviously being an important one). This intersection became a lot more likely as the pace of innovation in general began accelerating with the Enlightenment and the ease with which this information could be collected and synthesized. I suspect that the LLM itself may also prove to be less significant than the density of innovation and information of the world it's developed in. It's not a certainty that there's a killer app on the scale of mechanized steam, but the odds of such significant inventions arguably increase as the basics of modern AI become basic knowledge for more and more people.
  - orwin 11mo ago
    Its mostly metallurgy. The fact that we became so much better and precise at metallurgy enabled us to make use of steam machines. Of course a lot of stuff helped (Glassmaking, whale oil immediatly come to mind) but mostly, metallurgy.
    - int_19h 11mo ago
      I remember reading an article that argued that it was basically a matter of being path dependent. The earliest steam engines that could do useful work were notoriously large and fuel-inefficient, which is why their first application was for pumps in coal mines - it effectively made the fuel problem moot and similarly their other limitations were not important in that context, while at the same time rising wages in UK made even those inefficient engines more affordable than manual labor. And then their use in that very narrow niche allowed them to be gradually improved to the point where they became suitable for other contexts as well. But if that analogy holds, then LLM use in software development is the "new coal mines" where it will be perfected until it spills over into other areas. We're definitely not at the "Roman stage" anymore.
kunley 11mo ago
Can't wait
teaearlgraycold 11mo ago
LLMs have failed to live up to the hype, but they haven't failed outright.
- HardCodedBias 11mo ago
  Two claims here: 1) LLMs have failed to live up to the hype. Maybe. Depends upon's who's hype. But I think it is fine to say that we don't have AGI today (however that is defined) and that some people hyped that up. 2) LLMs haven't failed outright I think that this is a vast understatement. LLMs have been a wild success. At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is larger. ChatGPT has over 800 million weekly active users. Students throughout the world, and especially in the developed world are using "AI" at 85-90% (from some surveys). Between 40% of professionals and 90% (depending upon survey and profession) are using "AI". This is 3 years after the launch of ChatGPT (and the capabilities of chatGPT 3.5 were so limited compared to today that it is a shame that they get bundled together in our discussions). I would say instead of "failed outright" that they are the most successful consumer product of all time (so far).
  - baseballdork 11mo ago
    > At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is larger. I have a really hard time believing that stat without any context, is there a source for this?
    - Groxx 11mo ago
      from what I've seen in a several-thousand-eng company: LLMs generally produce vastly more code than is necessary, so they quickly out-pace human coders. they could easily be producing half or more of all of the code even if only 10% of the teams use it. particularly because huge changes often get approved with just a "lgtm", and LLM-coding teams also often use/trust LLMs for reviews. but they do that while making the codebase substantially worse for the next person or LLM. large code size, inconsistent behavior, duplicates of duplicates of duplicates strewn everywhere with little to no pattern so you might have to fix something a dozen times in a dozen ways for a dozen reasons before it actually works, nothing handles it efficiently. the only thing that matters in a business is value produced, and I'm far from convinced that they're even break-even if they were free in most cases. they're burning the future with tech debt, on the hopes that it will be able to handle it where humans cannot, which does not seem true at all to me.
moltar 11mo ago
> The technology is essentially a failure Really? I derive a ton of value from it. For me it’s a phenomenal advancement and not a failure at all.
deleted 11mo ago
[deleted]
tarr11 11mo ago
This has convinced many non-programmers that they can program, but the results are consistently disastrous, because it still requires genuine expertise to spot the hallucinations. I've been programming for 30+ years and now a people manager. Claude Code has enabled me to code again and I'm several times more productive than I ever was as an IC in the 2000s and 2010s. I suspect this person hasn't really tried the most recent generation, it is quite impressive and works very well if you do know what you are doing
- agubelu 11mo ago
  Isn't that what the author means? "it still requires genuine expertise to spot the hallucinations" "works very well if you do know what you are doing"
  - hombre_fatal 11mo ago
    But it can work well even if you don't know what you are doing (or don't look at the impl). For example, build a TUI or GUI with Claude Code while only giving it feedback on the UX/QA side. I've done it many times despite 20 years of software experience. -- Some stuff just doesn't justify me spending my time credentializing in the impl. Hallucinations that lead to code that doesn't work just get fixed. Most code I write isn't like "now write an accurate technical essay about hamsters" where hallucinations can sneak through lest I scrutinize it; rather the code would just fail to work and trigger the LLM's feedback loop to fix it when it tries to run/lint/compile/typecheck it. But the idea that you can only build with LLMs if you have a software engineer copilot isn't true and inches further away from true every month, so it kinda sounds like a convenient lie we tell ourselves as engineers (and understandably so: it's scary).
    - int_19h 11mo ago
      > Hallucinations that lead to code that doesn't work just get fixed How about hallucinations that lead to code that doesn't work outside of the specific conditions that happen to be true in your dev environment? Or, even more subtly, hallucinations that lead to code which works but has critical security vulnerabilities?
ZeroConcerns 11mo ago
Well, the original "AI winter" was caused by defense contracts running out without anything to show for it -- turns out, the generals of the time could only be fooled by Eliza clones for so long... The current AI hype is fueled by public markets, and as they found out during the pandemic, the first one to blink and acknowledge the elephant in the room loses, bigly. So, even in the face of a devastating demonstration of "AI" ineffectiveness (which I personally haven't seen, despite things being, well, entirely underwhelming), we may very well stuck in this cycle for a while yet...
- skerit 11mo ago
  > despite things being, well, entirely underwhelming People use LLMs for all kinds of things, but for coding it is absolutely not underwhelming. Can you treat it as a real independent developer, one that doesn't need supervision? No. Can it save you hours and hours of work? Yes.
citizenpaul 11mo ago
>Expect OpenAI to crash, hard, with investors losing their shirts. Lol someone doesn't understand how the power structure system works "the golden rule". There is a saying if you owe the bank 100k you have a problem. If you owe the bank ten million the bank has a problem. OpenAI and the other players have made this bubble so big that there is no way the power system will allow themselves to take the hit. Expect some sort of tax subsided bailout in the near future.
- AnimalMuppet 11mo ago
  The difference is that OpenAI isn't financed by borrowed money.
- dragonwriter 11mo ago
  Tax subsized bailouts usually are structured to protect the corporation as a separate entity, creditors (compared to letting the org continue on its unimpeded path), employees, and sometimes, beyond their position as current creditors, suppliers, but not as often do they protect existing equity holders.
aroman 11mo ago
When the hype is infinite (technological singularity and utopia), any reality will be a let down. But there is so much real economic value being created - not speculation, but actual business processes - billions of dollars - it’s hard to seriously defend the claim that LLMs are “failures” in any practical sense. Doesn’t mean we aren’t headed for a winter of sobering reality… but it doesn’t invalidate the disruption either.
- emp17344 11mo ago
  Other than inflated tech stocks making money off the promise of AI, what real economic impact has it actually had? I recall plenty of articles claiming that companies are having trouble actually manifesting the promised ROI.
  - phantasmish 11mo ago
    My company’s spending a lot of money doing things they could have done fifteen or more years ago with classic computer vision libraries and other pre-LLM techniques, cheaper and faster. Most of the value of AI for our organization is the hype itself providing the activation energy, if you will, to make these projects happen. The value added by the AI systems per se has been minimal. (YMMV but that’s what I’m seeing at the non-tech bigco I’m at—it’s pretty silly but the checks continue to clear so whatever)
- n4r9 11mo ago
  > not speculation, but actual business processes Is there really a clear-cut distinction between the two in today's VC and acquisition based economy?
- api 11mo ago
  This is why I hate hype. "We just cured cancer! All cancer! With a simple pill!" "But you promised it would rejuvenate everyone to the metabolism of a 20 year old and make us biologically immortal!" New headline: "After spending billions, project to achieve immortality has little to show..."
- keybored 11mo ago
stanfordkid 11mo ago
This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal models entirely. He also pretty much discounts everything that’s happened with AlphaFold, Alpha Go etc. reinforcement learning etc. The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why. Is the brain NP complete? Maybe maybe not. I could see many arguments about why modern research will fail to create AGI but just hand waving “reality is NP-hard” is not enough. The fact is: something fundamental has changed that enables a computer to pretty effectively understand natural language. That’s a discovery on the scale of the internet or google search and shouldn’t be discounted… and usage proves it. In 2 years there is a platform with billions of users. On top of that huge fields of new research are making leaps and bounds with novel methods utilizing AI for chemistry, computational geometry, biology etc. It’s a paradigm shift.
- gishh 11mo ago
  > something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't have an agenda.
  - sukhdeepprashit 11mo ago
    [dead]
  - danielvaughn 11mo ago
    I think it’s a disingenuous read to assume original commenter means “understanding” in the literal sense. When we talk about LLM “understanding”, we usually mean it from a practical sense. If you give an input to the computer, and it gives you an expected output, then colloquially the computer “understood” your input.
paganel 11mo ago
I've been reading about this supposed AI winter for at least 5 years by now, and in the meantime any AI-related stock has gone 10x and more.
- _se 11mo ago
  5 years? That's so interesting bro. Tell us more about what your thoughts on ChatGPT were back in 2020. Go ahead and double check when the LLM craze started and perhaps reconsider making things up.
  - paganel 11mo ago
    May 2018: AI winter is well on its way (piekniewski.info) [1] January 2020: Researchers: Are we on the cusp of an ‘AI winter’? (bbc.co.uk) [2] I'm sure you can easily find more. Felt good to be called a "bro", though, made me feel younger. [1] HN discussion, almost 500 comments: https://news.ycombinator.com/item?id=17184054 https://news.ycombinator.com/item?id=17184054 [2] HN discussion on BBC article, ~110 comments: https://news.ycombinator.com/item?id=22069204 https://news.ycombinator.com/item?id=22069204
neom 11mo ago
Blog posts like this make me think model adoption and appropriate use case for the model is...lumpy at best. Every time I read something like it I wonder what tools they are using and how? Modern systems are not raw transformers. A raw transformer will “always output something,” they're right, but nobody deploys naked transformers. This is like claiming CPUs can’t do long division because the ALU doesn’t natively understand decimals. Also, a model is stat aprox trained on the empirical distribution of human knowledge work. It is not trying to compute the exact solution to NP complete problems? Nature does not require worst case complexity, real world cognitive tasks are not worst case NP hardness instances...
numbers_guy 11mo ago
AlexNet was only released in 2011. The progress made in just 14 years has been insane. So while I do agree that we are approaching a "back to the drawing board" era, calling the past 14 years a "failure" is just not right.
cmiles8 11mo ago
LLMs are an amazing advancement. The tech side of things is very impressive. Credit where credit is due. Where the current wave all falls apart is on the financials. None of that makes any sense and there’s no obvious path forward. Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Tech wise I’m bullish. Business wise, AI is setting up to be a big disaster. Those that aimlessly chased the hype are heading for a world of financial pain.
- famouswaffles 11mo ago
  >Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Ok, so I think there's 2 things here that people get mixed on. First, Inference of the current state of the art is Cheap now. There's no 2 ways about it. Statements from Google, Altman as well as costs of 3rd parties selling tokens of top tier open source models paint a pretty good picture. Ads would be enough to make Open AI a profitable company selling current SOTA LLMs to consumers. Here's the other thing that mixes things up. Right now, Open AI is not just trying to be 'a profitable company'. They're not just trying to stay where they are and build a regular business off it. They are trying to build and serve 'AGI', or as they define it, 'highly autonomous systems that outperform humans at most economically valuable work'. They believe that, to build and serve this machine to hundreds of millions would require costs order(s) of magnitudes greater. In service of that purpose is where all the 'insane' levels of money is moving to. They don't need hundreds of billions of dollars in data centers to stay afloat or be profitable. If they manage to build this machine, then those costs don't matter, and if things are not working out midway, they can just drop the quest. They will still have an insanely useful product that is already used by hundreds of millions every week, as well as the margins and unit economics to actually make money off of it.
  - cmiles8 11mo ago
    If OpenAI was the only company doing this that argument might sort of make sense. The problem is they have real competition now and that market now looks like an expensive race to an undifferentiated bottom. If someone truly invents AGI and it’s not easily copied by others then I agree it’s a whole new ballgame. The reality is that years into this we seem to be hitting a limit to what LLMs can do with only marginal improvements with each release. On that path this get ugly fast.
bgwalter 11mo ago
The poster is right. LLMs are Gish Gallop machines that produce convincing sounding output. People have figured it out by now. Generative "AI" will fail, other forms may continue, though it it would be interesting to hear from experts in other fields how much fraud there is. There are tons of material science "AI" startups, it is hard to believe they all deliver.
- Dilettante_ 11mo ago
  >produce convincing sounding output Well, correctness(though not only correctness) sounds convincing, the most convincing even, and ought to be information-theory-wise cheaper to generate than a fabrication, I think. So if this assumption holds, the current tech might have some ceiling left if we just continue to pour resources down the hole.
  - knollimar 11mo ago
    How do LLMs do on things that are common confusions? Do they specifically have to be trained against them? I'm imagining a Monty Hall problem that isn't in the training set tripping them up the same way a full wine glass does
Barathkanna 11mo ago
Interesting take. His argument is basically that LLMs have hit their architectural ceiling and the industry is running on hype and unsustainable economics. I’m not fully convinced, but the points about rising costs and diminishing returns are worth paying attention to. The gap between what these models can actually do and what they’re marketed as might become a real problem if progress slows.
- ryanjshaw 11mo ago
  The existence of an AI hype train doesn’t mean there isn’t a productive AI no-hype train. Context: I have been writing software for 30 years. I taught myself assembly language and hacked games/apps as a kid, and have been a professional developer for 20 years. I’m not a noob. I’m currently building a real-time research and alerting side project using a little army of assistant AI developers. Given a choice, I would never go back to how I developed software before this. That isn’t my mind poisoned by hype and marketing.
  - Abekkus 11mo ago
    If you're using cloud AI assistants, they're being heavily discounted to you by investors, for now. Those discounts won't exist forever.
- DrewADesign 11mo ago
  I think the unsustainably cheap consumer-facing AI products are the spoonful of sugar getting us to swallow a technology that will almost entirely be used to make agents that justify mass layoffs.
  - buellerbueller 11mo ago
    And surveil and rat on us, sometimes incorrectly.
    - DrewADesign 11mo ago
      Yeah that’s a good point. I’m always astonished that people give anything even vaguely personal to a “free” saas service, but boy do they ever. Not that payments stop them but at least the incentives are a little different.
LarsDu88 11mo ago
Zero citations, random speculations. Random blogpost versus the most rapidly adopted technology in history...
- kunley 11mo ago
  This statement strikes me... but not the way you intended. We should say: most rapidly adopted ... speculation. Because this is what it is: not a technology, but speculation. Hint: technology has repeatable results. QED ;)
uejfiweun 11mo ago
"The technology is essentially a failure" is in the headline of this article. I have to disagree with that. For the first time in the history of the UNIVERSE, an entity exists that can converse in human language at the same level that humans can. But that's me being a sucker. Because in reality this is just a clickbait headline for an article basically saying that the tech won't fully get us to AGI and that the bubble will likely pop and only a few players will remain. Which I completely agree with. It's really not that profound.
abroszka33 11mo ago
I agree, I have over 20 years of software engineering experience and after vibe coding/engineering/architecting (or whatever you want to call it) for a couple months, I also don't see the technology progressing further. LLMs are more or less the same as 6 months ago, incremental improvements, but no meaningful progress. And what we have is just bad. I can use it because I know the exact code I want generated and I will just re-prompt if I don't get what I want, but I'm unconvinced that this is faster than a good search engine and writing code myself. I think I will keep using it while it's cheap, but once I have to pay the real costs of training/running a flagship modell I think I will quit. It's too expensive as it is for what it does.
- esafak 11mo ago
  Yet, the real cost of inference for a given model complexity is dropping.
- benjiro 11mo ago
  > LLMs are more or less the same as 6 months ago, incremental improvements, but no meaningful progress. Go back to a older version of a LLM and then say the same. You will notice that older LLM versions do less, have more issues, write worse code etc... There have been large jumps in the last 2 years but because its not like we go from a LLM to AGI, that people underestimate the gains. Trust me, try it, try Claude 3.7 > 4.0 > 4.5 > Opus 4.5 ... > I'm unconvinced that this is faster than a good search engine and writing code myself. What i see is mostly somebody who is standoffish on LLMs ... Just like i was. I tried to shoehorn their code generation into "my" code, and while it works reasonably, you tend to see the LLM working on "your code", as a invasion. So you never really use its full capabilities. LLMs really work the best, if you plan, plan, plan, and then have them create the code for you. The moment you try to get the LLMs work inside existing code, that is especially structured how YOU like it, people tend to be more standoffish. > It's too expensive as it is for what it does. CoPilot is like 27 Euro/month(year payment + dollar/euro) for 1500 requests here. We pay just for basic 100Mbit internet 45 Euro per month. I mean ... Will it get more expensive in the future, o yes, for sure. But we may also have alternatives by then. Open Source/Open Weight models are getting better and better, especially with MoE. Pricing is how you look at it... If i do the work what takes me a months in a few days, what is the price then? If i do work that i normally need to outsource. Or code that is some monotoon repeating end me %@#$ ... in a short time by paying a few cents to a LLM, ... Reality is, thing change, and just like the farmers that complained about the tractor, while their neighbor now did much more work thanks to that thing, LLMs are the same. I often see the most resistance from us older programmer folks, who are set in our ways, when its us who actually are the best at wrangling LLMs the best. As we have the experience to fast spot where a LLM goes wrong, and guide it down the right path. Tell it, the direction is debugging is totally wrong and where the bug more likely is ... For the last 2 years i paid just the basic cheap $10 subscription, and use it but never strongly. It helped with those monotoon tasks etc. Until ... a months ago, i needed a specific new large project and decided to just agent / vibe code it, just to try at first. And THEN i realized, how much i was missing out off. Yes, that was not "my" code, and when the click came in my head that "i am the manager, not the programmer", you suddenly gain a lot. Its that click that is hard for most seasoned veterans. And its ironically often the seasoned guys that complain the most about AI ... when its the same folks that can get the most out of LLMs.
binary132 11mo ago
what a curious coincidence that a soft-hard AI landing would happen to begin at the exact same time as the US government launches a Totally Not a Bailout Strategic Investment Plan Bro I Promise. who could have predicted this?
coffeecoders 11mo ago
I am of a belief that upcoming winter will look more like normalization than collapse. The reason is hype deflation and technical stagnation don't have to arrive together. Once people stop promising AGI by Christmas and clamp down on infinite growth + infinite GPU spend, things will start to look more normal. At this point, it feels more like the financing story was the shaky part not the tech or the workflows. LLMs’ve changed workflows in a way that’s very hard to unwind now.
edwin2 11mo ago
I think the author is onto something. but (s)he didn’t highlight that there are some scenarios where factual accuracy is unimportant, or maybe even a detractor. for example, fictional stories. If you want to be entertained and it doesn’t matter if it’s true or not, there’s no downsides to “hallucinations”. you could argue that stories ARE hallucinations. another example is advertisements. what matters is how people perceive them, not what’s actually true. or, content for a political campaign. the more i think about it, genAI really is a perfect match for social media companies
- skydhash 11mo ago
  Have you ever see how much work is spent on writing a novel? Even a short one? Character and world building are not easy and require very logical reasoning even if the premise are imaginary. You can say something is humans then give it three hands in the next sentence.
  - edwin2 11mo ago
    i see your point and you’ve changed my mind. also, i’d like to hear your story about the human with three hands
stego-tech 11mo ago
The winters are the best part, economic harm aside. Winters are when technology falls out of the vice grip of Capital and into the hands of the everyman. Winters are when you’ll see folks abandon this AIaaS model for every conceivable use case, and start shifting processing power back to the end user. Winters ensure only the strongest survive into the next Spring. They’re consequences for hubris (“LLMs will replace all the jobs”) that give space for new things to emerge. So, yeah, I’m looking forward to another AI winter, because that’s when we finally see what does and does not work. My personal guess is that agents and programming-assistants will be more tightly integrated into some local IDEs instead of pricey software subscriptions, foundational models won’t be trained nearly as often, and some accessibility interfaces will see improvement from the language processing capabilities of LLMs (real-time translation, as an example, or speech-to-action). That, I’m looking forward to. AI in the hands of the common man, not locked behind subscription paywalls, advertising slop, or VC Capital.
Nevermark 11mo ago
I am simply stunned at the negativity. Yes, there is hype. But if you actually filter it out, instead of (over) reacting to it in either direction, progress has been phenomenal and the fact there is visible progress in many areas, including LLMs, in the order of months demonstrates no walls. Visible progress doesn’t mean astounding progress. But any tech that is improving year to year is moving at a good speed. Huge apparent leaps in recent years seem to have spoiled some people. Or perhaps desensitized them. Or perhaps, created frustration that big leaps don’t happen every week. I can’t fathom anyone not using models for 1000 things. But we all operate differently, and have different kinds of lives, work and problems. So I take claims that individuals are not getting much from models at face value. But that some people are not finding the value isn’t an argument that those of us getting value, increasing value isn’t real.
- wellthisisgreat 11mo ago
  Yeah after reading the intro lines I noped out of this garbage blog (?). Maybe it's their SEO strategy to write the flat-out untrue incendiary stuff like "the technology is essentially a failure" - If that's the case, I don't have time for this. Maybe they actually think this way, then, I certainly have time for this. what I have time for? virtual bonding with fello HN ppl :)
- Sevii 11mo ago
  We're getting improvements coming out month by month. No reasonable person would look at a technology improving at this frequency and write a blog post about how we've hit the ceiling.
- lm28469 11mo ago
  There was value in leaded gas and asbestos insulation too, nobody denies that. You're blind to all the negative side effects, AI generated slop ads, engagement traps, political propaganda, scams, &c. The amount of pollution is incredible, search engines are dead, blogs are dead, YouTube is dead, social medias are dead, it's virtually impossible to find non slop content, the ratio is probably already 50:1 by now And these are only the most visible things, I know a few companies losing hundreds of hours every month replying to support tickets that are fully llm generated an more often than not don't make any sense. Another big topic is education.
raincole 11mo ago
While both are unlikely, if I have to choose one I would bet on AGI than AI winter in the next five years. AI just got better and better. People thought it couldn't solve math problems without some human formalizes them first. Then it did. People thought it couldn't generate legible text. Then it did. All while people swore it had reached a "plateau," "architecture ceiling," "inherent limit," or whatever synonym of the goalpost.
aaroninsf 11mo ago
Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon. Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false. Lemma: contemporary implementations have almost always already been improved upon, but are unevenly distributed. (Ximm's Law)
light_hue_1 11mo ago
As someone who is an expert in the area, everything in this article is misleading nonsense, failing at even the most basic CS101 principles. The level of confusion here is astounding. > People were saying that this meant that the AI winter was over The last AI winter was over 20 years ago. Transformers came during an AI boom. > First time around, AI was largely symbolic Neural networks were already hot and the state of the art across many disciplines when Transformers came out. > The other huge problem with traditional AI was that many of its algorithms were NP-complete Algorithms are not NP-complete. That's a type error. Problems can be NP-complete, not algorithms. > with the algorithm taking an arbitrarily long time to terminate This has no relationship to something being NP-complete at all. > but I strongly suspect that 'true AI', for useful definitions of that term, is at best NP-complete, possibly much worse I think the author means that "true AI" returns answers quickly and with high accuracy? A statement that has no relationship to NP-completeness at all. > For the uninitiated, a transformer is basically a big pile of linear algebra that takes a sequence of tokens and computes the likeliest next token This is wrong on many levels. A Transformer is not a linear network, linear networks are well-characterized and they aren't powerful enough to do much. It's the non-linearities in the Transformer that allows it to work. And only Decoders compute the distribution over the next token. > More specifically, they are fed one token at a time, which builds an internal state that ultimately guides the generation of the next token Totally wrong. This is why Transformers killed RNNs. Transformers are provided all tokens simultaneously and then produce a next token one at a time. RNNs don't have that ability to simultaneously process tokens. This is just totally the wrong mental model of what a Transformer is. > This sounds bizarre and probably impossible, but the huge research breakthrough was figuring out that, by starting with essentially random coefficients (weights and biases) in the linear algebra, and during training back-propagating errors, these weights and biases could eventually converge on something that worked. Again, totally wrong. Gradient descent dates back to the late 1800s early 1900s. Backprop dates back to the 60s and 70s. So this clearly wasn't the key breakthrough of Transformers. > This inner loop isn't Turing-complete – a simple program with a while loop in it is computationally more powerful. If you allow a transformer to keep generating tokens indefinitely this is probably Turing-complete, though nobody actually does that because of the cost. This isn't what Turing-completeness is. And by definition all practical computing is not a Turing Machine, simply because TMs require an infinite tape. Our actual machines are all roughly Linear Bounded Automata. What's interesting is that this doesn't really provide us with anything useful. > Transformers also solved scaling, because their training can be unsupervised Unsupervised methods predate Transformers by decades and were already the state of the art in computer vision by the time Transformers came out. > In practice, the transformer actually generates a number for every possible output token, with the highest number being chosen in order to determine the token. Greedy decoding isn't the default in most applications. > The problem with this approach is that the model will always generate a token, regardless of whether the context has anything to do with its training data. Absolutely not. We have things like end tokens exactly for this, to allow the model to stop generating. I got tired of reading at this point. This is drivel by someone who has no clue what's going on.
siva7 11mo ago
Is "A New AI Winter Is Coming" the new "The Year of Linux Desktop"?
- _se 11mo ago
  No, the new "Year of the Linux Desktop" is "This Year Software Engineers Won't Exist Any More Because Of LLMs". Very obviously.
bdangubic 11mo ago
> it's a horrible liability to have to maintain a codebase that nobody on the team actually authored. exactly - just like 99.9873% of all codebases currently running in production worldwide :)
aoeusnth1 11mo ago
Token economics are very sound - it's the training which is expensive. Tokens/week have gone up 23x year-over-year according to https://openrouter.ai/rankings https://openrouter.ai/rankings. This is probably around $500M-1B in sales per year. The real question is where the trajectory of this rocket ship is going. Will per-token pricing be a race to the bottom against budget chinese model providers? Will we see another 20x year year over the next 3 years, or will it level out sooner?
- tim333 11mo ago
  Yeah, that doesn't look at all wintery. More exponential.
andreyk 11mo ago
This blog post is full of bizarre statements and the author seems almost entirely ignorant of the history or present of AI. I think it's fair to argue there may be an AI bubble that will burst, but this blog post is plainly wrong in many ways. Here's a few clarifications (sorry this is so long...): "I should explain for anyone who hasn't heard that term [AI winter]... there was much hope, as there is now, but ultimately the technology stagnated. " The term AI winter typically refers to a period of reduced funding for AI research/development, not the technology stagnating (the technology failing to deliver on expectations was the cause of the AI winter, not the definition of AI winter). "[When GPT3 came out, pre-ChatGPT] People were saying that this meant that the AI winter was over, and a new era was beginning." People tend to agree there were two AI winters already, one having to do with symbolic AI disappointments/general lack of progress (70s), and the latter related to expert systems (late 80s). That AI winter has long been over. The Deep Learning revolution started in ~2012, and by 2020 (GPT 3) huge amount of talent and money were already going into AI for years. This trend just accelerated with ChatGPT. "[After symbolic AI] So then came transformers. Seemingly capable of true AI, or, at least, scaling to being good enough to be called true AI, with astonishing capabilities ... the huge research breakthrough was figuring out that, by starting with essentially random coefficients (weights and biases) in the linear algebra, and during training back-propagating errors, these weights and biases could eventually converge on something that worked." Transformers came about in 2017. The first wave of excitement about neural nets and backpropagation goes all the way back to the late 80s/early 90s, and AI (computer vision, NLP, to a lesser extent robotics) were already heavily ML-based by the 2000s, just not neural-net based (this changed in roughly 2012). "All transformers have a fundamental limitation, which can not be eliminated by scaling to larger models, more training data or better fine-tuning ... This is the root of the hallucination problem in transformers, and is unsolveable because hallucinating is all that transformers can do." The 'highest number' token is not necessarily chosen, this depends on the decoding algorithm. That aside, 'the next token will be generated to match that bad choice' makes it sound like once you generate one 'wrong' token the rest of the output is also wrong. A token is a few characters, and need not 'poison' the rest of the output. That aside, there are plenty of ways to 'recover' from starting to go down the wrong route. A key aspect of why reasoning in LLMs works well is that it typically incorporates backtracking - going earlier in the reasoning to verify details or whatnot. You can do uncertainty estimation in the decoding algorithm, use a secondary model, plenty of things (here is a detailed survey https://arxiv.org/pdf/2311.05232 https://arxiv.org/pdf/2311.05232 , one of several that is easy to find). "The technology won't disappear – existing models, particularly in the open source domain, will still be available, and will still be used, but expect a few 'killer app' use cases to remain, with the rest falling away." A quick google search shows ChatGPT currently has 800 million weekly active users who are using it for all sorts of things. AI-assisted programming is certainly here to stay, and there are plenty of other industries in which AI will be part of the workflow (helping do research, take notes, summarize, build presentations, etc.) I think discussion is good, but it's disappointing to see stuff with this level of accuracy being on front page of HN.
RamblingCTO 11mo ago
I'm so annoyed by this negativity. Is AI perfect? No, far from it. Does it have a positive impact on productivity? 100%. Do I care about financials? Absolutely not. There's the regular hype cycle. But these idiotic takes like "dotcom 2.0" and "AI winter" are just and show that the author has no clue what they are talking about.
- monkaiju 11mo ago
  That 100% claim is doing an awful lot of work. Theres is ample documentation from most corporate rollouts that show the opposite, and as a SWE I find it worse than useless...
johnfn 11mo ago
I’m fascinated by people who say that LLMs have failed in practice. Last week, when I was on PTO, I used AI to to a full redesign of a music community website I run. I touched about 40k lines of code in a week. The redesign is shipped and everyone is using it. AI let me go about 5-10x faster than if I would have done this by hand. (In fact, I have tried doing this in the past, so I really do have an apples to apples comparison for velocity. AI enabled it happening at all: I’ve tried a few other times in the past but never been able to squeeze it into a week.) The cited 40% inaccuracy rate doesn’t track for me at all. Claude basically one-shot anything I asked for, to the point that the bottleneck was mostly thinking of what I should ask it to do next. At this point, saying AI has failed feels like denying reality.
- svaha1728 11mo ago
  On a music blog, yes! Now go try to rewrite the firmware for your car.
  - reducesuffering 11mo ago
    95% of the work engineers on this forum do are high level apps like a "music blog", and not car firmware...
    - l1ng0 11mo ago
      And then there's the small % doing music firmware :) (and no, I don't find LLMs much use on this)
  - johnfn 11mo ago
    It's not a blog - it's a community website, so significantly more complex (running competitions, uploading tracks, moderation, etc, etc, etc). I don't understand the car firmware non-sequitur. I've already seen productivity gains in this domain. Why do I need to also prove them in another one? They are already demonstrated. It's like me saying "I can get places faster in a car than on foot" and you're saying "yeah but you can't get to the moon."
- 11mo ago
injidup 11mo ago
Most “AI is rubbish” takes treat it as an open-loop system: prompt → code → judgment. That’s not how development works. Even humans can’t read a spec, dump code, and ship it. Real work is closed-loop: test, compare to spec, refine, repeat. AI shines in that iterative feedback cycle, which is where these critiques miss the point.
- phantasmish 11mo ago
  What’s tough for me is figuring out where people are realizing significant improvements from this. If you have to set up good tests [edit: and gather/generate good test data!] and get the spec hammered out in detail and well-described in writing, plus all the ancillary stuff like access to any systems you need, sign-offs from stakeholders… dude that’s more than 90% of the work, I’d say. I mean fuck, lots of places just skip half that and figure it out in the code as they go. How’s this meaningfully speeding things up?
screye 11mo ago
> This means that they should never be used in medicine, for evaluation in school or college, for law enforcement, for tax assessment, or a myriad of other similar cases. If AI models can deliver measurably better accuracy than doctors, clearer evaluations than professors and fairer prosecutions than courts, then it should be adopted. Waymo has already shown a measurable decrease in loss of life by eliminating humans from driving. I believe, technically, moderns LLMs are sufficiently advanced to meaningfully disrupt the aforementioned professions as Waymo has done for taxis. Waymo's success relies on 2 non-llm factors that we've yet to see for other professions. First is exhaustive collection and labelling of in-domain high quality data. Second is the destruction of the pro-human regulatory lobby (thanks to work done by Uber in the Zirp era that came before). To me, an AI winter isn't a concern, because AI is not the bottleneck. It is regulatory opposition and sourcing human experts who will train their own replacements. Both are significantly harder to get around for high-status white collar work. The great-AI-replacement may still fail, but it won't be because of the limitations of LLMs. > My advice: unwind as much exposure as possible you might have to a forthcoming AI bubble crash. Hedging when you have much at stake is always a good idea. Bubble or no bubble.
josefritzishere 11mo ago
I trully hate how absurdly over-hyped this LLM concept of AI is. However, it can do some cool things. Normally that's the path to PMF. The real problem is the revenue. There is no extant revenue model whatsoever. But there are industries that are described as "a business of pennies" Ex. telephony. Someone may yet eek out a win. But the hype-to-reality conversion will come first.
czhu12 11mo ago
I think its worth discounting against the failure rates of humans also. > Depending on the context, and how picky you need to be about recognizing good or bad output, this might be anywhere from a 60% to a 95% success rate, with the remaining 5%-40% being bad results. This just isn't good enough for most practical purposes This seems to suggest that humans are 100%. I'd be surprised if i was anywhere close to that after 10 years of programming professionally
hadlock 11mo ago
> AI has failed. >The rumor mill has it that about 95% of generative AI projects in the corporate world are failures. AI tooling has only just barely reached the point where enterprise CRUD developers can start thinking about. Langchain only reached v1.0.0 in the last 60 days (Q4 2025); OpenAI effectively announced support for MCP in Q2 2025. The spec didn't even approach maturity until Q4 of 2024. Heck most LLMs didn't have support for tools in 2024. In 2-3 years a lot of these libraries will be part way through their roadmap towards v2.0.0 to fix many of the pain points and fleshing out QOL improvements, and standard patterns evolved for integrating different workflows. Consumer streaming of audio and video on the web was a disaster of a mess until around ~2009 despite browsers having plugins for it going back over a decade. LLMs continue to improve at a rapid rate, but tooling matures more slowly. Of course previous experiments failed or were abandoned; the technology has been moving faster than the average CRUD developer can implement features. A lot of "cutting edge" technology we put into our product in 2023 are now standard features for the free tier of market leaders like ChatGPT etc. Why bother maintaining a custom fork of 2023-era (effectively stone age) technology when free tier APIs do it better in 2025? MCP might not be the be-all, end-all, but at least it is a standard interface that's at least maintainable in a way that developers of mature software can begin conceiving of integrating it into their product as a permanent feature, rather than a curiosity MVP at the behest of a non technical exec. A lot of AI-adjacent libraries we've been using finally hit v1.0.0 this year, or creeping close to it; providing stable interfaces for maintainable software. It's time to hit the reset button on "X% of internal AI initiatives failed"
jgb1984 11mo ago
I sure hope another winter is on the way, the LLM hype wagon is beyond exhausting.
ivanche 11mo ago
These sort of articles are new "economists have predicted 17 of the past 3 recessions".
bitwize 11mo ago
Eric S. Raymond (yes, that ESR; insert long-winded DonHopkins rant, extensively cross-referenced to his own USENET and UNIX-HATERS mailing list postings, about what a horrible person "Eric the Flute" is) reports a shortening of time to implement his latest project from weeks to hours. He says the advantages of LLMs in the programming space are yet to truly unfold, because they not only let you do things faster, they let you try things you wouldn't have started otherwise because it'd be too much of a time/effort commitment. Assuming these claims are even partially true, we'd be stupid—at the personal and societal level—not to avail ourselves of these tools and reap the productivity gains. So I don't see AI going away any time soon. Nor will it be a passing fad like Krugman assumed the internet would be. We'd have to course-correct on its usage, but it truly is a game changer.
1vuio0pswjnm7 11mo ago
As illustrated by the top comment, it seems that "AI" relies on hype, i.e., careful manipulation of how it is perceived. Its continued existence depends on relentless fund raising because generally no one is going to pay fees to use it long-term As with so much of Silicon Valley's output the past few decades, "AI" is not something people will pay for. It could be a new form of Trojan Horse for data collection, surveillance and ads, though, adopting a hyperfocus on "growth" at any cost (human, environmental, etc.) but this might not even be necessary^1 Time will tell In order to raise funds in the absence of any clear business plan, "AI" cannot just be ordinary and useful. It cannot be "autocomplete on steroids". It must be "world-changing" No one shall underestimate or discount the value of "AI". Overestimation is permitted, however. Speculation is encouraged, but only if it's wild and extreme 1. "AI" is not depending on being brought inside the gates of Troy by the city's inhabitants. It's being placed inside the gates by others. Companies with large numbers of users are installing "AI" on users' computers without the users' consent
bunderbunder 11mo ago
I've run into what I think might be an even bigger problem with the "just keep reading/outputting the next token" design: there's no going backwards. This means that LLMs don't do well with input where text that comes later in the input alters how text that came earlier should be interpreted. Which is really quite common in practice. Humans like to do things where you start with a dump of information, and then ask questions about it afterward. If a human reader didn't remember something pertinent to the question, they can go back and re-read now that they know what they're looking for. An LLM can't do that by itself. You can paper over that limitation with hacks, but they're hacky and unreliable. Humans also correct misunderstandings and resolve ambiguities after the fact. LLMs have a really hard time with this. Once one way of interpreting some text has already made it into the LLM's context vectors, it can be surprisingly difficult for subsequent text to significantly alter that interpretation. It's these sorts of aggravating failure modes stemming from a fundamental feature of the algorithm that seem to always produce the next AI winter. Because they tend to mean the same thing: research has gone down (and then, quote often, massively over-invested in) yet another blind alley that doesn't put us on a short easy path to AGI any more than the previous 75 years' worth of breakthroughs have. And getting out of it is going to involve not only fixing that one problem, but also a huge reset on some substantial portion of whatever tools and techniques were built up to paper over the previous algorithm's limitations in an effort to generate positive ROI on all that massive over-investment.
- octoberfranklin 11mo ago
  No, that's not how a Transformer works. It gets the entire input all at once. Then it generates the output one token at a time.
starchild3001 11mo ago
The author is conflating a financial correction with a technological failure. I agree that the economics of GenAI are currently upside down. The CapEx spend is eye-watering, and the path to profitability for the foundational model providers is still hazy. We are almost certainly in an age of inflated-expectations hype-cycle peak that will self-correct, and yes, "winter is harsh on tulips". However, the claim that the technology itself is a failure is objectively disconnected from reality. Unlike crypto or VR (in their hype cycles), LLMs found immediate, massive product-market fit. I use K-means clustering and logistic regression every day; they aren't AGI either, but they aren't failures. If 95% of corporate AI projects fail, it's not because the tech is broken; it's because middle management is aspiring to replace humans with a terminal-bound chatbot instead of giving workers an AI companion. The tech isn't going away, even if AI valuations might be questioned in the short term.
babblingfish 11mo ago
Winter is coming[1] [1] https://www.youtube.com/watch?v=gxhknGARGt4 https://www.youtube.com/watch?v=gxhknGARGt4
kazinator 11mo ago
I'm skeptical for this reason: at the time of the first AI winter, AI was niche. Today, it is mainstream; vast numbers of consumers are using AI. AI in consumer hands is no longer just a niche program in a corner of your company or government organization that can defund and shut down. If the current purveyors of big cloud AI were to fold, their replacements are waiting in the wings to pick up the demand somehow.
zkmon 11mo ago
Not quite. The core capability of extracting a meaning out of unstructured text and media, and the reverse of it, would remain as a bedrock. Anything else built around it (agents etc) might be challenged by the realities.
tim333 11mo ago
>A new AI winter is coming. Counterpoint - no it isn't. The NP arguments are dumb and have nothing to do with thinking. I don't think Einstein was NP complete but could still think. Winter summer implies sine wave like progress but what we are seeing now is a result of exponential growth in computer hardware such that it is now passing human equivalent levels https://www.bvp.com/assets/uploads/2024/03/price-performance-slide-ray-kurzweil-atlas-image.jpg https://www.bvp.com/assets/uploads/2024/03/price-performance... LLMs having failings is a blip. Hinton got his nobel prize for work done in the 1980s but we are seeing AI takeoff now 45 years later, almost exactly when predicted by the likes of Moravec/Kurweil because of the hardware progress. (see the token use graph mentioned by aoeusnth1 - https://openrouter.ai/rankings https://openrouter.ai/rankings usage looking exponential too)
kittikitti 11mo ago
This is a good article and although it does sound like yet another AI doomer piece, it at least tackled it from a unique perspective. AI winter is coming and it's going to be harsh for those who have never heard of an AI winter before the AI summer that started in 2017. This isn't a tulip-mania that's it's so often compared to. People find these concepts distilled down to one neuron and think they know what they're talking about. This AI winter is going to be harsh with economic catastrophes because of climate change and political instability. There might even be another nuclear scare. Will they blame AI for all their problems? Definitely.
rdiddly 11mo ago
To the author's brief list of areas where AI can't be used because it's too risky, I would add "driving on the public roads," but like everything else on the list, it's currently being done anyway.
arisAlexis 11mo ago
Every article with this title is either a human psychology cry for a) we are going too fast I wish we could stop or b) I delight myself to be a contrarian
signa11 11mo ago
this is just _excellent_ ``` I would conjecture that this is another manifestation of the NP-completeness wall that slammed symbolic AI, causing the first AI winter. It's always possible to turn an NP-complete algorithm into one that runs quickly, if you don't mind that it fails to generate any output if you hit a timeout. The transformer equivalent of this is generating plausible, wrong, hallucinated output in cases where it can't pattern match a good result based on its training. The problem, though, is that with traditional AI algorithms you typically know if you've hit a timeout, or if none of your knowledge rules match. With transformers, generating wrong output looks exactly like generating correct output, and there is no way to know which is which. ```
Havoc 11mo ago
What this misses imo is that even if only 10% of tasks out there are suitable that is still a crazy big number economically. Possibly even big enough to plug the hole in OpenAI’s math
Olivia8 11mo ago
I too was excited when transformer-based models burst onto the scene — chatbots suddenly seemed smarter and more flexible than the old rule-based AI systems. Those early models were rough around the edges and sometimes produced garbage, but when they hit, it really felt like a step forward. The problem now is that these models always pick a next token — even when they have no good basis for it — so they can generate plausible but wrong or misleading content, and you often can’t tell the difference. That fundamental limitation means they aren’t reliable enough for high-stakes tasks like medicine, law, or financial decisions. Meanwhile, voice-based AI has also been growing rapidly this year, Coldi AI is a good example, with many more people and companies adopting assistants and voice interfaces.

199 points by voxleone 11mo ago

Modern LLMs could be like the equivalent of those steam powered toys the Romans had in their time. Steam tech went through a long winter before finally being fully utilized for the Industrial Revolution. We’ll probably never see the true AI revolution in our lifetime, only glimpses of what could be, through toys like LLMs. What we should underscore though, is that even if there is a new AI winter, the world isn’t going back to what it was before AI. This is it, forever. Generations ahead will gaslight themselves into thinking this AI world is better, because who wants to grow up knowing they live in a shitty era full of slop? Don’t believe it.

7thaccount 11mo ago
LLMs are useful tools, but certainly have big limitations. I think we'll continue to see anything be automated that can be automated in a way that reduces head count. So you have the dumb AI as a first line of defense and lay off half the customer service you had before. In the meantime, fewer and fewer jobs (especially entry level), a rising poor class as the middle class is eliminated and a greater wealth gap than ever before. The markets are going to also collapse from this AI bubble. It's just a matter of when.
cardanome 11mo ago
The development of LLM's required access to huge amounts of decent training data. It could very well that the current generation of AI has poisoned the well for any future endeavors of creating AI. You can't trivially filter out the AI slop and humans are less likely to make their handcrafted content freely available for training. In fact violating GPL code to train models on it might be ruled to be illegal as well generally stricter rules on which data you are allowed to use for training. We might have reached a local optimum that is very difficult to escape from. There might be a long, long AI winter ahead of us, for better or worse. > the world isn’t going back to what it was before AI. This is it, forever. I feel this so much. I though my longing for the pre-smartphone days was bad but damn we have lost so much.
- esafak 11mo ago
  That would create the impetus to create a training data marketplace.
  - deadbabe 11mo ago
    Maybe the path to redemption for AI stealing jobs would be if people would have to be rehired en masse to write and produce art, about whatever they want, so that it can be used to train more advanced AI.
rnxrx 11mo ago
The development of steam technology is a great metaphor. The basic understanding of steam as a thing that could yield some kind of mechanical force almost certainly predated even the Romans. That said, it was the synthesis of other technologies with these basic concepts that started yielding really interesting results. Put another way, the advent of industrialized steam power wasn't so much about steam per se, but rather the intersection of a number of factors (steam itself obviously being an important one). This intersection became a lot more likely as the pace of innovation in general began accelerating with the Enlightenment and the ease with which this information could be collected and synthesized. I suspect that the LLM itself may also prove to be less significant than the density of innovation and information of the world it's developed in. It's not a certainty that there's a killer app on the scale of mechanized steam, but the odds of such significant inventions arguably increase as the basics of modern AI become basic knowledge for more and more people.
- orwin 11mo ago
  Its mostly metallurgy. The fact that we became so much better and precise at metallurgy enabled us to make use of steam machines. Of course a lot of stuff helped (Glassmaking, whale oil immediatly come to mind) but mostly, metallurgy.
  - int_19h 11mo ago
    I remember reading an article that argued that it was basically a matter of being path dependent. The earliest steam engines that could do useful work were notoriously large and fuel-inefficient, which is why their first application was for pumps in coal mines - it effectively made the fuel problem moot and similarly their other limitations were not important in that context, while at the same time rising wages in UK made even those inefficient engines more affordable than manual labor. And then their use in that very narrow niche allowed them to be gradually improved to the point where they became suitable for other contexts as well. But if that analogy holds, then LLM use in software development is the "new coal mines" where it will be perfected until it spills over into other areas. We're definitely not at the "Roman stage" anymore.

kunley 11mo ago

Can't wait

teaearlgraycold 11mo ago

LLMs have failed to live up to the hype, but they haven't failed outright.

HardCodedBias 11mo ago
Two claims here: 1) LLMs have failed to live up to the hype. Maybe. Depends upon's who's hype. But I think it is fine to say that we don't have AGI today (however that is defined) and that some people hyped that up. 2) LLMs haven't failed outright I think that this is a vast understatement. LLMs have been a wild success. At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is larger. ChatGPT has over 800 million weekly active users. Students throughout the world, and especially in the developed world are using "AI" at 85-90% (from some surveys). Between 40% of professionals and 90% (depending upon survey and profession) are using "AI". This is 3 years after the launch of ChatGPT (and the capabilities of chatGPT 3.5 were so limited compared to today that it is a shame that they get bundled together in our discussions). I would say instead of "failed outright" that they are the most successful consumer product of all time (so far).
- baseballdork 11mo ago
  > At big tech over 40% of checked in code is LLM generated. At smaller companies the proportion is larger. I have a really hard time believing that stat without any context, is there a source for this?
  - Groxx 11mo ago
    from what I've seen in a several-thousand-eng company: LLMs generally produce vastly more code than is necessary, so they quickly out-pace human coders. they could easily be producing half or more of all of the code even if only 10% of the teams use it. particularly because huge changes often get approved with just a "lgtm", and LLM-coding teams also often use/trust LLMs for reviews. but they do that while making the codebase substantially worse for the next person or LLM. large code size, inconsistent behavior, duplicates of duplicates of duplicates strewn everywhere with little to no pattern so you might have to fix something a dozen times in a dozen ways for a dozen reasons before it actually works, nothing handles it efficiently. the only thing that matters in a business is value produced, and I'm far from convinced that they're even break-even if they were free in most cases. they're burning the future with tech debt, on the hopes that it will be able to handle it where humans cannot, which does not seem true at all to me.

moltar 11mo ago

> The technology is essentially a failure Really? I derive a ton of value from it. For me it’s a phenomenal advancement and not a failure at all.

deleted 11mo ago

[deleted]

tarr11 11mo ago

This has convinced many non-programmers that they can program, but the results are consistently disastrous, because it still requires genuine expertise to spot the hallucinations. I've been programming for 30+ years and now a people manager. Claude Code has enabled me to code again and I'm several times more productive than I ever was as an IC in the 2000s and 2010s. I suspect this person hasn't really tried the most recent generation, it is quite impressive and works very well if you do know what you are doing

agubelu 11mo ago
Isn't that what the author means? "it still requires genuine expertise to spot the hallucinations" "works very well if you do know what you are doing"
- hombre_fatal 11mo ago
  But it can work well even if you don't know what you are doing (or don't look at the impl). For example, build a TUI or GUI with Claude Code while only giving it feedback on the UX/QA side. I've done it many times despite 20 years of software experience. -- Some stuff just doesn't justify me spending my time credentializing in the impl. Hallucinations that lead to code that doesn't work just get fixed. Most code I write isn't like "now write an accurate technical essay about hamsters" where hallucinations can sneak through lest I scrutinize it; rather the code would just fail to work and trigger the LLM's feedback loop to fix it when it tries to run/lint/compile/typecheck it. But the idea that you can only build with LLMs if you have a software engineer copilot isn't true and inches further away from true every month, so it kinda sounds like a convenient lie we tell ourselves as engineers (and understandably so: it's scary).
  - int_19h 11mo ago
    > Hallucinations that lead to code that doesn't work just get fixed How about hallucinations that lead to code that doesn't work outside of the specific conditions that happen to be true in your dev environment? Or, even more subtly, hallucinations that lead to code which works but has critical security vulnerabilities?

ZeroConcerns 11mo ago

Well, the original "AI winter" was caused by defense contracts running out without anything to show for it -- turns out, the generals of the time could only be fooled by Eliza clones for so long... The current AI hype is fueled by public markets, and as they found out during the pandemic, the first one to blink and acknowledge the elephant in the room loses, bigly. So, even in the face of a devastating demonstration of "AI" ineffectiveness (which I personally haven't seen, despite things being, well, entirely underwhelming), we may very well stuck in this cycle for a while yet...

skerit 11mo ago
> despite things being, well, entirely underwhelming People use LLMs for all kinds of things, but for coding it is absolutely not underwhelming. Can you treat it as a real independent developer, one that doesn't need supervision? No. Can it save you hours and hours of work? Yes.

citizenpaul 11mo ago

>Expect OpenAI to crash, hard, with investors losing their shirts. Lol someone doesn't understand how the power structure system works "the golden rule". There is a saying if you owe the bank 100k you have a problem. If you owe the bank ten million the bank has a problem. OpenAI and the other players have made this bubble so big that there is no way the power system will allow themselves to take the hit. Expect some sort of tax subsided bailout in the near future.

AnimalMuppet 11mo ago
The difference is that OpenAI isn't financed by borrowed money.
dragonwriter 11mo ago
Tax subsized bailouts usually are structured to protect the corporation as a separate entity, creditors (compared to letting the org continue on its unimpeded path), employees, and sometimes, beyond their position as current creditors, suppliers, but not as often do they protect existing equity holders.

aroman 11mo ago

When the hype is infinite (technological singularity and utopia), any reality will be a let down. But there is so much real economic value being created - not speculation, but actual business processes - billions of dollars - it’s hard to seriously defend the claim that LLMs are “failures” in any practical sense. Doesn’t mean we aren’t headed for a winter of sobering reality… but it doesn’t invalidate the disruption either.

emp17344 11mo ago
Other than inflated tech stocks making money off the promise of AI, what real economic impact has it actually had? I recall plenty of articles claiming that companies are having trouble actually manifesting the promised ROI.
- phantasmish 11mo ago
  My company’s spending a lot of money doing things they could have done fifteen or more years ago with classic computer vision libraries and other pre-LLM techniques, cheaper and faster. Most of the value of AI for our organization is the hype itself providing the activation energy, if you will, to make these projects happen. The value added by the AI systems per se has been minimal. (YMMV but that’s what I’m seeing at the non-tech bigco I’m at—it’s pretty silly but the checks continue to clear so whatever)
n4r9 11mo ago
> not speculation, but actual business processes Is there really a clear-cut distinction between the two in today's VC and acquisition based economy?
api 11mo ago
This is why I hate hype. "We just cured cancer! All cancer! With a simple pill!" "But you promised it would rejuvenate everyone to the metabolism of a 20 year old and make us biologically immortal!" New headline: "After spending billions, project to achieve immortality has little to show..."
keybored 11mo ago

stanfordkid 11mo ago

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal models entirely. He also pretty much discounts everything that’s happened with AlphaFold, Alpha Go etc. reinforcement learning etc. The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why. Is the brain NP complete? Maybe maybe not. I could see many arguments about why modern research will fail to create AGI but just hand waving “reality is NP-hard” is not enough. The fact is: something fundamental has changed that enables a computer to pretty effectively understand natural language. That’s a discovery on the scale of the internet or google search and shouldn’t be discounted… and usage proves it. In 2 years there is a platform with billions of users. On top of that huge fields of new research are making leaps and bounds with novel methods utilizing AI for chemistry, computational geometry, biology etc. It’s a paradigm shift.

gishh 11mo ago
> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't have an agenda.
- sukhdeepprashit 11mo ago
  [dead]
- danielvaughn 11mo ago
  I think it’s a disingenuous read to assume original commenter means “understanding” in the literal sense. When we talk about LLM “understanding”, we usually mean it from a practical sense. If you give an input to the computer, and it gives you an expected output, then colloquially the computer “understood” your input.

paganel 11mo ago

I've been reading about this supposed AI winter for at least 5 years by now, and in the meantime any AI-related stock has gone 10x and more.

_se 11mo ago
5 years? That's so interesting bro. Tell us more about what your thoughts on ChatGPT were back in 2020. Go ahead and double check when the LLM craze started and perhaps reconsider making things up.
- paganel 11mo ago
  May 2018: AI winter is well on its way (piekniewski.info) [1] January 2020: Researchers: Are we on the cusp of an ‘AI winter’? (bbc.co.uk) [2] I'm sure you can easily find more. Felt good to be called a "bro", though, made me feel younger. [1] HN discussion, almost 500 comments: https://news.ycombinator.com/item?id=17184054 https://news.ycombinator.com/item?id=17184054 [2] HN discussion on BBC article, ~110 comments: https://news.ycombinator.com/item?id=22069204 https://news.ycombinator.com/item?id=22069204

neom 11mo ago

Blog posts like this make me think model adoption and appropriate use case for the model is...lumpy at best. Every time I read something like it I wonder what tools they are using and how? Modern systems are not raw transformers. A raw transformer will “always output something,” they're right, but nobody deploys naked transformers. This is like claiming CPUs can’t do long division because the ALU doesn’t natively understand decimals. Also, a model is stat aprox trained on the empirical distribution of human knowledge work. It is not trying to compute the exact solution to NP complete problems? Nature does not require worst case complexity, real world cognitive tasks are not worst case NP hardness instances...

numbers_guy 11mo ago

AlexNet was only released in 2011. The progress made in just 14 years has been insane. So while I do agree that we are approaching a "back to the drawing board" era, calling the past 14 years a "failure" is just not right.

cmiles8 11mo ago

LLMs are an amazing advancement. The tech side of things is very impressive. Credit where credit is due. Where the current wave all falls apart is on the financials. None of that makes any sense and there’s no obvious path forward. Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Tech wise I’m bullish. Business wise, AI is setting up to be a big disaster. Those that aimlessly chased the hype are heading for a world of financial pain.

famouswaffles 11mo ago
>Folks say handwavy things like “oh they’ll just sell ads” but even a cursory analysis shows that math doesn’t ad up relative to the sums of money being invested at the moment. Ok, so I think there's 2 things here that people get mixed on. First, Inference of the current state of the art is Cheap now. There's no 2 ways about it. Statements from Google, Altman as well as costs of 3rd parties selling tokens of top tier open source models paint a pretty good picture. Ads would be enough to make Open AI a profitable company selling current SOTA LLMs to consumers. Here's the other thing that mixes things up. Right now, Open AI is not just trying to be 'a profitable company'. They're not just trying to stay where they are and build a regular business off it. They are trying to build and serve 'AGI', or as they define it, 'highly autonomous systems that outperform humans at most economically valuable work'. They believe that, to build and serve this machine to hundreds of millions would require costs order(s) of magnitudes greater. In service of that purpose is where all the 'insane' levels of money is moving to. They don't need hundreds of billions of dollars in data centers to stay afloat or be profitable. If they manage to build this machine, then those costs don't matter, and if things are not working out midway, they can just drop the quest. They will still have an insanely useful product that is already used by hundreds of millions every week, as well as the margins and unit economics to actually make money off of it.
- cmiles8 11mo ago
  If OpenAI was the only company doing this that argument might sort of make sense. The problem is they have real competition now and that market now looks like an expensive race to an undifferentiated bottom. If someone truly invents AGI and it’s not easily copied by others then I agree it’s a whole new ballgame. The reality is that years into this we seem to be hitting a limit to what LLMs can do with only marginal improvements with each release. On that path this get ugly fast.

bgwalter 11mo ago

The poster is right. LLMs are Gish Gallop machines that produce convincing sounding output. People have figured it out by now. Generative "AI" will fail, other forms may continue, though it it would be interesting to hear from experts in other fields how much fraud there is. There are tons of material science "AI" startups, it is hard to believe they all deliver.

Dilettante_ 11mo ago
>produce convincing sounding output Well, correctness(though not only correctness) sounds convincing, the most convincing even, and ought to be information-theory-wise cheaper to generate than a fabrication, I think. So if this assumption holds, the current tech might have some ceiling left if we just continue to pour resources down the hole.
- knollimar 11mo ago
  How do LLMs do on things that are common confusions? Do they specifically have to be trained against them? I'm imagining a Monty Hall problem that isn't in the training set tripping them up the same way a full wine glass does

Barathkanna 11mo ago

Interesting take. His argument is basically that LLMs have hit their architectural ceiling and the industry is running on hype and unsustainable economics. I’m not fully convinced, but the points about rising costs and diminishing returns are worth paying attention to. The gap between what these models can actually do and what they’re marketed as might become a real problem if progress slows.

ryanjshaw 11mo ago
The existence of an AI hype train doesn’t mean there isn’t a productive AI no-hype train. Context: I have been writing software for 30 years. I taught myself assembly language and hacked games/apps as a kid, and have been a professional developer for 20 years. I’m not a noob. I’m currently building a real-time research and alerting side project using a little army of assistant AI developers. Given a choice, I would never go back to how I developed software before this. That isn’t my mind poisoned by hype and marketing.
- Abekkus 11mo ago
  If you're using cloud AI assistants, they're being heavily discounted to you by investors, for now. Those discounts won't exist forever.
DrewADesign 11mo ago
I think the unsustainably cheap consumer-facing AI products are the spoonful of sugar getting us to swallow a technology that will almost entirely be used to make agents that justify mass layoffs.
- buellerbueller 11mo ago
  And surveil and rat on us, sometimes incorrectly.
  - DrewADesign 11mo ago
    Yeah that’s a good point. I’m always astonished that people give anything even vaguely personal to a “free” saas service, but boy do they ever. Not that payments stop them but at least the incentives are a little different.

LarsDu88 11mo ago

Zero citations, random speculations. Random blogpost versus the most rapidly adopted technology in history...

kunley 11mo ago
This statement strikes me... but not the way you intended. We should say: most rapidly adopted ... speculation. Because this is what it is: not a technology, but speculation. Hint: technology has repeatable results. QED ;)

uejfiweun 11mo ago

"The technology is essentially a failure" is in the headline of this article. I have to disagree with that. For the first time in the history of the UNIVERSE, an entity exists that can converse in human language at the same level that humans can. But that's me being a sucker. Because in reality this is just a clickbait headline for an article basically saying that the tech won't fully get us to AGI and that the bubble will likely pop and only a few players will remain. Which I completely agree with. It's really not that profound.

abroszka33 11mo ago

I agree, I have over 20 years of software engineering experience and after vibe coding/engineering/architecting (or whatever you want to call it) for a couple months, I also don't see the technology progressing further. LLMs are more or less the same as 6 months ago, incremental improvements, but no meaningful progress. And what we have is just bad. I can use it because I know the exact code I want generated and I will just re-prompt if I don't get what I want, but I'm unconvinced that this is faster than a good search engine and writing code myself. I think I will keep using it while it's cheap, but once I have to pay the real costs of training/running a flagship modell I think I will quit. It's too expensive as it is for what it does.

esafak 11mo ago
Yet, the real cost of inference for a given model complexity is dropping.
benjiro 11mo ago
> LLMs are more or less the same as 6 months ago, incremental improvements, but no meaningful progress. Go back to a older version of a LLM and then say the same. You will notice that older LLM versions do less, have more issues, write worse code etc... There have been large jumps in the last 2 years but because its not like we go from a LLM to AGI, that people underestimate the gains. Trust me, try it, try Claude 3.7 > 4.0 > 4.5 > Opus 4.5 ... > I'm unconvinced that this is faster than a good search engine and writing code myself. What i see is mostly somebody who is standoffish on LLMs ... Just like i was. I tried to shoehorn their code generation into "my" code, and while it works reasonably, you tend to see the LLM working on "your code", as a invasion. So you never really use its full capabilities. LLMs really work the best, if you plan, plan, plan, and then have them create the code for you. The moment you try to get the LLMs work inside existing code, that is especially structured how YOU like it, people tend to be more standoffish. > It's too expensive as it is for what it does. CoPilot is like 27 Euro/month(year payment + dollar/euro) for 1500 requests here. We pay just for basic 100Mbit internet 45 Euro per month. I mean ... Will it get more expensive in the future, o yes, for sure. But we may also have alternatives by then. Open Source/Open Weight models are getting better and better, especially with MoE. Pricing is how you look at it... If i do the work what takes me a months in a few days, what is the price then? If i do work that i normally need to outsource. Or code that is some monotoon repeating end me %@#$ ... in a short time by paying a few cents to a LLM, ... Reality is, thing change, and just like the farmers that complained about the tractor, while their neighbor now did much more work thanks to that thing, LLMs are the same. I often see the most resistance from us older programmer folks, who are set in our ways, when its us who actually are the best at wrangling LLMs the best. As we have the experience to fast spot where a LLM goes wrong, and guide it down the right path. Tell it, the direction is debugging is totally wrong and where the bug more likely is ... For the last 2 years i paid just the basic cheap $10 subscription, and use it but never strongly. It helped with those monotoon tasks etc. Until ... a months ago, i needed a specific new large project and decided to just agent / vibe code it, just to try at first. And THEN i realized, how much i was missing out off. Yes, that was not "my" code, and when the click came in my head that "i am the manager, not the programmer", you suddenly gain a lot. Its that click that is hard for most seasoned veterans. And its ironically often the seasoned guys that complain the most about AI ... when its the same folks that can get the most out of LLMs.

binary132 11mo ago

what a curious coincidence that a soft-hard AI landing would happen to begin at the exact same time as the US government launches a Totally Not a Bailout Strategic Investment Plan Bro I Promise. who could have predicted this?

coffeecoders 11mo ago

I am of a belief that upcoming winter will look more like normalization than collapse. The reason is hype deflation and technical stagnation don't have to arrive together. Once people stop promising AGI by Christmas and clamp down on infinite growth + infinite GPU spend, things will start to look more normal. At this point, it feels more like the financing story was the shaky part not the tech or the workflows. LLMs’ve changed workflows in a way that’s very hard to unwind now.

edwin2 11mo ago

I think the author is onto something. but (s)he didn’t highlight that there are some scenarios where factual accuracy is unimportant, or maybe even a detractor. for example, fictional stories. If you want to be entertained and it doesn’t matter if it’s true or not, there’s no downsides to “hallucinations”. you could argue that stories ARE hallucinations. another example is advertisements. what matters is how people perceive them, not what’s actually true. or, content for a political campaign. the more i think about it, genAI really is a perfect match for social media companies

skydhash 11mo ago
Have you ever see how much work is spent on writing a novel? Even a short one? Character and world building are not easy and require very logical reasoning even if the premise are imaginary. You can say something is humans then give it three hands in the next sentence.
- edwin2 11mo ago
  i see your point and you’ve changed my mind. also, i’d like to hear your story about the human with three hands

stego-tech 11mo ago

The winters are the best part, economic harm aside. Winters are when technology falls out of the vice grip of Capital and into the hands of the everyman. Winters are when you’ll see folks abandon this AIaaS model for every conceivable use case, and start shifting processing power back to the end user. Winters ensure only the strongest survive into the next Spring. They’re consequences for hubris (“LLMs will replace all the jobs”) that give space for new things to emerge. So, yeah, I’m looking forward to another AI winter, because that’s when we finally see what does and does not work. My personal guess is that agents and programming-assistants will be more tightly integrated into some local IDEs instead of pricey software subscriptions, foundational models won’t be trained nearly as often, and some accessibility interfaces will see improvement from the language processing capabilities of LLMs (real-time translation, as an example, or speech-to-action). That, I’m looking forward to. AI in the hands of the common man, not locked behind subscription paywalls, advertising slop, or VC Capital.

Nevermark 11mo ago

I am simply stunned at the negativity. Yes, there is hype. But if you actually filter it out, instead of (over) reacting to it in either direction, progress has been phenomenal and the fact there is visible progress in many areas, including LLMs, in the order of months demonstrates no walls. Visible progress doesn’t mean astounding progress. But any tech that is improving year to year is moving at a good speed. Huge apparent leaps in recent years seem to have spoiled some people. Or perhaps desensitized them. Or perhaps, created frustration that big leaps don’t happen every week. I can’t fathom anyone not using models for 1000 things. But we all operate differently, and have different kinds of lives, work and problems. So I take claims that individuals are not getting much from models at face value. But that some people are not finding the value isn’t an argument that those of us getting value, increasing value isn’t real.

wellthisisgreat 11mo ago
Yeah after reading the intro lines I noped out of this garbage blog (?). Maybe it's their SEO strategy to write the flat-out untrue incendiary stuff like "the technology is essentially a failure" - If that's the case, I don't have time for this. Maybe they actually think this way, then, I certainly have time for this. what I have time for? virtual bonding with fello HN ppl :)
Sevii 11mo ago
We're getting improvements coming out month by month. No reasonable person would look at a technology improving at this frequency and write a blog post about how we've hit the ceiling.
lm28469 11mo ago
There was value in leaded gas and asbestos insulation too, nobody denies that. You're blind to all the negative side effects, AI generated slop ads, engagement traps, political propaganda, scams, &c. The amount of pollution is incredible, search engines are dead, blogs are dead, YouTube is dead, social medias are dead, it's virtually impossible to find non slop content, the ratio is probably already 50:1 by now And these are only the most visible things, I know a few companies losing hundreds of hours every month replying to support tickets that are fully llm generated an more often than not don't make any sense. Another big topic is education.

raincole 11mo ago

While both are unlikely, if I have to choose one I would bet on AGI than AI winter in the next five years. AI just got better and better. People thought it couldn't solve math problems without some human formalizes them first. Then it did. People thought it couldn't generate legible text. Then it did. All while people swore it had reached a "plateau," "architecture ceiling," "inherent limit," or whatever synonym of the goalpost.

aaroninsf 11mo ago

Every critique of AI assumes to some degree that contemporary implementations will not, or cannot, be improved upon. Lemma: any statement about AI which uses the word "never" to preclude some feature from future realization is false. Lemma: contemporary implementations have almost always already been improved upon, but are unevenly distributed. (Ximm's Law)

light_hue_1 11mo ago

As someone who is an expert in the area, everything in this article is misleading nonsense, failing at even the most basic CS101 principles. The level of confusion here is astounding. > People were saying that this meant that the AI winter was over The last AI winter was over 20 years ago. Transformers came during an AI boom. > First time around, AI was largely symbolic Neural networks were already hot and the state of the art across many disciplines when Transformers came out. > The other huge problem with traditional AI was that many of its algorithms were NP-complete Algorithms are not NP-complete. That's a type error. Problems can be NP-complete, not algorithms. > with the algorithm taking an arbitrarily long time to terminate This has no relationship to something being NP-complete at all. > but I strongly suspect that 'true AI', for useful definitions of that term, is at best NP-complete, possibly much worse I think the author means that "true AI" returns answers quickly and with high accuracy? A statement that has no relationship to NP-completeness at all. > For the uninitiated, a transformer is basically a big pile of linear algebra that takes a sequence of tokens and computes the likeliest next token This is wrong on many levels. A Transformer is not a linear network, linear networks are well-characterized and they aren't powerful enough to do much. It's the non-linearities in the Transformer that allows it to work. And only Decoders compute the distribution over the next token. > More specifically, they are fed one token at a time, which builds an internal state that ultimately guides the generation of the next token Totally wrong. This is why Transformers killed RNNs. Transformers are provided all tokens simultaneously and then produce a next token one at a time. RNNs don't have that ability to simultaneously process tokens. This is just totally the wrong mental model of what a Transformer is. > This sounds bizarre and probably impossible, but the huge research breakthrough was figuring out that, by starting with essentially random coefficients (weights and biases) in the linear algebra, and during training back-propagating errors, these weights and biases could eventually converge on something that worked. Again, totally wrong. Gradient descent dates back to the late 1800s early 1900s. Backprop dates back to the 60s and 70s. So this clearly wasn't the key breakthrough of Transformers. > This inner loop isn't Turing-complete – a simple program with a while loop in it is computationally more powerful. If you allow a transformer to keep generating tokens indefinitely this is probably Turing-complete, though nobody actually does that because of the cost. This isn't what Turing-completeness is. And by definition all practical computing is not a Turing Machine, simply because TMs require an infinite tape. Our actual machines are all roughly Linear Bounded Automata. What's interesting is that this doesn't really provide us with anything useful. > Transformers also solved scaling, because their training can be unsupervised Unsupervised methods predate Transformers by decades and were already the state of the art in computer vision by the time Transformers came out. > In practice, the transformer actually generates a number for every possible output token, with the highest number being chosen in order to determine the token. Greedy decoding isn't the default in most applications. > The problem with this approach is that the model will always generate a token, regardless of whether the context has anything to do with its training data. Absolutely not. We have things like end tokens exactly for this, to allow the model to stop generating. I got tired of reading at this point. This is drivel by someone who has no clue what's going on.

siva7 11mo ago

Is "A New AI Winter Is Coming" the new "The Year of Linux Desktop"?

_se 11mo ago
No, the new "Year of the Linux Desktop" is "This Year Software Engineers Won't Exist Any More Because Of LLMs". Very obviously.

bdangubic 11mo ago

> it's a horrible liability to have to maintain a codebase that nobody on the team actually authored. exactly - just like 99.9873% of all codebases currently running in production worldwide :)

aoeusnth1 11mo ago

Token economics are very sound - it's the training which is expensive. Tokens/week have gone up 23x year-over-year according to https://openrouter.ai/rankings https://openrouter.ai/rankings. This is probably around $500M-1B in sales per year. The real question is where the trajectory of this rocket ship is going. Will per-token pricing be a race to the bottom against budget chinese model providers? Will we see another 20x year year over the next 3 years, or will it level out sooner?

tim333 11mo ago
Yeah, that doesn't look at all wintery. More exponential.

andreyk 11mo ago

This blog post is full of bizarre statements and the author seems almost entirely ignorant of the history or present of AI. I think it's fair to argue there may be an AI bubble that will burst, but this blog post is plainly wrong in many ways. Here's a few clarifications (sorry this is so long...): "I should explain for anyone who hasn't heard that term [AI winter]... there was much hope, as there is now, but ultimately the technology stagnated. " The term AI winter typically refers to a period of reduced funding for AI research/development, not the technology stagnating (the technology failing to deliver on expectations was the cause of the AI winter, not the definition of AI winter). "[When GPT3 came out, pre-ChatGPT] People were saying that this meant that the AI winter was over, and a new era was beginning." People tend to agree there were two AI winters already, one having to do with symbolic AI disappointments/general lack of progress (70s), and the latter related to expert systems (late 80s). That AI winter has long been over. The Deep Learning revolution started in ~2012, and by 2020 (GPT 3) huge amount of talent and money were already going into AI for years. This trend just accelerated with ChatGPT. "[After symbolic AI] So then came transformers. Seemingly capable of true AI, or, at least, scaling to being good enough to be called true AI, with astonishing capabilities ... the huge research breakthrough was figuring out that, by starting with essentially random coefficients (weights and biases) in the linear algebra, and during training back-propagating errors, these weights and biases could eventually converge on something that worked." Transformers came about in 2017. The first wave of excitement about neural nets and backpropagation goes all the way back to the late 80s/early 90s, and AI (computer vision, NLP, to a lesser extent robotics) were already heavily ML-based by the 2000s, just not neural-net based (this changed in roughly 2012). "All transformers have a fundamental limitation, which can not be eliminated by scaling to larger models, more training data or better fine-tuning ... This is the root of the hallucination problem in transformers, and is unsolveable because hallucinating is all that transformers can do." The 'highest number' token is not necessarily chosen, this depends on the decoding algorithm. That aside, 'the next token will be generated to match that bad choice' makes it sound like once you generate one 'wrong' token the rest of the output is also wrong. A token is a few characters, and need not 'poison' the rest of the output. That aside, there are plenty of ways to 'recover' from starting to go down the wrong route. A key aspect of why reasoning in LLMs works well is that it typically incorporates backtracking - going earlier in the reasoning to verify details or whatnot. You can do uncertainty estimation in the decoding algorithm, use a secondary model, plenty of things (here is a detailed survey https://arxiv.org/pdf/2311.05232 https://arxiv.org/pdf/2311.05232 , one of several that is easy to find). "The technology won't disappear – existing models, particularly in the open source domain, will still be available, and will still be used, but expect a few 'killer app' use cases to remain, with the rest falling away." A quick google search shows ChatGPT currently has 800 million weekly active users who are using it for all sorts of things. AI-assisted programming is certainly here to stay, and there are plenty of other industries in which AI will be part of the workflow (helping do research, take notes, summarize, build presentations, etc.) I think discussion is good, but it's disappointing to see stuff with this level of accuracy being on front page of HN.

RamblingCTO 11mo ago

I'm so annoyed by this negativity. Is AI perfect? No, far from it. Does it have a positive impact on productivity? 100%. Do I care about financials? Absolutely not. There's the regular hype cycle. But these idiotic takes like "dotcom 2.0" and "AI winter" are just and show that the author has no clue what they are talking about.

monkaiju 11mo ago
That 100% claim is doing an awful lot of work. Theres is ample documentation from most corporate rollouts that show the opposite, and as a SWE I find it worse than useless...

johnfn 11mo ago

I’m fascinated by people who say that LLMs have failed in practice. Last week, when I was on PTO, I used AI to to a full redesign of a music community website I run. I touched about 40k lines of code in a week. The redesign is shipped and everyone is using it. AI let me go about 5-10x faster than if I would have done this by hand. (In fact, I have tried doing this in the past, so I really do have an apples to apples comparison for velocity. AI enabled it happening at all: I’ve tried a few other times in the past but never been able to squeeze it into a week.) The cited 40% inaccuracy rate doesn’t track for me at all. Claude basically one-shot anything I asked for, to the point that the bottleneck was mostly thinking of what I should ask it to do next. At this point, saying AI has failed feels like denying reality.

svaha1728 11mo ago
On a music blog, yes! Now go try to rewrite the firmware for your car.
- reducesuffering 11mo ago
  95% of the work engineers on this forum do are high level apps like a "music blog", and not car firmware...
  - l1ng0 11mo ago
    And then there's the small % doing music firmware :) (and no, I don't find LLMs much use on this)
- johnfn 11mo ago
  It's not a blog - it's a community website, so significantly more complex (running competitions, uploading tracks, moderation, etc, etc, etc). I don't understand the car firmware non-sequitur. I've already seen productivity gains in this domain. Why do I need to also prove them in another one? They are already demonstrated. It's like me saying "I can get places faster in a car than on foot" and you're saying "yeah but you can't get to the moon."
11mo ago

injidup 11mo ago

Most “AI is rubbish” takes treat it as an open-loop system: prompt → code → judgment. That’s not how development works. Even humans can’t read a spec, dump code, and ship it. Real work is closed-loop: test, compare to spec, refine, repeat. AI shines in that iterative feedback cycle, which is where these critiques miss the point.

phantasmish 11mo ago
What’s tough for me is figuring out where people are realizing significant improvements from this. If you have to set up good tests [edit: and gather/generate good test data!] and get the spec hammered out in detail and well-described in writing, plus all the ancillary stuff like access to any systems you need, sign-offs from stakeholders… dude that’s more than 90% of the work, I’d say. I mean fuck, lots of places just skip half that and figure it out in the code as they go. How’s this meaningfully speeding things up?

screye 11mo ago

> This means that they should never be used in medicine, for evaluation in school or college, for law enforcement, for tax assessment, or a myriad of other similar cases. If AI models can deliver measurably better accuracy than doctors, clearer evaluations than professors and fairer prosecutions than courts, then it should be adopted. Waymo has already shown a measurable decrease in loss of life by eliminating humans from driving. I believe, technically, moderns LLMs are sufficiently advanced to meaningfully disrupt the aforementioned professions as Waymo has done for taxis. Waymo's success relies on 2 non-llm factors that we've yet to see for other professions. First is exhaustive collection and labelling of in-domain high quality data. Second is the destruction of the pro-human regulatory lobby (thanks to work done by Uber in the Zirp era that came before). To me, an AI winter isn't a concern, because AI is not the bottleneck. It is regulatory opposition and sourcing human experts who will train their own replacements. Both are significantly harder to get around for high-status white collar work. The great-AI-replacement may still fail, but it won't be because of the limitations of LLMs. > My advice: unwind as much exposure as possible you might have to a forthcoming AI bubble crash. Hedging when you have much at stake is always a good idea. Bubble or no bubble.

josefritzishere 11mo ago

I trully hate how absurdly over-hyped this LLM concept of AI is. However, it can do some cool things. Normally that's the path to PMF. The real problem is the revenue. There is no extant revenue model whatsoever. But there are industries that are described as "a business of pennies" Ex. telephony. Someone may yet eek out a win. But the hype-to-reality conversion will come first.

czhu12 11mo ago

I think its worth discounting against the failure rates of humans also. > Depending on the context, and how picky you need to be about recognizing good or bad output, this might be anywhere from a 60% to a 95% success rate, with the remaining 5%-40% being bad results. This just isn't good enough for most practical purposes This seems to suggest that humans are 100%. I'd be surprised if i was anywhere close to that after 10 years of programming professionally

hadlock 11mo ago

> AI has failed. >The rumor mill has it that about 95% of generative AI projects in the corporate world are failures. AI tooling has only just barely reached the point where enterprise CRUD developers can start thinking about. Langchain only reached v1.0.0 in the last 60 days (Q4 2025); OpenAI effectively announced support for MCP in Q2 2025. The spec didn't even approach maturity until Q4 of 2024. Heck most LLMs didn't have support for tools in 2024. In 2-3 years a lot of these libraries will be part way through their roadmap towards v2.0.0 to fix many of the pain points and fleshing out QOL improvements, and standard patterns evolved for integrating different workflows. Consumer streaming of audio and video on the web was a disaster of a mess until around ~2009 despite browsers having plugins for it going back over a decade. LLMs continue to improve at a rapid rate, but tooling matures more slowly. Of course previous experiments failed or were abandoned; the technology has been moving faster than the average CRUD developer can implement features. A lot of "cutting edge" technology we put into our product in 2023 are now standard features for the free tier of market leaders like ChatGPT etc. Why bother maintaining a custom fork of 2023-era (effectively stone age) technology when free tier APIs do it better in 2025? MCP might not be the be-all, end-all, but at least it is a standard interface that's at least maintainable in a way that developers of mature software can begin conceiving of integrating it into their product as a permanent feature, rather than a curiosity MVP at the behest of a non technical exec. A lot of AI-adjacent libraries we've been using finally hit v1.0.0 this year, or creeping close to it; providing stable interfaces for maintainable software. It's time to hit the reset button on "X% of internal AI initiatives failed"

jgb1984 11mo ago

I sure hope another winter is on the way, the LLM hype wagon is beyond exhausting.

ivanche 11mo ago

These sort of articles are new "economists have predicted 17 of the past 3 recessions".

bitwize 11mo ago

Eric S. Raymond (yes, that ESR; insert long-winded DonHopkins rant, extensively cross-referenced to his own USENET and UNIX-HATERS mailing list postings, about what a horrible person "Eric the Flute" is) reports a shortening of time to implement his latest project from weeks to hours. He says the advantages of LLMs in the programming space are yet to truly unfold, because they not only let you do things faster, they let you try things you wouldn't have started otherwise because it'd be too much of a time/effort commitment. Assuming these claims are even partially true, we'd be stupid—at the personal and societal level—not to avail ourselves of these tools and reap the productivity gains. So I don't see AI going away any time soon. Nor will it be a passing fad like Krugman assumed the internet would be. We'd have to course-correct on its usage, but it truly is a game changer.

1vuio0pswjnm7 11mo ago

As illustrated by the top comment, it seems that "AI" relies on hype, i.e., careful manipulation of how it is perceived. Its continued existence depends on relentless fund raising because generally no one is going to pay fees to use it long-term As with so much of Silicon Valley's output the past few decades, "AI" is not something people will pay for. It could be a new form of Trojan Horse for data collection, surveillance and ads, though, adopting a hyperfocus on "growth" at any cost (human, environmental, etc.) but this might not even be necessary^1 Time will tell In order to raise funds in the absence of any clear business plan, "AI" cannot just be ordinary and useful. It cannot be "autocomplete on steroids". It must be "world-changing" No one shall underestimate or discount the value of "AI". Overestimation is permitted, however. Speculation is encouraged, but only if it's wild and extreme 1. "AI" is not depending on being brought inside the gates of Troy by the city's inhabitants. It's being placed inside the gates by others. Companies with large numbers of users are installing "AI" on users' computers without the users' consent

bunderbunder 11mo ago

I've run into what I think might be an even bigger problem with the "just keep reading/outputting the next token" design: there's no going backwards. This means that LLMs don't do well with input where text that comes later in the input alters how text that came earlier should be interpreted. Which is really quite common in practice. Humans like to do things where you start with a dump of information, and then ask questions about it afterward. If a human reader didn't remember something pertinent to the question, they can go back and re-read now that they know what they're looking for. An LLM can't do that by itself. You can paper over that limitation with hacks, but they're hacky and unreliable. Humans also correct misunderstandings and resolve ambiguities after the fact. LLMs have a really hard time with this. Once one way of interpreting some text has already made it into the LLM's context vectors, it can be surprisingly difficult for subsequent text to significantly alter that interpretation. It's these sorts of aggravating failure modes stemming from a fundamental feature of the algorithm that seem to always produce the next AI winter. Because they tend to mean the same thing: research has gone down (and then, quote often, massively over-invested in) yet another blind alley that doesn't put us on a short easy path to AGI any more than the previous 75 years' worth of breakthroughs have. And getting out of it is going to involve not only fixing that one problem, but also a huge reset on some substantial portion of whatever tools and techniques were built up to paper over the previous algorithm's limitations in an effort to generate positive ROI on all that massive over-investment.

octoberfranklin 11mo ago
No, that's not how a Transformer works. It gets the entire input all at once. Then it generates the output one token at a time.

starchild3001 11mo ago

The author is conflating a financial correction with a technological failure. I agree that the economics of GenAI are currently upside down. The CapEx spend is eye-watering, and the path to profitability for the foundational model providers is still hazy. We are almost certainly in an age of inflated-expectations hype-cycle peak that will self-correct, and yes, "winter is harsh on tulips". However, the claim that the technology itself is a failure is objectively disconnected from reality. Unlike crypto or VR (in their hype cycles), LLMs found immediate, massive product-market fit. I use K-means clustering and logistic regression every day; they aren't AGI either, but they aren't failures. If 95% of corporate AI projects fail, it's not because the tech is broken; it's because middle management is aspiring to replace humans with a terminal-bound chatbot instead of giving workers an AI companion. The tech isn't going away, even if AI valuations might be questioned in the short term.

babblingfish 11mo ago

Winter is coming[1] [1] https://www.youtube.com/watch?v=gxhknGARGt4 https://www.youtube.com/watch?v=gxhknGARGt4

kazinator 11mo ago

I'm skeptical for this reason: at the time of the first AI winter, AI was niche. Today, it is mainstream; vast numbers of consumers are using AI. AI in consumer hands is no longer just a niche program in a corner of your company or government organization that can defund and shut down. If the current purveyors of big cloud AI were to fold, their replacements are waiting in the wings to pick up the demand somehow.

zkmon 11mo ago

Not quite. The core capability of extracting a meaning out of unstructured text and media, and the reverse of it, would remain as a bedrock. Anything else built around it (agents etc) might be challenged by the realities.

tim333 11mo ago

>A new AI winter is coming. Counterpoint - no it isn't. The NP arguments are dumb and have nothing to do with thinking. I don't think Einstein was NP complete but could still think. Winter summer implies sine wave like progress but what we are seeing now is a result of exponential growth in computer hardware such that it is now passing human equivalent levels https://www.bvp.com/assets/uploads/2024/03/price-performance-slide-ray-kurzweil-atlas-image.jpg https://www.bvp.com/assets/uploads/2024/03/price-performance... LLMs having failings is a blip. Hinton got his nobel prize for work done in the 1980s but we are seeing AI takeoff now 45 years later, almost exactly when predicted by the likes of Moravec/Kurweil because of the hardware progress. (see the token use graph mentioned by aoeusnth1 - https://openrouter.ai/rankings https://openrouter.ai/rankings usage looking exponential too)

kittikitti 11mo ago

This is a good article and although it does sound like yet another AI doomer piece, it at least tackled it from a unique perspective. AI winter is coming and it's going to be harsh for those who have never heard of an AI winter before the AI summer that started in 2017. This isn't a tulip-mania that's it's so often compared to. People find these concepts distilled down to one neuron and think they know what they're talking about. This AI winter is going to be harsh with economic catastrophes because of climate change and political instability. There might even be another nuclear scare. Will they blame AI for all their problems? Definitely.

rdiddly 11mo ago

To the author's brief list of areas where AI can't be used because it's too risky, I would add "driving on the public roads," but like everything else on the list, it's currently being done anyway.

arisAlexis 11mo ago

Every article with this title is either a human psychology cry for a) we are going too fast I wish we could stop or b) I delight myself to be a contrarian

signa11 11mo ago

this is just _excellent_ ``` I would conjecture that this is another manifestation of the NP-completeness wall that slammed symbolic AI, causing the first AI winter. It's always possible to turn an NP-complete algorithm into one that runs quickly, if you don't mind that it fails to generate any output if you hit a timeout. The transformer equivalent of this is generating plausible, wrong, hallucinated output in cases where it can't pattern match a good result based on its training. The problem, though, is that with traditional AI algorithms you typically know if you've hit a timeout, or if none of your knowledge rules match. With transformers, generating wrong output looks exactly like generating correct output, and there is no way to know which is which. ```

Havoc 11mo ago

What this misses imo is that even if only 10% of tasks out there are suitable that is still a crazy big number economically. Possibly even big enough to plug the hole in OpenAI’s math

Olivia8 11mo ago

I too was excited when transformer-based models burst onto the scene — chatbots suddenly seemed smarter and more flexible than the old rule-based AI systems. Those early models were rough around the edges and sometimes produced garbage, but when they hit, it really felt like a step forward. The problem now is that these models always pick a next token — even when they have no good basis for it — so they can generate plausible but wrong or misleading content, and you often can’t tell the difference. That fundamental limitation means they aren’t reliable enough for high-stakes tasks like medicine, law, or financial decisions. Meanwhile, voice-based AI has also been growing rapidly this year, Coldi AI is a good example, with many more people and companies adopting assistants and voice interfaces.