8 ms·

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LL

This article uses the computational complexity hammer way too hard, discounts huge progress in every field of AI outside of the hot trend of transformers and LLMs. Nobody is saying the future of AI is autoregressive and this article pretty much ignores any of the research that has been posted here around diffusion based text generation or how it can be combined with autoregressive methods… discounts multi-modal models entirely. He also pretty much discounts everything that’s happened with AlphaFold, Alpha Go etc. reinforcement learning etc. The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why. Is the brain NP complete? Maybe maybe not. I could see many arguments about why modern research will fail to create AGI but just hand waving “reality is NP-hard” is not enough. The fact is: something fundamental has changed that enables a computer to pretty effectively understand natural language. That’s a discovery on the scale of the internet or google search and shouldn’t be discounted… and usage proves it. In 2 years there is a platform with billions of users. On top of that huge fields of new research are making leaps and bounds with novel methods utilizing AI for chemistry, computational geometry, biology etc. It’s a paradigm shift.

gishh 11mo ago
> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't have an agenda.
- sukhdeepprashit 11mo ago
  [dead]
- danielvaughn 11mo ago
  I think it’s a disingenuous read to assume original commenter means “understanding” in the literal sense. When we talk about LLM “understanding”, we usually mean it from a practical sense. If you give an input to the computer, and it gives you an expected output, then colloquially the computer “understood” your input.
- pawelduda 11mo ago
  The end effect certainly gives off "understanding" vibe. Even if method of achieving it is different. The commenter obviously didn't mean the way human brain understands
- frotaur 11mo ago
  It astonishes me how people can make categorical judgements on things as hard to define as 'understanding'. I would say that, except for the observable and testable performance, what else can you say about understanding? It is a fact that LLMs are getting better at many tasks. From their performance, they seem to have an understanding of say python. The mechanistic way this understanding arises is different than humans. How can you say then it is 'not real', without invoking the hard problem of consciousness, at which point, we've hit a completely open question.
  - deleted 11mo ago
    [deleted]
  - phantasmish 11mo ago
    To be fair, it can be hard to define “chair” to the satisfaction of an unsympathetic judge. “Do chairs exist?”: https://m.youtube.com/watch?v=fXW-QjBsruE https://m.youtube.com/watch?v=fXW-QjBsruE
  - matthewkayin 11mo ago
    I think it is fair to say that AIs do not yet "understand" what they say or what we ask them. When I ask it to use a specific MCP to complete a certain task, and it proceeds to not use that MCP, this indicates a clear lack of understanding. You might say that the fault was mine, that I didn't setup or initialize the MCP tool properly, but wouldn't an understanding AI recognize that it didn't have access to the MCP and tell me that it cannot satisfy my request, rather than blindly carrying on without it? LLMs consistently prove that they lack the ability to evaluate statements for truth. They lack, as well, an awareness of their unknowing, because they are not trying to understand; their job is to generate (to hallucinate). It astonishes me that people can be so blind to this weakness of the tool. And when we raise concerns, people always say "How can you define what 'thinking' is?" "How can you define 'understanding'?" These philosophical questions are missing the point. When we say it doesn't "understand", we mean that it doesn't do what we ask. It isn't reliable. It isn't as useful to us as perhaps it has been to you.
  - deleted 11mo ago
    [deleted]
- ilikeatari 11mo ago
  We could use a little more kindness in discussion. I think the commenter has a very solid understanding on how computer works. The “understanding” is somewhat complex but I do agree with you that we are not there yet. I do think that the paradigm shift though is more about the fact that now we can interact with the computer in a new way.
- neom 11mo ago
  Birds and planes operate using somewhat different mechanics, but they do both achieve flight.
  - HPsquared 11mo ago
    Birds and planes are very similar other than the propulsion and landing gear, and construction materials. Maybe bird vs helicopter, or bird vs rocket.
    - mejutoco 11mo ago
      > other than the propulsion and landing gear, and construction materials "Apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, the fresh water system and public health, what have the Romans ever done for us?" Monty Python's Life of Brian. P.S. It is relative, but quite a lot of differences IMHO.
      - HPsquared 11mo ago
        Yes indeed, but they still use wings, fly in the air and so on. Artificial neural networks have very little in common with real brains and have no structural or functional similarities besides "they process information, and they have things called neurons". They can perform some of the same tasks though, like how a quadcopter can perform some of the duties as a homing pigeon.
- Uehreka 11mo ago
  “You understand how the brain works right? It’s neurons and electrical charges. The brain understands nothing.” I’m always struck by how confidently people assert stuff like this, as if the fact that we can easily comprehend the low-level structure somehow invalidates the reality of the higher-level structures. As if we know concretely that the human mind is something other than emergent complexity arising from simpler mechanics. I’m not necessarily saying these machines are “thinking”. I wish I could say for sure that they’re not, but that would be dishonest: I feel like they aren’t thinking, but I have no evidence to back that up, and I haven’t seen non-self-referential evidence from anyone else.
  - claytongulick 11mo ago
    You understand how reality works right? It's all just atoms clinging to each other. Simple. Heh.
- Lambdanaut 11mo ago
  You understand how the brain works right? It's probability distributions mapped to sodium ion channels. The human understands nothing.
  - 0xdeadbeefbabe 11mo ago
    I've heard that this human brain is rigged to find what it wants to find.
  - aydyn 11mo ago
    Thats how the brain works, not how the mind works. We understand the hardware, not the software.
    - kipchak 11mo ago
      Are we even sure we understand the hardware? My understanding is even that is contested, for example orchestrated objective reduction, holonomic brain theory or GVF theory.
  - claytongulick 11mo ago
    You understand how the brain works? You're the one then. All those laggardly neurobiologists are still struggling.
- Eddy_Viscosity2 11mo ago
  It could very well be that statistics and tokens is how our brains work at the computational level too. Just that our algorithms have slightly better heuristics due to all those millennia of A/B testing of our ancestors.
  - emp17344 11mo ago
    Except we know for a fact that the brain doesn’t work that way. You’re ignoring the entire history of neuroscience.
    - Eddy_Viscosity2 11mo ago
      I didn't know that neuroscience even claimed to a full enough understanding of how people think to conclusively disprove this.
      - emp17344 11mo ago
        Certainly not a full understanding, but there’s enough understanding to refute vague claims that we are somehow analogous to a text model.
- LatencyKills 11mo ago
  As someone who was an engineer on the original Copilot team, yes I understand how tech works. You don’t know how your own mind “understands” something. No one on the planet can even describe how human understanding works. Yes, LLMs are vast statistical engines but that doesn’t mean something interesting isn’t going on. At this point I’d argue that humans “hallucinate” and/or provide wrong answers far more often than SOTA LLMs. I expect to see responses like yours on Reddit, not HN.
  - gishh 11mo ago
    > I expect to see responses like yours on Reddit, not HN. I suppose that says something about both of us.
    - szundi 11mo ago
      [dead]
  - szundi 11mo ago
    [dead]
  - 6510 11mo ago
    Before one may begin to understand something one must first be able to estimate the level of certainty. Our robot friends, while really helpful and polite, seem to be lacking in that department. They actually think the things we've written on the internet, in books, academic papers, court documents, newspapers, etc are actually true. Where the humans aren't omniscient it fills the blanks with nonsense.
    - LatencyKills 11mo ago
      > Where the humans aren't omniscient it fills the blanks with nonsense As do most humans. People lie. People make things up to look smart. People fervently believe things that are easily disproved. Some people are willfully ignorant, anti-science, anti-education, etc. The problem isn't the transformer architecture... it is the humans who advertise capabilities that are not there yet.
- goncharom 11mo ago
  Every time I see comments like these I think about this research from anthropic: https://www.anthropic.com/research/mapping-mind-language-model https://www.anthropic.com/research/mapping-mind-language-mod... LLMs activate similar neurons for similar concepts not only across languages, but also across input types. I’d like to know if you’d consider that as a good representation of “understanding” and if not, how would you define it?
  - gishh 11mo ago
    If i could understand what the brain scans actually meant, I would consider it a good representation. I don't think we know yet what they mean. I saw some headline the other day about a person with "low brain activity" and said person was in complete denial about it, I would be too.
    - goncharom 11mo ago
      As I said then, and probably echoing what other commenters are saying - what do you mean by understanding when you say computers understand nothing? do humans understand anything? if so, how?
      - gishh 11mo ago
        Does a computer understand how hot or cold it is outside? Does it understand that your offspring might be cranky because they’re hungry? Or not hungry, just tired? Can it divine the difference? Does a computer know if your boss is mad at you or if they had a fight with their spouse last night, or whatever other reason they may be grumpy? Can a computer establish relationships… with anything? How about when a computer goes through puberty? Or menopause? Or a car accident? How do those things affect them? Don’t bother responding, I think you get the point.
  - emp17344 11mo ago
    Anthropic is pretty notorious for peddling hype. This is a marketing article - it has not undergone peer-review and should not be mistaken for scientific research.
- deleted 11mo ago
  [deleted]
- stanfordkid 11mo ago
  What do you mean by “understand”? Do you mean conscious? Understand just means “parse language” and is highly subjective. If I talk to someone African in Chinese they do not understand me but they are still conscious. If I talk to an LLM in Chinese it will understand me but that doesn’t mean it is conscious. If I talk about physics to a kindergartner they will not understand but that doesn’t mean they don’t understand anything. Do you see where I am going?
- observationist 11mo ago
  You don't understand how the tech works, then. LLMs aren't as good as humans at understanding, but it's not just statistics. The stochastic parrot meme is wrong. The networks create symbolic representations in training, with huge multidimensional correlations between patterns in the data, whether its temporal or semantic. The models "understand" concepts like emotions, text, physics, arbitrary social rules and phenomena, and anything else present in the data and context in the same fundamental way that humans do it. We're just better, with representations a few orders of magnitude higher resolution, much wider redundancy, and multi-million node parallelism with asynchronous operation that silicon can't quite match yet. In some cases, AI is superhuman, and uses better constructs than humans are capable of, in other cases, it uses hacks and shortcuts in representations, mimics where it falls short, and in some cases fails entirely, and has a suite of failure modes that aren't anywhere in the human taxonomy of operation. LLMs and AI aren't identical to human cognition, but there's a hell of a lot of overlap, and the stochastic parrot "ItS jUsT sTaTiStIcS!11!!" meme should be regarded as an embarrassing opinion to hold. "Thinking" models that cycle context and systems of problem solving also don't do it the same way humans think, but overlap in some of the important pieces of how we operate. We are many orders of magnitude beyond old ALICE bots and MEgaHAL markov chains - you'd need computers the size of solar systems to run a markov chain equivalent to the effective equivalent 40B LLM, let alone one of the frontier models, and those performance gains are objectively within the domain of "intelligence." We're pushing the theory and practice of AI and ML squarely into the domain of architectures and behaviors that qualify biological intelligence, and the state of the art models clearly demonstrate their capabilities accordingly. For any definition of understanding you care to lay down, there's significant overlap between the way human brains do it and the way LLMs do it. LLMs are specifically designed to model constructs from data, and to model the systems that produce the data they're trained on, and the data they model comes from humans and human processes.
  - pwndByDeath 11mo ago
    You appear to be a proper alchemist, but you can't support an argument of understanding if there is no definition of understanding that isn't circular. If you want to believe the friendly voice really understands you, we have a word for that, faith. The skeptic sees the interactions with a chatbot as a statistical game that shows how uninteresting (e.g. predictable) humans and our stupid language are. There are useful gimmicks coming out like natural language processing, for low risk applications, but this form of AI pseudoscience isn't going to survive, but it will take some time for research to catch up to understanding how to describe the falsehoods of contemporary AI toys
- deepGem 11mo ago
  "I don't "understand" how LLMs "understand" anything." Why does the LLM need to understand anything. What today's chatbots have achieved is a software engineering feat. They have taken a stateless token generation machine that has compressed the entire internet's vocabulary to predict the next token and have 'hacked' a whole state management machinery around it. End result is a product that just feels like another human conversing with you and remembering your last birthday. Engineering will surely get better and while purists can argue that a new research perspective is needed, the current growth trajectory of chatbots, agents and code generation tools will carry the torch forward for years to come. If you ask me, this new AI winter will thaw in the atmosphere even before it settles on the ground.
john61 11mo ago
> The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why. OP says it is because that predicting the next token can be correct or not, but it always looks plausible because that is what it calculates. Therefore it is dangerous and can not be fixed because it is how it works in essence.
- dangus 11mo ago
  I just want to point out a random anecdote. Literally yesterday ChatGPT hallucinated an entire feature of a mod for a video game I am playing including making up a fake console command. It just straight up doesn’t exist, it just seemed like a relatively plausible thing to exist. This is still happening. It never stopped happening. I don’t even see a real slowdown in how often it happens. It sometimes feels like the only thing saving LLMs are when they’re forced to tap into a better system like running a search engine query.
  - bitwize 11mo ago
    [flagged]
    - bcrosby95 11mo ago
      Responding with "skill issue" in a discussion is itself a skill issue. Maybe invest in some conversational skills and learn to be constructive rather than parroting a useless meme.
    - dingnuts 11mo ago
      [dead]
    - dangus 11mo ago
      So you are saying the acceptable customer experience for these systems is that we need to explicitly tell them to accept defeat when they can’t find any training content/web search results that matches my query enough? Why don't they have any concept of having a percentage of confidence in their answer? It isn’t 2022 anymore, this is supposed to be a mature product. Why am I even using this thing rather than using the game’s own mod database search tool? Or the wiki documentation? What value is this system adding for me if I’m supposed to be a prompt engineer?
andy99 11mo ago
I agree with everything you wrote, the technology is unbelievable and 6 years ago, maybe even 3.1 years would have been considered magic. A steel man argument for why winter might be coming is all the dumb stuff companies are pushing AI for. On one hand (and I believe this) we argue it’s the most consequential technology in generations. On the other, everybody is using it for nonsense like helping you write an email that makes you sound like an empty suit, or providing a summary you didn’t ask for. There’s still a ton of product work to cross whatever that valleys called between concept and product, and if that doesn’t happen, money is going to start disappearing. The valuation isn’t justified by the dumb stuff we do with it, it needs PMF.
- baq 11mo ago
  > maybe even 3.1 years would have been considered magic I vividly remember sitting there for literally hours talking to a computer on launch day, it was a very short night. This feeling of living in the future has not left me since, it's got quieter, but it's still there. It's still magic after those three years, perhaps even more so. It wasn't supposed to work this well for decades! and yet.
  - bloppe 11mo ago
    I think you're missing the point of "AI winter". It's not about how good the products are now. It's about how quickly the products are improving and creating the potential for profit. That's what drives investment. 3 things we know about the AI revolution in 2025: - LLMs are amazing, but they have reached a plateau. AGI is not within reach. - LLM investment has sacrificed many hundreds of billions of dollars, much of it from the world's pension funds. - There is no credible path to a high-margin LLM product. Margins will be razor-thin positive at best once the free trial of the century starts to run out of steam. This all adds up to a rather nasty crunch. The thing about winter, though, is that it's eventually followed by another summer.
    - saltcured 11mo ago
      Many technologies plateau, but we don't say they all have winters. Terrestrial radio winter? Television winter? Automobile winter? Air travel winter? Nuclear power comes close in terms of its tarnished image and reluctance to reinvest. I personally believe contemporary AI is over-hyped, but I cannot say with confidence that it is going to lead to a similar winter as the last time. It seems like today's products satisfy enough users to remain as a significant area, even if it doesn't greatly expand... The only way I could see it fizzling as a product category is if it turns out it is not economically feasible to operate a sustainable service. Will users pay a fair price to keep the LLM datacenters running, without speculative investment subsidies? The other aspect of the winter is government investment, rather than commercial. What could the next cycle of academic AI research look like? E.g. exploration that needs to happen in grant-funded university labs instead of venture-funded companies? The federal funding picture seems unclear, but that's true across the board right now for reasons that have nothing to do with AI per se.
cess11 11mo ago
GOFAI was also a paradigm shift, regardless of that winter. For example, banks started automating assessments of creditworthiness. What we didn't get was what had been expected, namely things like expert systems that were actual experts, so called 'general intelligence' and war waged through 'blackboard systems'. We've had voice controlled electronics for a long time. On the other hand, machine vision applications have improved massively in certain niches, and also allowed for new forms of intense tyranny and surveillance where errors are actually considered a feature rather than a bug since they erode civil liberties and human rights but are still broadly accepted because 'computer says'. While you could likely argue "leaps and bounds with novel methods utilizing AI for chemistry, computational geometry, biology etc." by downplaying the first part or clarifying that it is mainly an expectation, I think most people are going to, for the foreseeable future, keep seeing "AI" as more or less synonymous with synthetic infantile chatbot personalities that substitute for human contact.
bloppe 11mo ago
I don't think anybody expects AI development to stop. A winter is defined by a relative drying-up of investment and, importantly, it's almost certain that any winter will eventually be followed by another summer. The pace of investment in the last 2 years has been so insane that even Altman has claimed that it's a bubble.
fragmede 11mo ago
> I could see many arguments about why modern research will fail to create AGI Why is AGI even necessary? If the loop between teaching the AI something, and it being able to repeat similar enough tasks; if that loop becomes short enough, days or hours instead of months, who cares if some ill-defined bar of AGI is met?
lamontcg 11mo ago
> that enables a computer to pretty effectively understand natural language I'd argue that it pretty effectively mimics natural language. I don't think it really understands anything, it is just the best madlibs generator that the world has ever seen. For many tasks, this is accurate 99+% of the time, and the failure cases may not matter. Most humans don't perform any better, and arguably regurgitate words without understanding as well. But if the failure cases matter, then there is no actual understanding and the language the model is generating isn't ever getting "marked to market/reality" because there's no mental world model to check against. That isn't going to be usable if there are real-world consequences of the LLM getting things wrong, and they can wind up making very basic mistakes that humans wouldn't make--because we can innately understand how the world works and aren't always just stringing words together that sound good.
- code_biologist 11mo ago
  Break the aspects of language understanding and language generation apart. While I would agree that generative LLMs are understanding-free madlibs for writing text, embedding vector spaces and LLM latent spaces seem are a pretty genuine understanding of natural language. High dimensional vector spaces seem like the best machine representation we currently have for meaning and LLMs are using it effectively.
Yoric 11mo ago
Technologically, I believe that you're right. On the other hands, the previous AI winters happened despite novel, useful technologies, some of which proved extremely useful and actually changed the world of software. They happened because of overhype, then investor moving on to the next opportunity. Here, the investors are investing in LLMs. Not in AlphaFold, AlphaGo, neurosymbolic, focus learning, etc. If (when) LLMs prove insufficient to the insane level of hype and if (when) experience shows that there is only so much money that you can make with LLMs, it's possible that the money will move on to other types of AI, but there are chances that it will actually go to something entirely different, perhaps quantum, leaving AI in winter.
encyclopedism 11mo ago
> The fact is: something fundamental has changed that enables a computer to pretty effectively understand natural language. I have commented elsewhere but this bears repeating LLM's do not think, understand, reason, reflect, comprehend and they never shall. If you had enough paper and ink and the patience to go through it, you could take all the training data and manually step through and train the same model. Then once you have trained the model you could use even more pen and paper to step through the correct prompts to arrive at the answer. All of this would be a completely mechanical process. This really does bear thinking about. It's amazing the results that LLM's are able to acheive. But let's not kid ourselves and start throwing about terms like AGI or emergence just yet. It makes a mechanical process seem magical (as do computers in general). I should add it also makes sense as to why it would, just look at the volume of human knowledge (the training data). It's the training data with the mass quite literally of mankind's knowledge, genius, logic, inferences, language and intellect that does the heavy lifting.
- lambdaone 11mo ago
  You could use a similar argument to argue that "brains do not think, understand, reason, reflect, comprehend and they never shall." After all, there's nothing in there but neurons, synapses and other biological gunk. If you look at it that way. Which you shouldn't.
  - encyclopedism 11mo ago
    That argument you posed does not follow. Brains do all of those things, at least I know mine does because those are the most intimate experiences I have. More intimate than even my sense experiences. It's what lead Kant to exclaim 'cogito ergo sum' and Ibn Sina before him to also note that fact. Moreover how the brain does what it does an open academic question. One of the most difficult, please see the Hard problem of consciousness for example. I can make certain determined judgments about things, my digital thermometer does not think or understand when it tells me the temperature, something I myself would be unable to determine. My digital LLM does not think for the same reason. Importantly my paper and pen version of that very same LLM would also not think.
ath3nd 11mo ago
[dead]

gishh 11mo ago

> something fundamental has changed that enables a computer to pretty effectively understand natural language. You understand how the tech works right? It's statistics and tokens. The computer understands nothing. Creating "understanding" would be a breakthrough. Edit: I wasn't trying to be a jerk. I sincerely wasn't. I don't "understand" how LLMs "understand" anything. I'd be super pumped to learn that bit. I don't have an agenda.

sukhdeepprashit 11mo ago
[dead]
danielvaughn 11mo ago
I think it’s a disingenuous read to assume original commenter means “understanding” in the literal sense. When we talk about LLM “understanding”, we usually mean it from a practical sense. If you give an input to the computer, and it gives you an expected output, then colloquially the computer “understood” your input.
pawelduda 11mo ago
The end effect certainly gives off "understanding" vibe. Even if method of achieving it is different. The commenter obviously didn't mean the way human brain understands
frotaur 11mo ago
It astonishes me how people can make categorical judgements on things as hard to define as 'understanding'. I would say that, except for the observable and testable performance, what else can you say about understanding? It is a fact that LLMs are getting better at many tasks. From their performance, they seem to have an understanding of say python. The mechanistic way this understanding arises is different than humans. How can you say then it is 'not real', without invoking the hard problem of consciousness, at which point, we've hit a completely open question.
- deleted 11mo ago
  [deleted]
- phantasmish 11mo ago
  To be fair, it can be hard to define “chair” to the satisfaction of an unsympathetic judge. “Do chairs exist?”: https://m.youtube.com/watch?v=fXW-QjBsruE https://m.youtube.com/watch?v=fXW-QjBsruE
- matthewkayin 11mo ago
  I think it is fair to say that AIs do not yet "understand" what they say or what we ask them. When I ask it to use a specific MCP to complete a certain task, and it proceeds to not use that MCP, this indicates a clear lack of understanding. You might say that the fault was mine, that I didn't setup or initialize the MCP tool properly, but wouldn't an understanding AI recognize that it didn't have access to the MCP and tell me that it cannot satisfy my request, rather than blindly carrying on without it? LLMs consistently prove that they lack the ability to evaluate statements for truth. They lack, as well, an awareness of their unknowing, because they are not trying to understand; their job is to generate (to hallucinate). It astonishes me that people can be so blind to this weakness of the tool. And when we raise concerns, people always say "How can you define what 'thinking' is?" "How can you define 'understanding'?" These philosophical questions are missing the point. When we say it doesn't "understand", we mean that it doesn't do what we ask. It isn't reliable. It isn't as useful to us as perhaps it has been to you.
- deleted 11mo ago
  [deleted]
ilikeatari 11mo ago
We could use a little more kindness in discussion. I think the commenter has a very solid understanding on how computer works. The “understanding” is somewhat complex but I do agree with you that we are not there yet. I do think that the paradigm shift though is more about the fact that now we can interact with the computer in a new way.
neom 11mo ago
Birds and planes operate using somewhat different mechanics, but they do both achieve flight.
- HPsquared 11mo ago
  Birds and planes are very similar other than the propulsion and landing gear, and construction materials. Maybe bird vs helicopter, or bird vs rocket.
  - mejutoco 11mo ago
    > other than the propulsion and landing gear, and construction materials "Apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, the fresh water system and public health, what have the Romans ever done for us?" Monty Python's Life of Brian. P.S. It is relative, but quite a lot of differences IMHO.
    - HPsquared 11mo ago
      Yes indeed, but they still use wings, fly in the air and so on. Artificial neural networks have very little in common with real brains and have no structural or functional similarities besides "they process information, and they have things called neurons". They can perform some of the same tasks though, like how a quadcopter can perform some of the duties as a homing pigeon.
Uehreka 11mo ago
“You understand how the brain works right? It’s neurons and electrical charges. The brain understands nothing.” I’m always struck by how confidently people assert stuff like this, as if the fact that we can easily comprehend the low-level structure somehow invalidates the reality of the higher-level structures. As if we know concretely that the human mind is something other than emergent complexity arising from simpler mechanics. I’m not necessarily saying these machines are “thinking”. I wish I could say for sure that they’re not, but that would be dishonest: I feel like they aren’t thinking, but I have no evidence to back that up, and I haven’t seen non-self-referential evidence from anyone else.
- claytongulick 11mo ago
  You understand how reality works right? It's all just atoms clinging to each other. Simple. Heh.
Lambdanaut 11mo ago
You understand how the brain works right? It's probability distributions mapped to sodium ion channels. The human understands nothing.
- 0xdeadbeefbabe 11mo ago
  I've heard that this human brain is rigged to find what it wants to find.
- aydyn 11mo ago
  Thats how the brain works, not how the mind works. We understand the hardware, not the software.
  - kipchak 11mo ago
    Are we even sure we understand the hardware? My understanding is even that is contested, for example orchestrated objective reduction, holonomic brain theory or GVF theory.
- claytongulick 11mo ago
  You understand how the brain works? You're the one then. All those laggardly neurobiologists are still struggling.
Eddy_Viscosity2 11mo ago
It could very well be that statistics and tokens is how our brains work at the computational level too. Just that our algorithms have slightly better heuristics due to all those millennia of A/B testing of our ancestors.
- emp17344 11mo ago
  Except we know for a fact that the brain doesn’t work that way. You’re ignoring the entire history of neuroscience.
  - Eddy_Viscosity2 11mo ago
    I didn't know that neuroscience even claimed to a full enough understanding of how people think to conclusively disprove this.
    - emp17344 11mo ago
      Certainly not a full understanding, but there’s enough understanding to refute vague claims that we are somehow analogous to a text model.
LatencyKills 11mo ago
As someone who was an engineer on the original Copilot team, yes I understand how tech works. You don’t know how your own mind “understands” something. No one on the planet can even describe how human understanding works. Yes, LLMs are vast statistical engines but that doesn’t mean something interesting isn’t going on. At this point I’d argue that humans “hallucinate” and/or provide wrong answers far more often than SOTA LLMs. I expect to see responses like yours on Reddit, not HN.
- gishh 11mo ago
  > I expect to see responses like yours on Reddit, not HN. I suppose that says something about both of us.
  - szundi 11mo ago
    [dead]
- szundi 11mo ago
  [dead]
- 6510 11mo ago
  Before one may begin to understand something one must first be able to estimate the level of certainty. Our robot friends, while really helpful and polite, seem to be lacking in that department. They actually think the things we've written on the internet, in books, academic papers, court documents, newspapers, etc are actually true. Where the humans aren't omniscient it fills the blanks with nonsense.
  - LatencyKills 11mo ago
    > Where the humans aren't omniscient it fills the blanks with nonsense As do most humans. People lie. People make things up to look smart. People fervently believe things that are easily disproved. Some people are willfully ignorant, anti-science, anti-education, etc. The problem isn't the transformer architecture... it is the humans who advertise capabilities that are not there yet.
goncharom 11mo ago
Every time I see comments like these I think about this research from anthropic: https://www.anthropic.com/research/mapping-mind-language-model https://www.anthropic.com/research/mapping-mind-language-mod... LLMs activate similar neurons for similar concepts not only across languages, but also across input types. I’d like to know if you’d consider that as a good representation of “understanding” and if not, how would you define it?
- gishh 11mo ago
  If i could understand what the brain scans actually meant, I would consider it a good representation. I don't think we know yet what they mean. I saw some headline the other day about a person with "low brain activity" and said person was in complete denial about it, I would be too.
  - goncharom 11mo ago
    As I said then, and probably echoing what other commenters are saying - what do you mean by understanding when you say computers understand nothing? do humans understand anything? if so, how?
    - gishh 11mo ago
      Does a computer understand how hot or cold it is outside? Does it understand that your offspring might be cranky because they’re hungry? Or not hungry, just tired? Can it divine the difference? Does a computer know if your boss is mad at you or if they had a fight with their spouse last night, or whatever other reason they may be grumpy? Can a computer establish relationships… with anything? How about when a computer goes through puberty? Or menopause? Or a car accident? How do those things affect them? Don’t bother responding, I think you get the point.
- emp17344 11mo ago
  Anthropic is pretty notorious for peddling hype. This is a marketing article - it has not undergone peer-review and should not be mistaken for scientific research.
deleted 11mo ago
[deleted]
stanfordkid 11mo ago
What do you mean by “understand”? Do you mean conscious? Understand just means “parse language” and is highly subjective. If I talk to someone African in Chinese they do not understand me but they are still conscious. If I talk to an LLM in Chinese it will understand me but that doesn’t mean it is conscious. If I talk about physics to a kindergartner they will not understand but that doesn’t mean they don’t understand anything. Do you see where I am going?
observationist 11mo ago
You don't understand how the tech works, then. LLMs aren't as good as humans at understanding, but it's not just statistics. The stochastic parrot meme is wrong. The networks create symbolic representations in training, with huge multidimensional correlations between patterns in the data, whether its temporal or semantic. The models "understand" concepts like emotions, text, physics, arbitrary social rules and phenomena, and anything else present in the data and context in the same fundamental way that humans do it. We're just better, with representations a few orders of magnitude higher resolution, much wider redundancy, and multi-million node parallelism with asynchronous operation that silicon can't quite match yet. In some cases, AI is superhuman, and uses better constructs than humans are capable of, in other cases, it uses hacks and shortcuts in representations, mimics where it falls short, and in some cases fails entirely, and has a suite of failure modes that aren't anywhere in the human taxonomy of operation. LLMs and AI aren't identical to human cognition, but there's a hell of a lot of overlap, and the stochastic parrot "ItS jUsT sTaTiStIcS!11!!" meme should be regarded as an embarrassing opinion to hold. "Thinking" models that cycle context and systems of problem solving also don't do it the same way humans think, but overlap in some of the important pieces of how we operate. We are many orders of magnitude beyond old ALICE bots and MEgaHAL markov chains - you'd need computers the size of solar systems to run a markov chain equivalent to the effective equivalent 40B LLM, let alone one of the frontier models, and those performance gains are objectively within the domain of "intelligence." We're pushing the theory and practice of AI and ML squarely into the domain of architectures and behaviors that qualify biological intelligence, and the state of the art models clearly demonstrate their capabilities accordingly. For any definition of understanding you care to lay down, there's significant overlap between the way human brains do it and the way LLMs do it. LLMs are specifically designed to model constructs from data, and to model the systems that produce the data they're trained on, and the data they model comes from humans and human processes.
- pwndByDeath 11mo ago
  You appear to be a proper alchemist, but you can't support an argument of understanding if there is no definition of understanding that isn't circular. If you want to believe the friendly voice really understands you, we have a word for that, faith. The skeptic sees the interactions with a chatbot as a statistical game that shows how uninteresting (e.g. predictable) humans and our stupid language are. There are useful gimmicks coming out like natural language processing, for low risk applications, but this form of AI pseudoscience isn't going to survive, but it will take some time for research to catch up to understanding how to describe the falsehoods of contemporary AI toys
deepGem 11mo ago
"I don't "understand" how LLMs "understand" anything." Why does the LLM need to understand anything. What today's chatbots have achieved is a software engineering feat. They have taken a stateless token generation machine that has compressed the entire internet's vocabulary to predict the next token and have 'hacked' a whole state management machinery around it. End result is a product that just feels like another human conversing with you and remembering your last birthday. Engineering will surely get better and while purists can argue that a new research perspective is needed, the current growth trajectory of chatbots, agents and code generation tools will carry the torch forward for years to come. If you ask me, this new AI winter will thaw in the atmosphere even before it settles on the ground.

john61 11mo ago

> The argument that computational complexity has something to do with this could have merit but the article certainly doesn’t give indication as to why. OP says it is because that predicting the next token can be correct or not, but it always looks plausible because that is what it calculates. Therefore it is dangerous and can not be fixed because it is how it works in essence.

dangus 11mo ago
I just want to point out a random anecdote. Literally yesterday ChatGPT hallucinated an entire feature of a mod for a video game I am playing including making up a fake console command. It just straight up doesn’t exist, it just seemed like a relatively plausible thing to exist. This is still happening. It never stopped happening. I don’t even see a real slowdown in how often it happens. It sometimes feels like the only thing saving LLMs are when they’re forced to tap into a better system like running a search engine query.
- bitwize 11mo ago
  [flagged]
  - bcrosby95 11mo ago
    Responding with "skill issue" in a discussion is itself a skill issue. Maybe invest in some conversational skills and learn to be constructive rather than parroting a useless meme.
  - dingnuts 11mo ago
    [dead]
  - dangus 11mo ago
    So you are saying the acceptable customer experience for these systems is that we need to explicitly tell them to accept defeat when they can’t find any training content/web search results that matches my query enough? Why don't they have any concept of having a percentage of confidence in their answer? It isn’t 2022 anymore, this is supposed to be a mature product. Why am I even using this thing rather than using the game’s own mod database search tool? Or the wiki documentation? What value is this system adding for me if I’m supposed to be a prompt engineer?

andy99 11mo ago

I agree with everything you wrote, the technology is unbelievable and 6 years ago, maybe even 3.1 years would have been considered magic. A steel man argument for why winter might be coming is all the dumb stuff companies are pushing AI for. On one hand (and I believe this) we argue it’s the most consequential technology in generations. On the other, everybody is using it for nonsense like helping you write an email that makes you sound like an empty suit, or providing a summary you didn’t ask for. There’s still a ton of product work to cross whatever that valleys called between concept and product, and if that doesn’t happen, money is going to start disappearing. The valuation isn’t justified by the dumb stuff we do with it, it needs PMF.

baq 11mo ago
> maybe even 3.1 years would have been considered magic I vividly remember sitting there for literally hours talking to a computer on launch day, it was a very short night. This feeling of living in the future has not left me since, it's got quieter, but it's still there. It's still magic after those three years, perhaps even more so. It wasn't supposed to work this well for decades! and yet.
- bloppe 11mo ago
  I think you're missing the point of "AI winter". It's not about how good the products are now. It's about how quickly the products are improving and creating the potential for profit. That's what drives investment. 3 things we know about the AI revolution in 2025: - LLMs are amazing, but they have reached a plateau. AGI is not within reach. - LLM investment has sacrificed many hundreds of billions of dollars, much of it from the world's pension funds. - There is no credible path to a high-margin LLM product. Margins will be razor-thin positive at best once the free trial of the century starts to run out of steam. This all adds up to a rather nasty crunch. The thing about winter, though, is that it's eventually followed by another summer.
  - saltcured 11mo ago
    Many technologies plateau, but we don't say they all have winters. Terrestrial radio winter? Television winter? Automobile winter? Air travel winter? Nuclear power comes close in terms of its tarnished image and reluctance to reinvest. I personally believe contemporary AI is over-hyped, but I cannot say with confidence that it is going to lead to a similar winter as the last time. It seems like today's products satisfy enough users to remain as a significant area, even if it doesn't greatly expand... The only way I could see it fizzling as a product category is if it turns out it is not economically feasible to operate a sustainable service. Will users pay a fair price to keep the LLM datacenters running, without speculative investment subsidies? The other aspect of the winter is government investment, rather than commercial. What could the next cycle of academic AI research look like? E.g. exploration that needs to happen in grant-funded university labs instead of venture-funded companies? The federal funding picture seems unclear, but that's true across the board right now for reasons that have nothing to do with AI per se.

cess11 11mo ago

GOFAI was also a paradigm shift, regardless of that winter. For example, banks started automating assessments of creditworthiness. What we didn't get was what had been expected, namely things like expert systems that were actual experts, so called 'general intelligence' and war waged through 'blackboard systems'. We've had voice controlled electronics for a long time. On the other hand, machine vision applications have improved massively in certain niches, and also allowed for new forms of intense tyranny and surveillance where errors are actually considered a feature rather than a bug since they erode civil liberties and human rights but are still broadly accepted because 'computer says'. While you could likely argue "leaps and bounds with novel methods utilizing AI for chemistry, computational geometry, biology etc." by downplaying the first part or clarifying that it is mainly an expectation, I think most people are going to, for the foreseeable future, keep seeing "AI" as more or less synonymous with synthetic infantile chatbot personalities that substitute for human contact.

bloppe 11mo ago

I don't think anybody expects AI development to stop. A winter is defined by a relative drying-up of investment and, importantly, it's almost certain that any winter will eventually be followed by another summer. The pace of investment in the last 2 years has been so insane that even Altman has claimed that it's a bubble.

fragmede 11mo ago

> I could see many arguments about why modern research will fail to create AGI Why is AGI even necessary? If the loop between teaching the AI something, and it being able to repeat similar enough tasks; if that loop becomes short enough, days or hours instead of months, who cares if some ill-defined bar of AGI is met?

lamontcg 11mo ago

> that enables a computer to pretty effectively understand natural language I'd argue that it pretty effectively mimics natural language. I don't think it really understands anything, it is just the best madlibs generator that the world has ever seen. For many tasks, this is accurate 99+% of the time, and the failure cases may not matter. Most humans don't perform any better, and arguably regurgitate words without understanding as well. But if the failure cases matter, then there is no actual understanding and the language the model is generating isn't ever getting "marked to market/reality" because there's no mental world model to check against. That isn't going to be usable if there are real-world consequences of the LLM getting things wrong, and they can wind up making very basic mistakes that humans wouldn't make--because we can innately understand how the world works and aren't always just stringing words together that sound good.

code_biologist 11mo ago
Break the aspects of language understanding and language generation apart. While I would agree that generative LLMs are understanding-free madlibs for writing text, embedding vector spaces and LLM latent spaces seem are a pretty genuine understanding of natural language. High dimensional vector spaces seem like the best machine representation we currently have for meaning and LLMs are using it effectively.

Yoric 11mo ago

Technologically, I believe that you're right. On the other hands, the previous AI winters happened despite novel, useful technologies, some of which proved extremely useful and actually changed the world of software. They happened because of overhype, then investor moving on to the next opportunity. Here, the investors are investing in LLMs. Not in AlphaFold, AlphaGo, neurosymbolic, focus learning, etc. If (when) LLMs prove insufficient to the insane level of hype and if (when) experience shows that there is only so much money that you can make with LLMs, it's possible that the money will move on to other types of AI, but there are chances that it will actually go to something entirely different, perhaps quantum, leaving AI in winter.

encyclopedism 11mo ago

> The fact is: something fundamental has changed that enables a computer to pretty effectively understand natural language. I have commented elsewhere but this bears repeating LLM's do not think, understand, reason, reflect, comprehend and they never shall. If you had enough paper and ink and the patience to go through it, you could take all the training data and manually step through and train the same model. Then once you have trained the model you could use even more pen and paper to step through the correct prompts to arrive at the answer. All of this would be a completely mechanical process. This really does bear thinking about. It's amazing the results that LLM's are able to acheive. But let's not kid ourselves and start throwing about terms like AGI or emergence just yet. It makes a mechanical process seem magical (as do computers in general). I should add it also makes sense as to why it would, just look at the volume of human knowledge (the training data). It's the training data with the mass quite literally of mankind's knowledge, genius, logic, inferences, language and intellect that does the heavy lifting.

lambdaone 11mo ago
You could use a similar argument to argue that "brains do not think, understand, reason, reflect, comprehend and they never shall." After all, there's nothing in there but neurons, synapses and other biological gunk. If you look at it that way. Which you shouldn't.
- encyclopedism 11mo ago
  That argument you posed does not follow. Brains do all of those things, at least I know mine does because those are the most intimate experiences I have. More intimate than even my sense experiences. It's what lead Kant to exclaim 'cogito ergo sum' and Ibn Sina before him to also note that fact. Moreover how the brain does what it does an open academic question. One of the most difficult, please see the Hard problem of consciousness for example. I can make certain determined judgments about things, my digital thermometer does not think or understand when it tells me the temperature, something I myself would be unable to determine. My digital LLM does not think for the same reason. Importantly my paper and pen version of that very same LLM would also not think.

ath3nd 11mo ago

[dead]