8 ms·

I don't understand this view. How I see it the fundamental bottleneck to AGI is continual learning and backpropagation. Models today are static, and human brain

by andy12_ 7mo ago

I don't understand this view. How I see it the fundamental bottleneck to AGI is continual learning and backpropagation. Models today are static, and human brains don't learn or adapt themselves with anything close to backpropagation. World models don't solve any of these problems; they are fundamentally the same kind of deep learning architectures we are used to work with. Heck, if you think learning from the world itself is the bottleneck, you can just put a vision-action LLM on a reinforcement learning loop in a robotic/simulated body.

zelphirkalt 7mo ago
> I don't understand this view. How I see it the fundamental bottleneck to AGI is continual learning and backpropagation. Models today are static, and human brains don't learn or adapt themselves with anything close to backpropagation. Even with continuous backpropagation and "learning", enriching the training data, so called online-learning, the limitations will not disappear. The LLMs will not be able to conclude things about the world based on fact and deduction. They only consider what is likely from their training data. They will not foresee/anticipate events, that are unlikely or non-existent in their training data, but are bound to happen due to real world circumstances. They are not intelligent in that way. Whether humans always apply that much effort to conclude these things is another question. The point is, that humans fundamentally are capable of doing that, while LLMs are structurally not. The problems are structural/architectural. I think it will take another 2-3 major leaps in architectures, before these AI models reach human level general intelligence, if they ever reach it. So far they can "merely" often "fake it" when things are statistically common in their training data.
- andy12_ 7mo ago
  > Even with continuous backpropagation and "learning" That's what I said. Backpropagation cannot be enough; that's not how neurons work in the slightest. When you put biological neurons in a Pong environment they learn to play not through some kind of loss or reward function; they self-organize to avoid unpredictable stimulation. As far as I know, no architecture learns in such an unsupervised way. https://www.sciencedirect.com/science/article/pii/S0896627322008066 https://www.sciencedirect.com/science/article/pii/S089662732...
  - torginus 7mo ago
    Forgive me for being ignorant - but 'loss' in supervised learning ML context encode the difference between how unlikely (high loss) or likely (low loss) was the network in predicting the output based on the input. This sounds very similar to me as to what neurons do (avoid unpredictable stimulation)
    - andy12_ 7mo ago
      So, I have been thinking about this for a little while. Image a model f that takes a world x and makes a prediciton y. At a high-level, a traditional supervised model is trained like this f(x)=y' => loss(y',y) => how good was my prediction? Train f through backprop with that error. While a model trained with reinforcement learning is more similar to this. Where m(y) is the resulting world state of taking an action y the model predicted. f(x)=y' => m(y')=z => reward(z) => how good was the state I was in based on my actions? Train f with an algorithm like REINFORCE with the reward, as the world m is a non-differentiable black-box. While a group of neurons is more like predicting what is the resulting word state of taking my action, g(x,y), and trying to learn by both tuning g and the action taken f(x). f(x)=y' => m(y')=z => g(x,y)=z' => loss(z,z') => how predictable was the results of my actions? Train g normally with backprop, and train f with an algorithm like REINFORCE with negative surprise as a reward. After talking with GPT5.2 for a little while, it seems like Curiosity-driven Exploration by Self-supervised Prediction[1] might be an architecture similar to the one I described for neurons? But with the twist that f is rewarded by making the prediction error bigger (not smaller!) as a proxy of "curiosity". [1] https://arxiv.org/pdf/1705.05363 https://arxiv.org/pdf/1705.05363
  - latentsea 7mo ago
    So can't you just use how real neurons learn as training data to to learn how to learn the same way?
- wiz21c 7mo ago
  I'm sure that if a car appeared from nowhere in the middle of your living room, you would not be prepared at all. So my question is: when is there enough training data that you can handle 99.99% of the world ?
- jstummbillig 7mo ago
  > They will not foresee/anticipate events, that are unlikely or non-existent in their training data, but are bound to happen due to real world circumstances. They are not intelligent in that way. Can you be a bit more specific at all bounds? Maybe via an example?
- steego 7mo ago
  I think people MOSTLY foresee and anticipate events in OUR training data, which mostly comprises information collected by our senses. Our training data is a lot more diverse than an LLMs. We also leverage our senses as a carrier for communicating abstract ideas using audio and visual channels that may or may not be grounded in reality. We have TV shows, video games, programming languages and all sorts of rich and interesting things we can engage with that do not reflect our fundamental reality. Like LLMs, we can hallucinate while we sleep or we can delude ourselves with untethered ideas, but UNLIKE LLMs, we can steer our own learning corpus. We can train ourselves with our own untethered “hallucinations” or we can render them in art and share them with others so they can include it in their training corpus. Our hallucinations are often just erroneous models of the world. When we render it into something that has aesthetic appeal, we might call it art. If the hallucination helps us understand some aspect of something, we call it a conjecture or hypothesis. We live in a rich world filled with rich training data. We don’t magically anticipate events not in our training data, but we’re also not void of creativity (“hallucinations”) either. Most of us are stochastic parrots most of the time. We’ve only gotten this far because there are so many of us and we’ve been on this earth for many generations. Most of us are dazzled and instinctively driven to mimic the ideas that a small minority of people “hallucinate”. There is no shame in mimicking or being a stochastic parrot. These are critical features that helped our ancestors survive.
  - robwwilliams 7mo ago
    > We can steer our own learning corpus This is critical. We have some degree of attentional autonomy. And we have a complex tapestry of algorithms running in thalamocortical circuits that generate “Nows”. Truncation commands produce sequences of acts (token-like products).
- perfmode 7mo ago
  Humans are notoriously bad at formal logic. The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. That looks a lot more like pattern matching than rule application. Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. The slow, deliberate, step-by-step stuff is effortful and error-prone, and people avoid it when they can. And even when they do engage it, they’re often confabulating a logical-sounding justification for a conclusion they already reached by other means. So maybe the honest answer is: the gap between what LLMs do and what most humans do most of the time might be smaller than people assume. The story that humans have access to some pure deductive engine and LLMs are just faking it with statistics might be flattering to humans more than it’s accurate. Where I’d still flag a possible difference is something like adaptability. A person can learn a totally new formal system and start applying its rules, even if clumsily. Whether LLMs can genuinely do that outside their training distribution or just interpolate convincingly is still an open question. But then again, how often do humans actually reason outside their own “training distribution”? Most human insight happens within well-practiced domains.
  - lich_king 7mo ago
    > The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. I've never heard about the Wason selection task, looked it up, and could tell the right answer right away. But I can also tell you why: because I have some familiarity with formal logic and can, in your words, pattern-match the gotcha that "if x then y" is distinct from "if not x then not y". In contrast to you, this doesn't make me believe that people are bad at logic or don't really think. It tells me that people are unfamiliar with "gotcha" formalities introduced by logicians that don't match the everyday use of language. If you added a simple additional to the problem, such as "Note that in this context, 'if' only means that...", most people would almost certainly answer it correctly. Mind you, I'm not arguing that human thinking is necessarily more profound from what what LLMs could ever do. However, judging from the output, LLMs have a tenuous grasp on reality, so I don't think that reductionist arguments along the lines of "humans are just as dumb" are fair. There's a difference that we don't really know how to overcome.
- conartist6 7mo ago
  Models don't care. They aren't alive. This is the source of the chasm between here and AGI. You have to fear death to reason about the world and how to behave in it. I guess I just always thought it was obvious that you can't do better than nature. You can do different things, sure, but if a society of unique individuals wasn't the most effective way of making progress, nature itself would not have chosen it. So in a way I think Yan is smart because he got money, but in a way I think he's a fucking idiot if he can't see just how very, very very far we are from competing with organic intelligence.
  - conartist6 7mo ago
    Not only that but people like this aren't actually interested in understanding the physical world. Because we don't understand it yet. If you care about understanding the world I think you become someone more like Jane Goodall than Yan LeCun
  - j4k0o 7mo ago
    "You have to fear death to reason about the world and how to behave in it." You're onto something there. If everyone knew they were to die tomorrow, all of a sudden they'd choose to act differently. There is no logical thought process that determines that - it's something else. Something we can't concretely point toward as an object.
energy123 7mo ago
I don't understand why online learning is that necessary. If you took Einstein at 40 and surgically removed his hippocampus so he can't learn anything he didn't already know (meaning no online learning), that's still a very useful AGI. A hippocampus is a nice upgrade to that, but not super obviously on the critical path.
- andy12_ 7mo ago
  That's true. Though could that hippocampus-less Einstein be able to keep making novel complex discoveries from that point forward? Seems difficult. He would rapidly reach the limits of his short term memory (the same way current models rapidly reach the limits of their context windows).
- zelphirkalt 7mo ago
  I guess the sheer amount and also variety of information you would need to pre-encode to get an Einstein at 40 is huge. Every day stream of high resolution video feed and actions and consequences and thoughts and ideas he has had until the age of 40 of every single moment. That includes social interactions, like a conversation and mimic of the other person in combination with what was said and background knowledge about the other person. Even a single conversation's data is a huge amount of data. But one might say that the brain is not lossless ... True, good point. But in what way is it lossy? Can that be simulated well enough to learn an Einstein? What gives events significance is very subjective.
- andsoitis 7mo ago
  Where does that training data come from?
- staticman2 7mo ago
  > If you took Einstein at 40 and surgically removed his hippocampus so he can't learn anything he didn't already know (meaning no online learning), that's still a very useful AGI. I like how people are accepting this dubious assertion that Einstein would be "useful" if you surgically removed his hippocampus and engaging with this. It also calls this Einstein an AGI rather than a disabled human???
A_D_E_P_T 7mo ago
You could have continual learning on text and still be stuck in the same "remixing baseline human communications" trap. It's a nasty one, very hard to avoid, possibly even structurally unavoidable. As for the "just put a vision LLM in a robot body" suggestion: People are trying this (e.g. Physical Intelligence) and it looks like it's extraordinarily hard! The results so far suggest that bolting perception and embodiment onto a language-model core doesn't produce any kind of causal understanding. The architecture behind the integration of sensory streams, persistent object representations, and modeling time and causality is critically important... and that's where world models come in.
ben_w 7mo ago
> Models today are static, and human brains don't learn or adapt themselves with anything close to backpropagation. While I suspect latter is a real problem (because all mammal brains* are much more example-efficient than all ML), the former is more about productisation than a fundamental thing: the models can be continuously updated already, but that makes it hard to deal with regressions. You kinda want an artefact with a version stamp that doesn't change itself before you release the update, especially as this isn't like normal software where specific features can be toggled on or off in isolation of everything else. * I think. Also, I'm saying "mammal" because of an absence of evidence (to my *totally amateur* skill level) not evidence of absence.
- program_whiz 7mo ago
  they can be continuously updated, assuming you re-run representative samples of the training set through them continuously. Unlike a mammal brain which preserves the function of neurons unless they activate in a situation which causes a training signal, deep nets have catastrophic forgetting because signals get scattered everywhere. If you had a model continuously learning about you in your pocket, without tons of cycles spent "remembering" old examples. In fact, this is a major stumbling block in standard training, sampling is a huge problem. If you just iterate through the training corpus, you'll have forgotten most of the english stuff by the time you finish with chinese or spanish. You have to constantly mix and balance training info due to this limitation. The fundamental difference is that physical neurons have a discrete on/off activation, while digital "neurons" in a network are merely continuous differentiable operations. They also don't have a notion of "spike timining dependency" to avoid overwriting activations that weren't related to an outcome. There are things like reward-decay over time, but this applies to the signal at a very coarse level, updates are still scattered to almost the entire system with every training example.
10xDev 7mo ago
The fact that models aren't continually updating seems more like a feature. I want to know the model is exactly the same as it was the last time I used it. Any new information it needs can be stored in its context window or stored in a file to read the next it needs to access it.
- kergonath 7mo ago
  > The fact that models aren't continually updating seems more like a feature. I think this is true to some extent: we like our tools to be predictable. But we’ve already made one jump by going from deterministic programs to stochastic models. I am sure the moment a self-evolutive AI shows up that clears the "useful enough" threshold we’ll make that jump as well.
  - 10xDev 7mo ago
    Stochastic and unpredictability aren't exactly the same. I would claim current LLMs are generally predictable even if it is not as predictable as a deterministic program.
    - kergonath 7mo ago
      No, but my point is that to some extent we value determinism. By making the jump to stochastic models we already move away from the status quo; further jumps are entirely possible. Depending on use case we can accept more uncertainty if it comes with benefits. I also don’t think there is a reason to believe that self-learning models must be unpredictable.
- jnd-cz 7mo ago
  Unless you use your oen local models then you don't even know when OpenAI or Anthropic tweaked the model less or more. One week it's a version x, next week it's a version y. Just like your operating system is continuously evolving with smaller patches of specific apps to whole new kernel version and new OS release.
  - 7mo ago
charcircuit 7mo ago
Agents have the ability of continual learning.
- andy12_ 7mo ago
  Putting stuff you have learned into a markdown file is a very "shallow" version of continual learning. It can remember facts, yes, but I doubt a model can master new out-of-distribution tasks this way. If anything, I think that Google's Titans[1] and Hope[2] architectures are more aligned with true continual learning (without being actual continual learning still, which is why they call it "test-time memorization"). [1] https://arxiv.org/pdf/2501.00663 https://arxiv.org/pdf/2501.00663 [2] https://arxiv.org/pdf/2512.24695 https://arxiv.org/pdf/2512.24695
  - charcircuit 7mo ago
    I have had it master tasks by doing this. The first time it tries to solve an issue it may take a long time, but it documents its findings and how it was able to do it and then it applies that knowledge the next time the task comes up.
    - andy12_ 7mo ago
      There is some things that just don't transfer really well without specific training. I tried to create diagrams in Typst with Cetz (a Processing and Tikz inspired graphing library), and even with documentation, GPT 5.2-thinking can't really do complex nice diagrams like it can in Tikz. It can do simple things that are similar to the shown examples, but nothing really interesting. Typst and specially Cetz is too new for any current model to really "get it", so they can't use it. I need to wait to the next batch of frontier models so that they learn Typst and Cetz examples during pre-training.
  - patapong 7mo ago
    It really reminds me of the movie Memento - it has to constantly put notes down to remember who it is and what it should do after waking up without memory every n minutes.
nurettin 7mo ago
Who knows? Perhaps attention really is all you need. Maybe our context window is really large. Or our compression is really effective. Perhaps adding external factors might be able to indirectly teach the models to act more in line with social expectations such as being embarrassed to repeat the same mistake, unlocking the final piece of the puzzle. We are still stumbling in the dark for answers.
jacquesm 7mo ago
The main difference is humans are learning all the time and models learn batch wise and forget whatever happened in a previous session unless someone makes it part of the training data so there is a massive lag. Whoever cracks the continuous customized (per user, for instance) learning problem without just extending the context window is going to be making a big splash. And I don't mean cheats and shortcuts, I mean actually tuning the model based on received feedback.
- aurareturn 7mo ago
  Why not just provide more compute for say, 1 billion token context for each user to mimic continuous learning. Then retrain the model in the background to include learnings. The user wouldn’t know if the continuous learning came from the context or the model retrained. It wouldn’t matter. Continuous learning seems to be a compute and engineering problem.
  - jacquesm 7mo ago
    Because that re-training is not strong enough to hold, or so it seems. The same dumb factual errors keep coming up on different generations of the same models. I've yet to see proof that something 'stuck' from model to model. They get better in a general sense but not in the specific sense that what was corrected stays put, not from session to session and not from one generation to the next. My solution is to have this massive 'boot up' prompt but it becomes extremely tedious to maintain.
- eloisant 7mo ago
  They can write to files then refer to them in a next session. A bit like the main character played by Guy Pierce in the movie Memento (which doesn't work great for him to be honest).
edgyquant 7mo ago
Iirc LeCunn talks about a self organizing hierarchy of real world objects and imo this is exactly how the human brain actually learns
stanfordkid 7mo ago
It's pretty simple... the word circle and what you can correlate to it via english language description has somewhat less to do with reality than a physical 3D model of a circle and what it would do in an environment. You can't just add more linguistic description via training data to change that. It doesn't really matter that you can keep back propagating because what you are back propagating over is fundamentally and qualitatively less rich.
mxkopy 7mo ago
The reason LLMs fail today is because there’s no meaning inherent to the tokens they produce other than the one captured by cooccurrence within text. Efforts like these are necessary because so much of “general intelligence” is convention defined by embodied human experience, for example arrows implying directionality and even directionality itself.
anon7000 7mo ago
I don’t understand your view. Reality is that we need some way to encode the rules of the world in a more definitive way. If we want models to be able to make assertive claims about important information and be correct, it’s very fair to theorize they might need a more deterministic approach than just training them more. But it’s just a theory that this will actually solve the problem. Ultimately, we still have a lot to learn and a lot of experiments to do. It’s frankly unscientific to suggest any approaches are off the table, unless the data & research truly proves that. Why shouldn’t we take this awesome LLM technology and bring in more techniques to make it better? A really, really basic example is chess. Current top AI models still don’t know how to play it (https://www.software7.com/blog/ai_chess_vs_1983_atari/ https://www.software7.com/blog/ai_chess_vs_1983_atari/) The models are surely trained on source material that include chess rules, and even high level chess games. But the models are not learning how to play chess correctly. They don’t have a model to understand how chess actually works — they only have a non-deterministic prediction based on what they’ve seen, even after being trained on more data than any chess novice has ever seen about the topic. And this is probably one of the easiest things for AI to stimulate. Very clear/brief rules, small problem space, no hidden information, but it can’t handle the massive decision space because its prediction isn’t based on the actual rules, but just “things that look similar” (And yeah, I’m sure someone could build a specific LLM or agent system that can handle chess, but the point is that the powerful general purpose models can’t do it out of the box after training.) Maybe more training & self-learning can solve this, but it’s clearly still unsolved. So we should definitely be experimenting with more techniques.
- andy12_ 7mo ago
  > Reality is that we need some way to encode the rules of the world in a more definitive way I mean, sure. But do world models the way LeCun proposes them solves this? I don't think so. JEPAs are just an unsupervised machine learning model at the end of the day; they might end up being better that just autoregressive pretraining on text+images+video, but they are not magic. For example, if you train a JEPA model on data of orbital mechanics, will it learn actually sensible algorithms to predict the planets' motions or will it just learn a mix of heuristic?
slashdave 7mo ago
If your model is poor, no amount of learning can fix it. If you don't think your model architecture is limited, you aren't looking hard enough.
the_black_hand 7mo ago
yes those are bottlenecks that world models don't solve. but the promise of world models is, unlike LLMs, they might be able to learn things about the world that humans haven't written. For example, we still don't fully know how insects fly. A world model could be trained on thousands of videos of insects and make a novel observation about insect trajectories. The premise is that despite being here for millenia, humans have only observed a tiny fraction of the world. So I do buy his idea. But I disagree that you need world models to get to human level capabilities. IMO there's no fundamental reason why models can't develop human understanding based on the known human observations.
eloisant 7mo ago
LeCun is a researcher. From his point of view, there are not much research left on LLM. Sure we can still improve them a bit with engineering around, but he's more interested in basic research.
a1371 7mo ago
I never understood why we believe humans don't backprop. Isn't it that during the day we fill up our context (short term memory) and sleep is actually where we use that to backprop? Heck, everyone knows what "sleep on it" means.
- cedilla 7mo ago
  Brains are not doing linear algebra, and they don't follow a concise algorithm. What LLM do is even farther away from what neural nets do, and even there - artificial neurons are inspired by but not reimplementing biological neurons. You can understand human thought in terms of LLMs, but that is just a simile, like understanding physical reality in terms of computers or clockworks.
carlmr 7mo ago
Especially they will require even more compute to get anything close to usable output. Human brains are super efficient at learning and producing output. We will need exponentially more compute for real time learning from video + audio + haptic data.

by andy12_ 7mo ago

zelphirkalt 7mo ago

> I don't understand this view. How I see it the fundamental bottleneck to AGI is continual learning and backpropagation. Models today are static, and human brains don't learn or adapt themselves with anything close to backpropagation. Even with continuous backpropagation and "learning", enriching the training data, so called online-learning, the limitations will not disappear. The LLMs will not be able to conclude things about the world based on fact and deduction. They only consider what is likely from their training data. They will not foresee/anticipate events, that are unlikely or non-existent in their training data, but are bound to happen due to real world circumstances. They are not intelligent in that way. Whether humans always apply that much effort to conclude these things is another question. The point is, that humans fundamentally are capable of doing that, while LLMs are structurally not. The problems are structural/architectural. I think it will take another 2-3 major leaps in architectures, before these AI models reach human level general intelligence, if they ever reach it. So far they can "merely" often "fake it" when things are statistically common in their training data.

andy12_ 7mo ago
> Even with continuous backpropagation and "learning" That's what I said. Backpropagation cannot be enough; that's not how neurons work in the slightest. When you put biological neurons in a Pong environment they learn to play not through some kind of loss or reward function; they self-organize to avoid unpredictable stimulation. As far as I know, no architecture learns in such an unsupervised way. https://www.sciencedirect.com/science/article/pii/S0896627322008066 https://www.sciencedirect.com/science/article/pii/S089662732...
- torginus 7mo ago
  Forgive me for being ignorant - but 'loss' in supervised learning ML context encode the difference between how unlikely (high loss) or likely (low loss) was the network in predicting the output based on the input. This sounds very similar to me as to what neurons do (avoid unpredictable stimulation)
  - andy12_ 7mo ago
    So, I have been thinking about this for a little while. Image a model f that takes a world x and makes a prediciton y. At a high-level, a traditional supervised model is trained like this f(x)=y' => loss(y',y) => how good was my prediction? Train f through backprop with that error. While a model trained with reinforcement learning is more similar to this. Where m(y) is the resulting world state of taking an action y the model predicted. f(x)=y' => m(y')=z => reward(z) => how good was the state I was in based on my actions? Train f with an algorithm like REINFORCE with the reward, as the world m is a non-differentiable black-box. While a group of neurons is more like predicting what is the resulting word state of taking my action, g(x,y), and trying to learn by both tuning g and the action taken f(x). f(x)=y' => m(y')=z => g(x,y)=z' => loss(z,z') => how predictable was the results of my actions? Train g normally with backprop, and train f with an algorithm like REINFORCE with negative surprise as a reward. After talking with GPT5.2 for a little while, it seems like Curiosity-driven Exploration by Self-supervised Prediction[1] might be an architecture similar to the one I described for neurons? But with the twist that f is rewarded by making the prediction error bigger (not smaller!) as a proxy of "curiosity". [1] https://arxiv.org/pdf/1705.05363 https://arxiv.org/pdf/1705.05363
- latentsea 7mo ago
  So can't you just use how real neurons learn as training data to to learn how to learn the same way?
wiz21c 7mo ago
I'm sure that if a car appeared from nowhere in the middle of your living room, you would not be prepared at all. So my question is: when is there enough training data that you can handle 99.99% of the world ?
jstummbillig 7mo ago
> They will not foresee/anticipate events, that are unlikely or non-existent in their training data, but are bound to happen due to real world circumstances. They are not intelligent in that way. Can you be a bit more specific at all bounds? Maybe via an example?
steego 7mo ago
I think people MOSTLY foresee and anticipate events in OUR training data, which mostly comprises information collected by our senses. Our training data is a lot more diverse than an LLMs. We also leverage our senses as a carrier for communicating abstract ideas using audio and visual channels that may or may not be grounded in reality. We have TV shows, video games, programming languages and all sorts of rich and interesting things we can engage with that do not reflect our fundamental reality. Like LLMs, we can hallucinate while we sleep or we can delude ourselves with untethered ideas, but UNLIKE LLMs, we can steer our own learning corpus. We can train ourselves with our own untethered “hallucinations” or we can render them in art and share them with others so they can include it in their training corpus. Our hallucinations are often just erroneous models of the world. When we render it into something that has aesthetic appeal, we might call it art. If the hallucination helps us understand some aspect of something, we call it a conjecture or hypothesis. We live in a rich world filled with rich training data. We don’t magically anticipate events not in our training data, but we’re also not void of creativity (“hallucinations”) either. Most of us are stochastic parrots most of the time. We’ve only gotten this far because there are so many of us and we’ve been on this earth for many generations. Most of us are dazzled and instinctively driven to mimic the ideas that a small minority of people “hallucinate”. There is no shame in mimicking or being a stochastic parrot. These are critical features that helped our ancestors survive.
- robwwilliams 7mo ago
  > We can steer our own learning corpus This is critical. We have some degree of attentional autonomy. And we have a complex tapestry of algorithms running in thalamocortical circuits that generate “Nows”. Truncation commands produce sequences of acts (token-like products).
perfmode 7mo ago
Humans are notoriously bad at formal logic. The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. That looks a lot more like pattern matching than rule application. Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. The slow, deliberate, step-by-step stuff is effortful and error-prone, and people avoid it when they can. And even when they do engage it, they’re often confabulating a logical-sounding justification for a conclusion they already reached by other means. So maybe the honest answer is: the gap between what LLMs do and what most humans do most of the time might be smaller than people assume. The story that humans have access to some pure deductive engine and LLMs are just faking it with statistics might be flattering to humans more than it’s accurate. Where I’d still flag a possible difference is something like adaptability. A person can learn a totally new formal system and start applying its rules, even if clumsily. Whether LLMs can genuinely do that outside their training distribution or just interpolate convincingly is still an open question. But then again, how often do humans actually reason outside their own “training distribution”? Most human insight happens within well-practiced domains.
- lich_king 7mo ago
  > The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. I've never heard about the Wason selection task, looked it up, and could tell the right answer right away. But I can also tell you why: because I have some familiarity with formal logic and can, in your words, pattern-match the gotcha that "if x then y" is distinct from "if not x then not y". In contrast to you, this doesn't make me believe that people are bad at logic or don't really think. It tells me that people are unfamiliar with "gotcha" formalities introduced by logicians that don't match the everyday use of language. If you added a simple additional to the problem, such as "Note that in this context, 'if' only means that...", most people would almost certainly answer it correctly. Mind you, I'm not arguing that human thinking is necessarily more profound from what what LLMs could ever do. However, judging from the output, LLMs have a tenuous grasp on reality, so I don't think that reductionist arguments along the lines of "humans are just as dumb" are fair. There's a difference that we don't really know how to overcome.
conartist6 7mo ago
Models don't care. They aren't alive. This is the source of the chasm between here and AGI. You have to fear death to reason about the world and how to behave in it. I guess I just always thought it was obvious that you can't do better than nature. You can do different things, sure, but if a society of unique individuals wasn't the most effective way of making progress, nature itself would not have chosen it. So in a way I think Yan is smart because he got money, but in a way I think he's a fucking idiot if he can't see just how very, very very far we are from competing with organic intelligence.
- conartist6 7mo ago
  Not only that but people like this aren't actually interested in understanding the physical world. Because we don't understand it yet. If you care about understanding the world I think you become someone more like Jane Goodall than Yan LeCun
- j4k0o 7mo ago
  "You have to fear death to reason about the world and how to behave in it." You're onto something there. If everyone knew they were to die tomorrow, all of a sudden they'd choose to act differently. There is no logical thought process that determines that - it's something else. Something we can't concretely point toward as an object.

energy123 7mo ago

I don't understand why online learning is that necessary. If you took Einstein at 40 and surgically removed his hippocampus so he can't learn anything he didn't already know (meaning no online learning), that's still a very useful AGI. A hippocampus is a nice upgrade to that, but not super obviously on the critical path.

andy12_ 7mo ago
That's true. Though could that hippocampus-less Einstein be able to keep making novel complex discoveries from that point forward? Seems difficult. He would rapidly reach the limits of his short term memory (the same way current models rapidly reach the limits of their context windows).
zelphirkalt 7mo ago
I guess the sheer amount and also variety of information you would need to pre-encode to get an Einstein at 40 is huge. Every day stream of high resolution video feed and actions and consequences and thoughts and ideas he has had until the age of 40 of every single moment. That includes social interactions, like a conversation and mimic of the other person in combination with what was said and background knowledge about the other person. Even a single conversation's data is a huge amount of data. But one might say that the brain is not lossless ... True, good point. But in what way is it lossy? Can that be simulated well enough to learn an Einstein? What gives events significance is very subjective.
andsoitis 7mo ago
Where does that training data come from?
staticman2 7mo ago
> If you took Einstein at 40 and surgically removed his hippocampus so he can't learn anything he didn't already know (meaning no online learning), that's still a very useful AGI. I like how people are accepting this dubious assertion that Einstein would be "useful" if you surgically removed his hippocampus and engaging with this. It also calls this Einstein an AGI rather than a disabled human???

A_D_E_P_T 7mo ago

You could have continual learning on text and still be stuck in the same "remixing baseline human communications" trap. It's a nasty one, very hard to avoid, possibly even structurally unavoidable. As for the "just put a vision LLM in a robot body" suggestion: People are trying this (e.g. Physical Intelligence) and it looks like it's extraordinarily hard! The results so far suggest that bolting perception and embodiment onto a language-model core doesn't produce any kind of causal understanding. The architecture behind the integration of sensory streams, persistent object representations, and modeling time and causality is critically important... and that's where world models come in.

ben_w 7mo ago

> Models today are static, and human brains don't learn or adapt themselves with anything close to backpropagation. While I suspect latter is a real problem (because all mammal brains* are much more example-efficient than all ML), the former is more about productisation than a fundamental thing: the models can be continuously updated already, but that makes it hard to deal with regressions. You kinda want an artefact with a version stamp that doesn't change itself before you release the update, especially as this isn't like normal software where specific features can be toggled on or off in isolation of everything else. * I think. Also, I'm saying "mammal" because of an absence of evidence (to my *totally amateur* skill level) not evidence of absence.

program_whiz 7mo ago
they can be continuously updated, assuming you re-run representative samples of the training set through them continuously. Unlike a mammal brain which preserves the function of neurons unless they activate in a situation which causes a training signal, deep nets have catastrophic forgetting because signals get scattered everywhere. If you had a model continuously learning about you in your pocket, without tons of cycles spent "remembering" old examples. In fact, this is a major stumbling block in standard training, sampling is a huge problem. If you just iterate through the training corpus, you'll have forgotten most of the english stuff by the time you finish with chinese or spanish. You have to constantly mix and balance training info due to this limitation. The fundamental difference is that physical neurons have a discrete on/off activation, while digital "neurons" in a network are merely continuous differentiable operations. They also don't have a notion of "spike timining dependency" to avoid overwriting activations that weren't related to an outcome. There are things like reward-decay over time, but this applies to the signal at a very coarse level, updates are still scattered to almost the entire system with every training example.

10xDev 7mo ago

The fact that models aren't continually updating seems more like a feature. I want to know the model is exactly the same as it was the last time I used it. Any new information it needs can be stored in its context window or stored in a file to read the next it needs to access it.

kergonath 7mo ago
> The fact that models aren't continually updating seems more like a feature. I think this is true to some extent: we like our tools to be predictable. But we’ve already made one jump by going from deterministic programs to stochastic models. I am sure the moment a self-evolutive AI shows up that clears the "useful enough" threshold we’ll make that jump as well.
- 10xDev 7mo ago
  Stochastic and unpredictability aren't exactly the same. I would claim current LLMs are generally predictable even if it is not as predictable as a deterministic program.
  - kergonath 7mo ago
    No, but my point is that to some extent we value determinism. By making the jump to stochastic models we already move away from the status quo; further jumps are entirely possible. Depending on use case we can accept more uncertainty if it comes with benefits. I also don’t think there is a reason to believe that self-learning models must be unpredictable.
jnd-cz 7mo ago
Unless you use your oen local models then you don't even know when OpenAI or Anthropic tweaked the model less or more. One week it's a version x, next week it's a version y. Just like your operating system is continuously evolving with smaller patches of specific apps to whole new kernel version and new OS release.
- 7mo ago

charcircuit 7mo ago

Agents have the ability of continual learning.

andy12_ 7mo ago
Putting stuff you have learned into a markdown file is a very "shallow" version of continual learning. It can remember facts, yes, but I doubt a model can master new out-of-distribution tasks this way. If anything, I think that Google's Titans[1] and Hope[2] architectures are more aligned with true continual learning (without being actual continual learning still, which is why they call it "test-time memorization"). [1] https://arxiv.org/pdf/2501.00663 https://arxiv.org/pdf/2501.00663 [2] https://arxiv.org/pdf/2512.24695 https://arxiv.org/pdf/2512.24695
- charcircuit 7mo ago
  I have had it master tasks by doing this. The first time it tries to solve an issue it may take a long time, but it documents its findings and how it was able to do it and then it applies that knowledge the next time the task comes up.
  - andy12_ 7mo ago
    There is some things that just don't transfer really well without specific training. I tried to create diagrams in Typst with Cetz (a Processing and Tikz inspired graphing library), and even with documentation, GPT 5.2-thinking can't really do complex nice diagrams like it can in Tikz. It can do simple things that are similar to the shown examples, but nothing really interesting. Typst and specially Cetz is too new for any current model to really "get it", so they can't use it. I need to wait to the next batch of frontier models so that they learn Typst and Cetz examples during pre-training.
- patapong 7mo ago
  It really reminds me of the movie Memento - it has to constantly put notes down to remember who it is and what it should do after waking up without memory every n minutes.

nurettin 7mo ago

Who knows? Perhaps attention really is all you need. Maybe our context window is really large. Or our compression is really effective. Perhaps adding external factors might be able to indirectly teach the models to act more in line with social expectations such as being embarrassed to repeat the same mistake, unlocking the final piece of the puzzle. We are still stumbling in the dark for answers.

jacquesm 7mo ago

The main difference is humans are learning all the time and models learn batch wise and forget whatever happened in a previous session unless someone makes it part of the training data so there is a massive lag. Whoever cracks the continuous customized (per user, for instance) learning problem without just extending the context window is going to be making a big splash. And I don't mean cheats and shortcuts, I mean actually tuning the model based on received feedback.

aurareturn 7mo ago
Why not just provide more compute for say, 1 billion token context for each user to mimic continuous learning. Then retrain the model in the background to include learnings. The user wouldn’t know if the continuous learning came from the context or the model retrained. It wouldn’t matter. Continuous learning seems to be a compute and engineering problem.
- jacquesm 7mo ago
  Because that re-training is not strong enough to hold, or so it seems. The same dumb factual errors keep coming up on different generations of the same models. I've yet to see proof that something 'stuck' from model to model. They get better in a general sense but not in the specific sense that what was corrected stays put, not from session to session and not from one generation to the next. My solution is to have this massive 'boot up' prompt but it becomes extremely tedious to maintain.
eloisant 7mo ago
They can write to files then refer to them in a next session. A bit like the main character played by Guy Pierce in the movie Memento (which doesn't work great for him to be honest).

edgyquant 7mo ago

Iirc LeCunn talks about a self organizing hierarchy of real world objects and imo this is exactly how the human brain actually learns

stanfordkid 7mo ago

It's pretty simple... the word circle and what you can correlate to it via english language description has somewhat less to do with reality than a physical 3D model of a circle and what it would do in an environment. You can't just add more linguistic description via training data to change that. It doesn't really matter that you can keep back propagating because what you are back propagating over is fundamentally and qualitatively less rich.

mxkopy 7mo ago

The reason LLMs fail today is because there’s no meaning inherent to the tokens they produce other than the one captured by cooccurrence within text. Efforts like these are necessary because so much of “general intelligence” is convention defined by embodied human experience, for example arrows implying directionality and even directionality itself.

anon7000 7mo ago

I don’t understand your view. Reality is that we need some way to encode the rules of the world in a more definitive way. If we want models to be able to make assertive claims about important information and be correct, it’s very fair to theorize they might need a more deterministic approach than just training them more. But it’s just a theory that this will actually solve the problem. Ultimately, we still have a lot to learn and a lot of experiments to do. It’s frankly unscientific to suggest any approaches are off the table, unless the data & research truly proves that. Why shouldn’t we take this awesome LLM technology and bring in more techniques to make it better? A really, really basic example is chess. Current top AI models still don’t know how to play it (https://www.software7.com/blog/ai_chess_vs_1983_atari/ https://www.software7.com/blog/ai_chess_vs_1983_atari/) The models are surely trained on source material that include chess rules, and even high level chess games. But the models are not learning how to play chess correctly. They don’t have a model to understand how chess actually works — they only have a non-deterministic prediction based on what they’ve seen, even after being trained on more data than any chess novice has ever seen about the topic. And this is probably one of the easiest things for AI to stimulate. Very clear/brief rules, small problem space, no hidden information, but it can’t handle the massive decision space because its prediction isn’t based on the actual rules, but just “things that look similar” (And yeah, I’m sure someone could build a specific LLM or agent system that can handle chess, but the point is that the powerful general purpose models can’t do it out of the box after training.) Maybe more training & self-learning can solve this, but it’s clearly still unsolved. So we should definitely be experimenting with more techniques.

andy12_ 7mo ago
> Reality is that we need some way to encode the rules of the world in a more definitive way I mean, sure. But do world models the way LeCun proposes them solves this? I don't think so. JEPAs are just an unsupervised machine learning model at the end of the day; they might end up being better that just autoregressive pretraining on text+images+video, but they are not magic. For example, if you train a JEPA model on data of orbital mechanics, will it learn actually sensible algorithms to predict the planets' motions or will it just learn a mix of heuristic?

slashdave 7mo ago

If your model is poor, no amount of learning can fix it. If you don't think your model architecture is limited, you aren't looking hard enough.

the_black_hand 7mo ago

yes those are bottlenecks that world models don't solve. but the promise of world models is, unlike LLMs, they might be able to learn things about the world that humans haven't written. For example, we still don't fully know how insects fly. A world model could be trained on thousands of videos of insects and make a novel observation about insect trajectories. The premise is that despite being here for millenia, humans have only observed a tiny fraction of the world. So I do buy his idea. But I disagree that you need world models to get to human level capabilities. IMO there's no fundamental reason why models can't develop human understanding based on the known human observations.

eloisant 7mo ago

LeCun is a researcher. From his point of view, there are not much research left on LLM. Sure we can still improve them a bit with engineering around, but he's more interested in basic research.

a1371 7mo ago

I never understood why we believe humans don't backprop. Isn't it that during the day we fill up our context (short term memory) and sleep is actually where we use that to backprop? Heck, everyone knows what "sleep on it" means.

cedilla 7mo ago
Brains are not doing linear algebra, and they don't follow a concise algorithm. What LLM do is even farther away from what neural nets do, and even there - artificial neurons are inspired by but not reimplementing biological neurons. You can understand human thought in terms of LLMs, but that is just a simile, like understanding physical reality in terms of computers or clockworks.

carlmr 7mo ago

Especially they will require even more compute to get anything close to usable output. Human brains are super efficient at learning and producing output. We will need exponentially more compute for real time learning from video + audio + haptic data.