6 ms·
Why AI systems don't learn – On autonomous learning from cognitive science
- aanet 7mo agoby Emmanuel Dupoux, Yann LeCun, Jitendra Malik "he proposed framework integrates learning from observation (System A) and learning from active behavior (System B) while flexibly switching between these learning modes as a function of internally generated meta-control signals (System M). We discuss how this could be built by taking inspiration on how organisms adapt to real-world, dynamic environments across evolutionary and developmental timescales. "
- dasil003 7mo agoIf this was done well in a way that was productive for corporate work, I suspect the AI would engage in Machievelian maneuvering and deception that would make typical sociopathic CEOs look like Mister Rogers in comparison. And I'm not sure our legal and social structures have the capacity to absorb that without very very bad things happening.
- marsten 7mo agoAgents playing the iterated prisoner's dilemma learn to cooperate. It's usually not a dominant strategy to be entirely sociopathic when other players are involved.
- ehnto 7mo agoYou don't get that many iterations in the real world though, and if one of your first iterations is particularly bad you don't get any more iterations.
- cortesoft 7mo agoBut AI will train in the artificial world
- ehnto 7mo agoThey still fail in the real world, where a single failure can be highly consequential. AI coding is lucky it has early failure modes, pretty low consequence. But I don't see how that looks for an autonomous management agent with arbitrary metrics as goals. Anyone doing AI coding can tell you once an agent gets on the wrong path, it can get very confused and is usually irrecoverable. What does that look like in other contexts? Is restarting the process from scratch even possible in other types of work, or is that unique to only some kinds of work?
- naasking 7mo ago> You don't get that many iterations in the real world though True, for iterations between the same two players, but humans evolved the ability to communicate and so can share the results of past interactions through a network with other agents, aka a reputation. Thus any interaction with a new person doesn't start from a neutral prior.
- gotwaz 7mo agoNot just CEOs, Legal and social structures will also be run by AI. Chimps with 3 inch brains cant handle the level of complexity global systems are currently producing.
- AdieuToLogic 7mo ago> If this was done well in a way that was productive for corporate work, I suspect the AI would engage in Machievelian maneuvering and deception that would make typical sociopathic CEOs look like Mister Rogers in comparison. Algorithms do not possess ethics nor morality[0] and therefore cannot engage in Machiavellianism[1]. At best, algorithms can simulate same as pioneered by ELIZA[2], from which the ELIZA effect[3] could be argued as being one of the best known forms of anthropomorphism. 0 - https://www.psychologytoday.com/us/basics/ethics-and-morality https://www.psychologytoday.com/us/basics/ethics-and-moralit... 1 - https://en.wikipedia.org/wiki/Machiavellianism_(psychology) https://en.wikipedia.org/wiki/Machiavellianism_(psychology) 2 - https://en.wikipedia.org/wiki/ELIZA https://en.wikipedia.org/wiki/ELIZA 3 - https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect
- qsera 7mo agohttps://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect >As Weizenbaum later wrote, "I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."... That pretty much explain the AI Hysteria that we observe today.
- reverius42 7mo agoELIZA couldn't write working code from an English-language prompt though. I think the "AI Hysteria" comes more from current LLMs being actually good at replacing a lot of activity that coders are used to doing regularly. I wonder what Weizenbaum would think of Claude or ChatGPT.
- qsera 7mo ago>ELIZA couldn't write working code from an English-language prompt though. Yea, that is kind of the point. Even such a system could trick people into delusional thinking. > actually good at replacing a lot of activity that coders are used to... I think even that is unrealistic. But that is not what I was thinking. I was thinking when people say that current LLMs will go on improving and reach some kind of real human like intelligence. And ELIZA effect provides a prefect explanation for this. It is very curious that this effect is the perfect thing for scamming investors who are typically bought into such claims, but under ELIZA effect with this, they will do 10x or 100x investment....
- tim333 7mo agoI was kind of worried by them going Machiavellian or evil but it doesn't seem the default state for current ones, I think because they are basically trained on the whole internet which has a lot of be nice type stuff. No doubt some individual humans my try to make them go that way though. I guess it would depend a bit whos interests the AI would be serving. If serving the shareholders it would probably reward creating value for customers, but if it was serving an individual manager competing with others to be CEO say then the optimum strategy might be to go machiavellian on the rivals.
- estearum 7mo ago> I think because they are basically trained on the whole internet which has a lot of be nice type stuff. Is this not just because their goals are currently to be seen as "nice"? Surely they can be not-nice if directed to, and then the question is just whether someone can accidentally direct them to do that by e.g. setting up goals that can be more readily achieved by being not-nice. Which... is how many goals in the real world are, which is why the very concept and danger of Machiavellianism exists.
- tim333 7mo agoI've been amused at Musk vs Grok with Grok saying he's the biggest spreader of misinformation and not doing very well when he tells it to go on about white genocide in South Africa. I don't know how easy it is to modify these things in a subtle manner.
- iFire 7mo agohttps://github.com/plastic-labs/honcho https://github.com/plastic-labs/honcho has the idea of one sided observations for RAG.
- beernet 7mo agoThe paper's critique of the 'data wall' and language-centrism is spot on. We’ve been treating AI training like an assembly line where the machine is passive, and then we wonder why it fails in non-stationary environments. It’s the ultimate 'padded room' architecture: the model is isolated from reality and relies on human-curated data to even function. The proposed System M (Meta-control) is a nice theoretical fix, but the implementation is where the wheels usually come off. Integrating observation (A) and action (B) sounds great until the agent starts hallucinating its own feedback loops. Unless we can move away from this 'outsourced learning' where humans have to fix every domain mismatch, we're just building increasingly expensive parrots. I’m skeptical if 'bilevel optimization' is enough to bridge that gap or if we’re just adding another layer of complexity to a fundamentally limited transformer architecture.
- lock-locku 7mo ago[dead]
- g_a_a_s 7mo ago[dead]
- jdkee 7mo agoLeCun has been talking about his JEPA models for awhile. https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/ https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
- Xunjin 7mo agoIn this podcast episode[0] he does talk about this kind of model and how it "learns about physics" through experience instead of just ingesting theorical material. It's quite eye opening. 0. https://youtu.be/qvNCVYkHKfg https://youtu.be/qvNCVYkHKfg
- aurareturn 7mo agoThe way I see it, the "world models" he wants to train require a magnitude more compute than what LLM training requires since physical data is likely much more unstructured than internet data. He raised $1b but that seems way too little to buy enough compute to train. My bet is that OpenAI or Anthropic or both will eventually train the model that he always wanted because they will use revenue from LLMs to train a world model.
- zhangchen 7mo ago[flagged]
- robot-wrangler 7mo ago> Curious how you'd handle the reward signal for deciding when to switch between observation and active exploration without it collapsing into one mode. If you like biomimetic approaches to computer science, there's evidence that we want something besides neural networks. Whether we call such secondary systems emotions, hormones, or whatnot doesn't really matter much if the dynamics are useful. It seems at least possible that studying alignment-related topics is going to get us closer than any perspective that's purely focused on learning. Coincidentally quanta is on some related topics today: https://www.quantamagazine.org/once-thought-to-support-neurons-astrocytes-turn-out-to-be-in-charge-20260130/ https://www.quantamagazine.org/once-thought-to-support-neuro...
- t-writescode 7mo agoOr possibly “in addition to”, yeah. I think this is where it needs to go. We can’t keep training HUGE neural networks every 3 months and throw out all the work we did and the billions of dollars in gear and training just to use another model a few months. That loops is unsustainable. Active learning needs to be discovered / created.
- exe34 7mo agoif that's the arguement for active learning, wouldn't it also apply in that case? it learns something and 5 minutes later my old prompts are useless.
- t-writescode 7mo agoThat depends on the goals of the prompts you use with the LLM: * as a glorified natural language processor (like I have done), you'll probably be fine, maybe * as someone to communicate with, you'll also probably be fine * as a *very* basic prompt-follower? Like, natural language processing-level of prompt "find me the important words", etc. Probably fine, or close enough. * as a robust prompt system with complicated logic each prompt? Yes, it will begin to fail catastrophically, especially if you're wanting to be repeatable. I'm not sure that the general public is that interested in perfectly repeatable work, though. I think they're looking for consistent and improving work.
- tranchms 7mo agoWe are rediscovering Cybernetics
- walterbell 7mo agoBiological Computer Laboratory (1958-1976), https://web.archive.org/web/20190829234412/http://bcl.ece.illinois.edu/ https://web.archive.org/web/20190829234412/http://bcl.ece.il...
- QuesnayJr 7mo agoIt's striking how cybernetics has gone from dated to timely.
- internet_points 7mo agoI've tried figuring out what the big deal about cybernetics was, but I always come away with a feeling of it being a bit wish-washy. Is it a bit like Philosophy in that it birthed individual fields that were inspired by and made applications of the thoughts, models and ideas laid out by its forebears? Or were there actual proofs, discoveries or applications in the field itself? (I guess one could call projects like https://en.wikipedia.org/wiki/Project_Cybersyn https://en.wikipedia.org/wiki/Project_Cybersyn an "application" of its ideas, though cut off before one could see the results.)
- Frannky 7mo agoCan I run it?
- theLewisLu 7mo ago[dead]
- Animats 7mo agoNot learning from new input may be a feature. Back in 2016 Microsoft launched one that did, and after one day of talking on Twitter it sounded like 4chan.[1] If all input is believed equally, there's a problem. Today's locked-down pre-trained models at least have some consistency. [1] https://www.bbc.com/news/technology-35890188 https://www.bbc.com/news/technology-35890188
- Earw0rm 7mo agoIncredible to accomplish that in a day - it took the rest of the world another decade to make Twitter sound like 4chan, but thanks to Elon we got there in the end.
- bheadmaster 7mo ago[flagged]
- Culonavirus 7mo ago[flagged]
- bonesss 7mo agoTwitter is not like it always was. The presence of oranges doesn’t speak to the volume or rot-level of the apples. Twitter has lost advertisers, credibility, and legitimacy. That’s objectively demonstrable in the calibre, quantity, and aims of their advertisers, and their loss of revenue. Twitter is hurting humanity, and has swaths of the population trapped in misinformation clouds. Arguably Elon bought the last election by purchasing it, and current administration issues are the result. But for the slow acclimatization and general brain fog of the “etch a sketch voters” we’d see Twitters direct reprogramming of opinion and behaviour as a psychic virus. You can tell which app people are hooked on by the lies they believe (with great emotional resonance). Social Media is becoming increasingly restricted from children based on objective developmental and cognitive impacts, I dare speculate we and our parents are the asbestos eating unfiltered cigarette smoking pre-modern victims who misused something terribly until we figured out how bad that shizz is for us.
- est 7mo ago"don't learn" might be a good feature from a business point of view Imagine if AI learns all your source code and apply them to your competitor /facepalm
- utopiah 7mo agoI remember a joke from few years ago that was showing an "AI" that was "learning" on its "own" which meant periodically starting from scratch with a new training set curated by a large team of researchers themselves relying on huge teams (far away) of annotators. TL;DR: depends where you defined the boundaries of your "system".
- p_v_doom 7mo agoI think from a proper systemic view that joke is more correct than not. AI is just the frontend of people ...
- krinne 7mo agoBut doesnt existing AI systems already learn in some way ? Like the training steps are actually the AI learning already. If you have your training material being setup by something like claude code, then it kind of is already autonomous learning.
- LovelyButterfly 7mo agoMost, if not all, commercially available AI models are doing offline learning. The cognition is a skill that is only possible on online learning which is the autonomous part the authors refer to, that is, learning by observing, interacting. In that sense the "autonomous" part you said simply meant that the data source is coming from a different place, but the model itself is not free to explore with a knowledge base to deduce from, but rather infer on what is provided to it.
- reverius42 7mo ago> The cognition is a skill that is only possible on online learning which is the autonomous part the authors refer to, that is, learning by observing, interacting. This is the "Claude Code" part, or even the ChatGPT (web interface/app) part. Large context window full of relevant context. Auto-summarization of memories and inclusion in context. Tool calling. Web searching. If not LLMs, I think we can say that those systems that use them in an "agentic" way perhaps have cognition?
- troupo 7mo agoNo, no they don't. Actual learning survives beyond "sufficient context window". Start a new chat, and the "agentic" system will be as clueless as before
- reverius42 7mo agoThey can write to the filesystem, and future instances can read it (and write more). The agentic system does not remain as clueless as its LLM's first instantiation.
- logicchains 7mo agoThere's already a model capable of autonomous learning on the small scale, just nobody's tried to scale it up yet: https://arxiv.org/abs/2202.05780 https://arxiv.org/abs/2202.05780
- lovebite4u_ai 7mo agoclaude is learning very fast
- followin_io82 7mo ago[dead]
- himata4113 7mo agoEh, honestly? We're not that far away from models training themselves (opus 4.6 and codex 5.3 were both 'instrumental' in training themselves). They're capable enough to put themselves in a loop and create improvement which often includes processing new learnings from bruteforcing. It's not in real-time, but that probably a good thing if anyone remembers microsofts twitter attempt.
- tim333 7mo agoI was thinking in the same way that the human brain's design came about from evolutionary trial and error, we may be close to a situation where we can do something like that for the artificial neural networks and have the computers improve them by fiddling about.
- shevy-java 7mo agoThe whole AI field is a misnomer. It stole so much from neurobiology. However had, there will come a time when AI will really learn. My prediction is that it will come with a different hardware; you already see huge strides here with regards to synthetic biology. While this focuses more on biology still, you'll eventually see a bridging effort; cyborg novels paved the way. Once you have real hardware that can learn, you'll also have real intelligence in AI too.
- seedpi 7mo ago[flagged]
- jvanderbot 7mo agoWhat I find interesting is the supposition that weights must change. The connections of my motherboard do not change, yet it can simulate any system. Perhaps there is an architecture that is write-once-read-forever, and all that matters is context. There's almost certainly some of this in the human mind, and I bet there is much more of it than we are willing to admit. No amount of mental gymnastics is going to let you visualize 6D structures.
- tim333 7mo ago>supposition that weights must change The thing is that's where most of the leaning and 'intelligence' is. If you don't change them the model doesn't really get smarter.
- jvanderbot 7mo ago> The thing is that's where most of the leaning and 'intelligence' is The question is: Is it required for AGI that the model changes its weights _during deployment_, or can we train up and deploy like we do now and manage learning via context? Taken to extreme, "context" could be defined as the "change in weights from training time" so the answer is trivially "yes", but that seems like cheating.
- nylon4831 7mo agoLike Memento
- Garlef 7mo agoI think restrcicting this discussion to LLMs - as it is often done - misses the point: LLMs + harnesses can actually learn. That's why I think the term "system" as used in the paper is much better.
- troupo 7mo ago> LLMs + harnesses can actually learn. No. No, they don't
- BrianFHearn 7mo ago[dead]
- theptip 7mo agoIt’s interesting, LeCun seems to have a blind spot around in-context learning. I didn’t find one mention in this paper (only skimmed the full paper so far so may have missed), which is odd as it is the way that agents come closest to autonomous learning in the real world. I would say his core point does still apply; autonomous learning is not solved by ICL. But it seems a strawman to ignore the topic entirely and focus on training. From what I see on the ground, some degree of autonomous learning is possible; Agents can already be set up to use meta-learning skills for skill authoring, introspection, rumination, etc - but these loops are not very effective currently. I wonder if this is the myopic viewpoint of a scientist who doesn’t engage with the engineering of how these systems are actually used in the real world (ie “my work is done once Llama is released with X score on Y eval”) which results in a markedly different stance than the guys like Sutskever, Karpathy, Amodei who have built end-to-end systems and optimized for customer/business outcomes.
- scotttaylor 7mo ago[dead]
- webagent255 7mo ago[dead]