5 ms·
> Contrast with the way a human learns skills - as we gain experience with a skill, we get better at understanding when it's the right tool for the job. Which
by mbesto 1y ago
> Contrast with the way a human learns skills - as we gain experience with a skill, we get better at understanding when it's the right tool for the job.
Which is precisely why Richard Sutton doesn't think LLMs will evolve to AGI[0]. LLMs are based on mimicry, not experience, so it's more likely (according to Sutton) that AGI will be based on some form of RL (reinforcement learning) and not neural networks (LLMs).
More specifically, LLMs don't have goals and consequences of actions, which is the foundation for intelligence. So, to your point, the idea of a "skill" is more akin to a reference manual, than it is a skill building exercise that can be applied to developing an instrument, task, solution, etc.
[0] https://www.youtube.com/watch?v=21EYKqUsPfg https://www.youtube.com/watch?v=21EYKqUsPfg
- buildbot 1y agoThe industry has been doing RL on many kinds of neural networks, including LLMs, for quite some time. Is this person saying we RL on some kind of non neural network design? Why is that more likely to bring AGI than an LLM?. > More specifically, LLMs don't have goals and consequences of actions, which is the foundation for intelligence. Citation?
- jfarina 1y agoWhy are you asking them to cite something for that statement? Are you questioning whether it's the foundation for intelligence or whether LLMS understand goals and consequences?
- anomaloustho 1y agoLooks like they added the link. But I think it’s doing RL in realtime vs pre-trained as an LLM is. And I associate that part to AGI being able to do cutting edge research and explore new ideas like humans can. Where, when that seems to “happen” with LLMs it’s been more debatable. (e.g. there was an existing paper that the LLM was able to tap into) I guess another example would be to get an AGI doing RL in realtime to get really good at a video game with completely different mechanics in the same way a human could. Today, that wouldn’t really happen unless it was able to pre-train on something similar.
- ibejoeb 1y agoI don't think any of the commercial models are doing RL at the consumer. The R is just accepting or rejecting the action, right?
- hbarka 1y agoFor humans, it’s not uncommon to have a clever realization by way of serendipity. How do you skill AI to have serendipity.
- mediaman 1y agoIt's a false dichotomy. LLMs are already being trained with RL to have goal directedness. He is right that non-RL'd LLMs are just mimicry, but the field already moved beyond that.
- leptons 1y agoI can't wait to try to convince an LLM/RL/whatever-it-is that what it "thinks" is right is actually wrong.
- dingnuts 1y agoExplain something to me that I've long wondered: how does Reinforcement Learning work if you cannot measure your distance from the goal? In other words, how can RL be used for literally anything qualitative?
- kmacdough 1y agoThis is one of known hardest parts of RL. The short answer is human feedback. But this is easier said than done. Current models require vastly more learning events than humans, making direct supervision infeasable. One strategy is to train models on human supervisors, so they can bear the bulk of the supervision. This is tricky, but has proven more effective than direct supervision. But, in my experience, AIs don't specifically struggle with the "qualitative" side of things per-se. In fact, they're great at things like word choice, color theory, etc. Rather, they struggle to understand continuity, consequence and to combine disparate sources of input. They also suck at differentiating fact from fabrication. To speculate wildly, it feels like it's missing the the RL of living in the "real world". In order to eat, sleep and breath, you must operate within the bounds of physics and society and live forever with the consequences of an ever-growing history of choices.
- mbesto 1y agoThis 100%. While we might agreed that language is foundational to what it is to be human, it's myopic to think its the only thing. LLMs are based on training sets of language (period).
- vonneumannstan 1y agoThis is an uninformed take. Much of the improvement in performance of LLM based models has been through RLHF and other RL techniques.
- mbesto 1y ago> This is an uninformed take. You may disagree with this take but its not uninformed. Many LLMs use self‑supervised pretraining followed by RL‑based fine‑tuning but that's essentially it - it's fine tuning.
- vonneumannstan 1y agoI think you're seriously underestimating the importance of the RL steps on LLM performance. Also how do you think the most successful RL models have worked? AlphaGo/AlphaZero both use Neural Networks for their policy and value networks which are the central mechanism of those models.
- skurilyak 1y agoBesides a "reference manual", Claude Skills is analogous to a "toolkit with an instruction manual" in that it includes both instructions (manuals) and executable functions (tools/code)
- Weeenion 1y agoI would love to understand were this notion of LLM becoming AGI ever came from? ChatGPT broke upen the dam to massive budget on AI/LM and LLM will probably be a puzzle peace to AGI. But otherwise? I mean it should be clear that we have so much work to do like RL (which now happens btw. on massive scale because you thumb up or down every day), thinking, Model of Experts, toolcalling and super super critical: Architecture. Compute is a hard upper limit too. And the math isn't done either. The performance of Context length has advanced, we also saw other approcheas like a diffusion based models. Whenever you hear the leading experts talking, they mention world models. We are still in a phase were we have plenty of very obivous ideas people need to try out. But alone the quality of whispher, llm as an interface and tool calling can solve problems with robotics and stuff, no one was able to solve that easy ever before.