4 ms·
This doesn't work like that. An analogy would be giving a 5 year old a task that requires the understanding of the world of an 18 year old. It doesn't matter wh
by lossolo 11mo ago
This doesn't work like that. An analogy would be giving a 5 year old a task that requires the understanding of the world of an 18 year old. It doesn't matter whether you give that child 5 minutes or 10 hours, they won't be capable of solving it.
- HarHarVeryFunny 11mo agoI think the question of what can be achieved with a small model comes down to what needs knowledge vs what needs experience. A small model can use tools like RAG if it is just missing knowledge, but it seems hard to avoid training/parameters where experience is needed - knowing how to perceive then act. There is obviously also some amount (maybe a lot) of core knowledge and capability needed even to be able to ask the right questions and utilize the answers.
- lossolo 11mo agoSmall models handle simple, low context tasks most of the time correctly. But for more complex tasks, they fail due to insufficient training capacity and too few parameters to integrate the necessary relationships.
- homarp 11mo agobut in 13 years, will they be capable?
- lossolo 11mo agoNo. They will go out of context and collapse into loops.
- nkmnz 11mo agoWhat if you give them 13 years?
- fluoridation 11mo agoThen they're not a 5-year-old anymore.
- lossolo 11mo agoNothing will change. They will go out of context and collapse into loops.
- nkmnz 11mo agoI mean the 5yo child, not the LLM