3 ms·
I'd say the main issue is why does it take ingesting the entire collective knowledge of humanity for LLMs to work if they are actually reasoning and understand
by ren_engineer 3y ago
I'd say the main issue is why does it take ingesting the entire collective knowledge of humanity for LLMs to work if they are actually reasoning and understand things from first principles? Humans don't require that volume of training data to learn how to write or do basic math
- Detrytus 3y agoWell, I think you underestimate the amount of "training data" that human gets in the early stages of their life. Just learning to speak takes a few years, during which all people around you talk to you for most of your waking hours. Learning to write and doing basic math typically does not start until age of 6.
- asadotzler 3y agoNo chance it comes close to the content of the internet.
- hnfong 3y agoYou (humanity) don't learn from first principles either. The 3-year-old toddler doesn't learn the rules of English grammar before they learn to speak. Sure, we don't know how humans manage to learn with so "little" data yet, but it's definitely not "understand things from first principles". In fact, if you randomly pick an average person from the streets, they wouldn't be able to explain most things from first principles. (eg. How does their car work?)