4 ms·
I did not state that children learn from little. I did state that they are able to achieve impressive outcomes quickly with very small amounts of domain specifi
by korse 3y ago
I did not state that children learn from little. I did state that they are able to achieve impressive outcomes quickly with very small amounts of domain specific knowledge (movement not included).
Our best language models at this point appear to be narrow and deep where the child model is wide and shallow. For example, how many pictures of apples do I need in order to train a robot to recognize apples? How many apples (or representations thereof) does a child need to see to identify an apple? I won't argue about image density, but human eyes aren't that high resolution and people don't seem to have excellent high resolution multi-frame memory.
This just makes we wonder if we should be spending more time working on how we utilize our wide but shallow data corpus rather than playing I can't believe it's not statistics with hugely inefficient and focused data sets.
- lostmsu 3y agoYou are wrong again. There's no difference between ChatGPT and a child in that regard. That's like half of the point of GPT's zero shot performance on the tasks that involve made up language that you describe directly in the query. You are doubly wrong because somehow for a robot you are counting pictures of apples, and for a child you are counting distinct apples. The answer to "how many pictures of apples does child need to see" is somewhere in thousands. Remember, brain does an equivalent of 24 fps, so a child holding an apple for 5 seconds gets an equivalent of over 100 pictures.
- lostmsu 3y agoHere, stumbled upon in a different thread: https://news.ycombinator.com/item?id=37273290 https://news.ycombinator.com/item?id=37273290 Too bad they are not saying how long it took to get to the recognition.