3 ms·

They train on years of real-time streaming video. Feedback loops all over the place. Higher order structures and rules all get learned way, way before they even

They train on years of real-time streaming video. Feedback loops all over the place. Higher order structures and rules all get learned way, way before they even are able to touch a book. I cannot phantom the amount of data and analyses that passes a typical human child in a day. So, sure, they can learn fast after all that. But so can GPT, which also one-shots a lot.

lelanthran 4y ago
> They train on years of real-time streaming video. ... > I cannot phantom the amount of data and analyses that passes a typical human child in a day. You're counting every second of every image that a child sees as a separate "image" that is learned. I'm counting the number of objects that a child sees, because people aren't doing frame-capture and analysis on every image, they're doing lazy interpolation (which is why all your peripheral vision in in monochrome but you perceive it as full color). A 6 month old baby who has only ever seen close family (parents, siblings, pets) is quite able to discern the same features on strangers. Nothing we have in AI right now can take 3 example models of humans, and detect hair, nose and mouth in a brand new never before seen model of a human. Not really the same thing as training on millions of different images and videos. The number of distinct and different objects in a 7 yo child's training set is rarely the same as "every single youtube video ever uploaded". Going back to the original point: ChatGPT needs millions of books to learn to regurgitate information as well as a semi-intelligent young adult, while typical toddlers are quite able to regurgitate what they've just seen. Humans and animals get by with millions of times less training data than AI models need.
- sebzim4500 4y ago
  >You're counting every second of every image that a child sees as a separate "image" that is learned. I'm counting the number of objects that a child sees, because people aren't doing frame-capture and analysis on every image, they're doing lazy interpolation (which is why all your peripheral vision in in monochrome but you perceive it as full color). Then it sounds like he is slightly overestimating the dataset (since some of the frame is monochrome) and you are massively underestimating it.
- hexane360 4y ago
  Faces may not be the best example, because there's a lot of evidence babies are 'hardcoded' to be interested in and pay attention to faces. This implies there's some knowledge of them built in. However, if you substitute abstract shapes (blocks, balls, etc.), your point absolutely stands up. In some ways, though, isn't it more interesting to have an intelligence that learns in a completely different way from humans, as opposed to one that learns through essentially the same method?

lelanthran 4y ago

> They train on years of real-time streaming video. ... > I cannot phantom the amount of data and analyses that passes a typical human child in a day. You're counting every second of every image that a child sees as a separate "image" that is learned. I'm counting the number of objects that a child sees, because people aren't doing frame-capture and analysis on every image, they're doing lazy interpolation (which is why all your peripheral vision in in monochrome but you perceive it as full color). A 6 month old baby who has only ever seen close family (parents, siblings, pets) is quite able to discern the same features on strangers. Nothing we have in AI right now can take 3 example models of humans, and detect hair, nose and mouth in a brand new never before seen model of a human. Not really the same thing as training on millions of different images and videos. The number of distinct and different objects in a 7 yo child's training set is rarely the same as "every single youtube video ever uploaded". Going back to the original point: ChatGPT needs millions of books to learn to regurgitate information as well as a semi-intelligent young adult, while typical toddlers are quite able to regurgitate what they've just seen. Humans and animals get by with millions of times less training data than AI models need.

sebzim4500 4y ago
>You're counting every second of every image that a child sees as a separate "image" that is learned. I'm counting the number of objects that a child sees, because people aren't doing frame-capture and analysis on every image, they're doing lazy interpolation (which is why all your peripheral vision in in monochrome but you perceive it as full color). Then it sounds like he is slightly overestimating the dataset (since some of the frame is monochrome) and you are massively underestimating it.
hexane360 4y ago
Faces may not be the best example, because there's a lot of evidence babies are 'hardcoded' to be interested in and pay attention to faces. This implies there's some knowledge of them built in. However, if you substitute abstract shapes (blocks, balls, etc.), your point absolutely stands up. In some ways, though, isn't it more interesting to have an intelligence that learns in a completely different way from humans, as opposed to one that learns through essentially the same method?