5 ms·
> This result makes it very clear that without regularity imposed by the ground truth mechanism which generates the actual data (e.g., some natural process), a
by throwawaysea 6y ago
> This result makes it very clear that without regularity imposed by the ground truth mechanism which generates the actual data (e.g., some natural process), a neural net is unlikely to perform well on an example which deviates strongly (as defined by the kernel) from all training examples.
Is this another way of saying that neural networks are just another statistical estimation method and not a path to general artificial intelligence? Or is it saying that problems like self-driving cars are not suitable to the current state of the art for AI since we have to ensure that reality doesn't deviate from training examples? Or both?
I'd love to understand the "real life" implications of this finding better.
- phreeza 6y agoIs it not possible to achieve AGI with a good statistical estimation method?
- viraptor 6y agoDepends if you think "a human unlikely to perform well on an example which deviates strongly (as defined by experience) from all training examples." is true.
- fallingfrog 6y agoA human can learn from a single training sample. I can show my 2 year old a cartoon picture of an unfamiliar animal- say, an anteater- and then show him a page of drawings by a different artist in a different style, of several animals, in various poses, and he can immediately point to the anteater. Think about that: a single example, crudely drawn. You and I could do that too. Human beings are masters of learning from training sets of n=1. I’m not sure why humans are so good at that. I feel like it might be that by spending the first couple years of life picking things up and rotating them and putting them in our mouths, we have some deeper understanding that images of an animal like an anteater are not really two dimensional but are supposed to represent volume and form. That’s my guess. A neural net trained to recognize 2d images will not have that.
- viraptor 6y agoI don't think that's a fair comparison. It's not 1 sample. It's 1 sample + existing training in segmentation + existing model for expected animal build + existing model for 2d/3d extrapolation from the drawing + pre-trained capacity for reinforcement learning, etc. If you start slightly before 2yo, you may find that after they see a cow they will call every large animal a cow for weeks/months - that's closer to one sample without all the models.
- im3w1l 6y agoI think he is saying that a competition where both sides are allowed to bring a lot of preconceptions into the learning task is an interesting case where people currently beat the machines by a lot.
- fallingfrog 6y agoIt’s not a fair comparison, of course, but you know, I don’t think the public cares as much about whether it’s fair as they care about how well it works. I guess what I’m saying is that to match human performance you’d need all that prior understanding that you just listed. Maybe that’s what’s missing from our models.
- Grimm1 6y agoWhen this was discussed about two months ago the conclusion I took away was not very many at the moment beyond somewhat formal equivalence. https://news.ycombinator.com/item?id=25314830 https://news.ycombinator.com/item?id=25314830
- 6gvONxR4sf7o 6y agoNeither. It's more akin to the fundamental theorem of calculus. If you follow your data from point A to point B along some differential path, you can sum up/integrate the steps to follow your data in a single jump from point A to point B directly (with the jump in terms of those integrals). It's a super interesting viewpoint on gradient descent and models that use it that could be really useful in looking at and understanding those models abstractly, but isn't saying anything about suitability for different tasks.