3 ms·

The fact that NN does not generalize in a particular (arguably very challenging) task trained on a particular data set etc. does not mean NN never generalizes.

The fact that NN does not generalize in a particular (arguably very challenging) task trained on a particular data set etc. does not mean NN never generalizes. Don't know exactly what is meant by "generalization" in this context. I'd argue that it's very unclear what the difference between "interpolation" and "extrapolation" even is in a high dimensional and sparsely sampled space. Would a NN trained to recognize "fur" trained on different kinds of dog fur also activate when it encounters wolf, cat or even bear fur? This seems quite plausible, and seems like a kind of generalization.

euphetar 5y ago
It's indeed a very challenging task. I picked it because it's one of those where the lack of generalization is apparent. In fact, researchers study commonsense QA and similar tasks to find how we can reach generalization. I agree, it's very hard to fidn the difference. Especially for GPT or BERT, where the training set is basically the whole internet. It's a very good question about fur. I would suspect that it would correctly recognise all kinds of dog-like fur and sometimes fail on different furs, like bear furs. But in general NN's are very good at textures, so maybe it will just be good on all kinds of fur. One problem that might arise is that you will show it something fur-like but not really fur, and it will think it's fur. I agree it's some kind of generalization. Here I don't have enough background to draw the line, perhaps a more theorically oriented person could, but I can't. I guess the most important judge is that if you devise a benchmark like commonsense Q&A, a neural network fails it. Or how a Tesla will recognise a truck full of red stop signs as a real stop sign, while a "generalizing" thing like a human would definitely know that a core property of a stop sign is that it should be installed near a road. So there is a real problem.

euphetar 5y ago

It's indeed a very challenging task. I picked it because it's one of those where the lack of generalization is apparent. In fact, researchers study commonsense QA and similar tasks to find how we can reach generalization. I agree, it's very hard to fidn the difference. Especially for GPT or BERT, where the training set is basically the whole internet. It's a very good question about fur. I would suspect that it would correctly recognise all kinds of dog-like fur and sometimes fail on different furs, like bear furs. But in general NN's are very good at textures, so maybe it will just be good on all kinds of fur. One problem that might arise is that you will show it something fur-like but not really fur, and it will think it's fur. I agree it's some kind of generalization. Here I don't have enough background to draw the line, perhaps a more theorically oriented person could, but I can't. I guess the most important judge is that if you devise a benchmark like commonsense Q&A, a neural network fails it. Or how a Tesla will recognise a truck full of red stop signs as a real stop sign, while a "generalizing" thing like a human would definitely know that a core property of a stop sign is that it should be installed near a road. So there is a real problem.