2 ms·
> The whole "Could it happen"? section is a bit strange. On the one hand, it focuses on CNNs when it's clear we're talking about a binary classifier (the artic
by gwern 9y ago
> The whole "Could it happen"? section is a bit strange. On the one hand, it focuses on CNNs when it's clear we're talking about a binary classifier (the article itself points that out). If Fredkin was really the originator of the story, then discussing CNNs is an anachronism (they were 30 years away at the time).
? How is it strange? Presumably people are not, right now, still retelling the story because they are terribly afraid that there are perceptrons out there from the 1960s lurking, waiting to strike, or that anyone is going to go out and try to use 1960s style perceptrons. People are telling it as a cautionary story about current NNs, in the 2000s and 2010s and 2017. Which means... CNNs. So it's worth asking, can it happen with CNNs as trained by any reasonably standard workflow?
> More importantly, it's obvious that "it" could definitely happen and in fact happens a lot- "it" being overfitting to examples.
Overfitting is not dataset bias, as I note several times. For example, dropout or heldout datasets or crossvalidation are highly effective in fighting/detecting overfitting, but do nothing about dataset bias.