3 ms·
>> "This almost always happen if you train a deep neural network long enough: train performance goes up, but test performance goes down." This is a problem tha
by sdenton4 5y ago
>> "This almost always happen if you train a deep neural network long enough: train performance goes up, but test performance goes down."
This is a problem that is more common for classification problems, I think. Generative and self-supervised models (trained with augmentation) tend to just get better forever (with some asymptote) because memorization isn't a viable strategy.
I personally think image classification is mostly a silly problem to judge new algos on as a result, and leads to all kinds of nonsense as people try to extrapolate meaning from new results.
- dontreact 5y agoNearly all useful machine learning is supervised, still. And if you are using a neural network, it will eventually memorize. This is fine though, we have early stopping :)
- sdenton4 5y agoEh, I work in audio ml, where most of the interesting and useful work is in conditioned generative models. Compression, TTS, source separation. The bias towards classification is a side effect of people starting at imagenet too long; I really think it's holding the field back.