3 ms·
Probably the failure I consider the biggest is that I spent most of my PhD trying to solve supervised learning for computer vision using unsupervised feature le
by nl 9y ago
Probably the failure I consider the biggest is that I spent most of my PhD trying to solve supervised learning for computer vision using unsupervised feature learning methods, and was caught totally off guard when Alex, Ilya, and Geoff won the ImageNet contest with purely supervised methods. I think that in general wasting time writing papers that turn out to be dead ends is the main way that I fail in my own eyes.
Wow, what an amazing story.
And worth pointing out that Goodfellows work on unsupervised feature learning is clearly seen in his invention of GANs.
Edit, and:
I think it’s hard to extract value from negative results in machine learning because it can be so hard to tell what caused the negative result. A negative result might point to something very fundamental wrong with an idea, but it might also just be the result of a very small software bug, the wrong idea of the hyperparameter values to try out, too small of a model, etc.
So, so true. I read comments on HN on how important negative results are in ML and I look at all the things I’ve tried and someone else made work, and all I see is reporting negative results just discourages ppl from trying things.
This is a great interview.