3 ms·

just curious - can you give an example for a big breaktrhough he co-authored? nonetheless - some of his remarks are very specific and I don't see how informal

just curious - can you give an example for a big breaktrhough he co-authored? nonetheless - some of his remarks are very specific and I don't see how informal style applies here to excuse them.

lars 12y ago
He was second author on the AlexNet paper, wherein Alex Krizhevsky, Sutskever and Hinton blew everyone else out of the water on the ImageNet competition [2]. Their error rate was about 10 percentage points lower than others. Relatively speaking they had about 40% fewer errors than anyone else. This is possibly the biggest result in computer vision the last five years. So it seems a little silly to educate him on the basics of machine learning :) [1] http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf [2] http://www.image-net.org/challenges/LSVRC/2012/results.html http://www.image-net.org/challenges/LSVRC/2012/results.html
- joyofdata 12y ago
  well thanks for the info - but then I shift my critique to that I find it unnecessary to distort ML and biological concepts just to simplify the subject, when an accurate depiction wouldn't be much more difficult. Especially to not differentiate properly between memorization and generalization/learning is odd b/c this is one of the most prominent mistakes - it is specifically not the goal to minimize the in-sample-error! that would lead to very bad results most of the time
  - p1esk 12y ago
    Actually, Ilya explains his statement regarding minimizing training errors in his comment exchange with Bengio: "Although I didn't define it in the article, generalization (to me) means that the gap between the training and the test error is small. So for example, a very bad model that has similar training and test errors does not overfit, and hence generalizes, according to the way I use these concepts. It follows that generalization is easy to achieve whenever the capacity of the model (as measured by the number of parameters or its VC-dimension) is limited --- we merely need to use more training cases than the model has parameters / VC dimension. Thus, the difficult part is to get a low training error."

just curious - can you give an example for a big breaktrhough he co-authored? nonetheless - some of his remarks are very specific and I don't see how informal style applies here to excuse them.

lars 12y ago

He was second author on the AlexNet paper, wherein Alex Krizhevsky, Sutskever and Hinton blew everyone else out of the water on the ImageNet competition [2]. Their error rate was about 10 percentage points lower than others. Relatively speaking they had about 40% fewer errors than anyone else. This is possibly the biggest result in computer vision the last five years. So it seems a little silly to educate him on the basics of machine learning :) [1] http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf [2] http://www.image-net.org/challenges/LSVRC/2012/results.html http://www.image-net.org/challenges/LSVRC/2012/results.html

joyofdata 12y ago
well thanks for the info - but then I shift my critique to that I find it unnecessary to distort ML and biological concepts just to simplify the subject, when an accurate depiction wouldn't be much more difficult. Especially to not differentiate properly between memorization and generalization/learning is odd b/c this is one of the most prominent mistakes - it is specifically not the goal to minimize the in-sample-error! that would lead to very bad results most of the time
- p1esk 12y ago
  Actually, Ilya explains his statement regarding minimizing training errors in his comment exchange with Bengio: "Although I didn't define it in the article, generalization (to me) means that the gap between the training and the test error is small. So for example, a very bad model that has similar training and test errors does not overfit, and hence generalizes, according to the way I use these concepts. It follows that generalization is easy to achieve whenever the capacity of the model (as measured by the number of parameters or its VC-dimension) is limited --- we merely need to use more training cases than the model has parameters / VC dimension. Thus, the difficult part is to get a low training error."