4 ms·

I feel like this is more an argument against the efficiency of flat (non-hierarchical, not model-based) Deep Reinforcement Learning than an argument for Genetic

I feel like this is more an argument against the efficiency of flat (non-hierarchical, not model-based) Deep Reinforcement Learning than an argument for Genetic/Evolutionary Algorithms. As in, if you're not solving a task more efficiently than Evolutionary Algorithms or more difficult than they can handle, you're not doing something right. Similar to how Deep PCA and Deep Random Forests blew ConvNets out of the water on basic benchmarks like MNIST, but couldn't compete on larger datasets, indicated the proof we should require before getting excited about a new technique.

Seanny123 10y ago
Deep PCA: https://arxiv.org/abs/1404.3606 https://arxiv.org/abs/1404.3606 Deep Forest: https://arxiv.org/abs/1702.08835 https://arxiv.org/abs/1702.08835
- YeGoblynQueenne 10y ago
  I think this is the Deep Forest paper you meant: http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kontschieder_Deep_Neural_Decision_ICCV_2015_paper.pdf http://www.cv-foundation.org/openaccess/content_iccv_2015/pa... That's Deep Neural Decision Forests; they benchmark against MNIST and obtain state of the art results. The other paper is about improving decision forests, although it also uses MNIST as a benchmark. Apologies if I'm wrong and you did mean the Zhou & Feng paper.
  - Seanny123 10y ago
    That one is good too!
argonaut 10y ago
Not sure where you're getting this from, but CNNs are definitely still state of the art on MNIST. Papers often cite outdated CNN numbers; in fact the best published CNN accuracy numbers are probably lower than they could be - vanilla MNIST for supervised learning is a pretty useless benchmark for computer vision researchers now.
- Seanny123 10y ago
  My bad, I should have been more specific. The Deep PCA paper shows superior performance not on baseline MNIST, but on the MNIST variations. I also agree that vanilla MNIST is pretty useless for Computer Vision researches and was trying (awkwardly) to support that idea by showing how these other non Deep Learning techniques performed equally well.
  - argonaut 10y ago
    The Deep PCA paper shows no such thing. They've cherry-picked bad CNN baselines. The red flag for me is that the "state of the art" methods they compare to achieve > 1% test error on vanilla MNIST, which is totally wrong since CNNs routinely achieve <0.5 % test error (even by the standards of 2014, when the paper was published).

Seanny123 10y ago

Deep PCA: https://arxiv.org/abs/1404.3606 https://arxiv.org/abs/1404.3606 Deep Forest: https://arxiv.org/abs/1702.08835 https://arxiv.org/abs/1702.08835

YeGoblynQueenne 10y ago
I think this is the Deep Forest paper you meant: http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kontschieder_Deep_Neural_Decision_ICCV_2015_paper.pdf http://www.cv-foundation.org/openaccess/content_iccv_2015/pa... That's Deep Neural Decision Forests; they benchmark against MNIST and obtain state of the art results. The other paper is about improving decision forests, although it also uses MNIST as a benchmark. Apologies if I'm wrong and you did mean the Zhou & Feng paper.
- Seanny123 10y ago
  That one is good too!

argonaut 10y ago

Not sure where you're getting this from, but CNNs are definitely still state of the art on MNIST. Papers often cite outdated CNN numbers; in fact the best published CNN accuracy numbers are probably lower than they could be - vanilla MNIST for supervised learning is a pretty useless benchmark for computer vision researchers now.

Seanny123 10y ago
My bad, I should have been more specific. The Deep PCA paper shows superior performance not on baseline MNIST, but on the MNIST variations. I also agree that vanilla MNIST is pretty useless for Computer Vision researches and was trying (awkwardly) to support that idea by showing how these other non Deep Learning techniques performed equally well.
- argonaut 10y ago
  The Deep PCA paper shows no such thing. They've cherry-picked bad CNN baselines. The red flag for me is that the "state of the art" methods they compare to achieve > 1% test error on vanilla MNIST, which is totally wrong since CNNs routinely achieve <0.5 % test error (even by the standards of 2014, when the paper was published).