4 ms·

So if this works well why is there no comparison on ImageNet?

plafl 10y ago
That's exactly what I was going to post, but you beat me. I found surprising that they claim competitive performance over neural networks just using a CPU. The advantage I think neural networks have over all other methods is how efficiently they run on GPUs. I would love to see other methods take advantage of modern hardware or parallelism and see if they can catch up with neural networks on image classification.
- BickNowstrom 10y ago
  Random forests can be run on the GPU: https://github.com/EasonLiao/CudaTree https://github.com/EasonLiao/CudaTree
kercker 10y ago
In the conclusion section the authors said that : "If we had stronger computational facilities, we would like to try big data and deeper forest, which is left for future work." and that: "As a seminar study, we have only explored a little in this direction."
- DanielleMolloy 10y ago
  If someone proposes a method as an alternative for a field, they need to test this method on the accepted benchmark dataset for that field. For object recognition in static images this dataset is the ImageNet competition. Computing power can be bought from AWS if no cluster is available. The lack of it can't be an argument. Not saying that the paper has no reason to exist, I think it is generally well written and decision trees certainly deserve attention. If they can do representation learning on high level this is certainly something to look into. But it shouldn't claim to be an alternative to state-of-the-art deep learning if there is no data for this comparison. Everyone can solve MNIST (or even CIFAR).
  - cgearhart 10y ago
    I agree their claim is a bit hyperbolic, but that's an unreasonably high bar to expect for the scope of this paper.
levesque 10y ago
It might be an alternative for deep neural networks, but I doubt it will be an alternative for deep convolutional neural networks. The key breakthrough for deep learning was the stacking of many convolutional layers and being able to train those (key elements include SGD for large scale, dropout or batch norm, etc.). Perhaps if they build some convolutional features for the decision tree they will be able to achieve comparable results, but it will start to look a lot like a deep convolutional neural net (i.e., the same thing with a different name).
PeterisP 10y ago
It doesn't do well - e.g. on the toy example of MNIST they report 98.96% which is, frankly, rather horrible. They claim that deep neural approaches get 98.75% or 99.05% by referencing obsolete decade old results, while in fact state of art exceeds 99.8% (i.e. 0.2% error rate, which is five times lower than 1.0% error rate reported in this paper). I have seen MNIST given as a homework exercise for undergraduate students in ML/NN class, and getting 99.0% would indicate that your code has some serious bugs, a decent undergrad with no prior experience can get 99.7% on MNIST after a few lecture introduction to basics and a dozen hours of homework coding practice.

plafl 10y ago

That's exactly what I was going to post, but you beat me. I found surprising that they claim competitive performance over neural networks just using a CPU. The advantage I think neural networks have over all other methods is how efficiently they run on GPUs. I would love to see other methods take advantage of modern hardware or parallelism and see if they can catch up with neural networks on image classification.

BickNowstrom 10y ago
Random forests can be run on the GPU: https://github.com/EasonLiao/CudaTree https://github.com/EasonLiao/CudaTree

kercker 10y ago

In the conclusion section the authors said that : "If we had stronger computational facilities, we would like to try big data and deeper forest, which is left for future work." and that: "As a seminar study, we have only explored a little in this direction."

DanielleMolloy 10y ago
If someone proposes a method as an alternative for a field, they need to test this method on the accepted benchmark dataset for that field. For object recognition in static images this dataset is the ImageNet competition. Computing power can be bought from AWS if no cluster is available. The lack of it can't be an argument. Not saying that the paper has no reason to exist, I think it is generally well written and decision trees certainly deserve attention. If they can do representation learning on high level this is certainly something to look into. But it shouldn't claim to be an alternative to state-of-the-art deep learning if there is no data for this comparison. Everyone can solve MNIST (or even CIFAR).
- cgearhart 10y ago
  I agree their claim is a bit hyperbolic, but that's an unreasonably high bar to expect for the scope of this paper.

levesque 10y ago

It might be an alternative for deep neural networks, but I doubt it will be an alternative for deep convolutional neural networks. The key breakthrough for deep learning was the stacking of many convolutional layers and being able to train those (key elements include SGD for large scale, dropout or batch norm, etc.). Perhaps if they build some convolutional features for the decision tree they will be able to achieve comparable results, but it will start to look a lot like a deep convolutional neural net (i.e., the same thing with a different name).

PeterisP 10y ago

It doesn't do well - e.g. on the toy example of MNIST they report 98.96% which is, frankly, rather horrible. They claim that deep neural approaches get 98.75% or 99.05% by referencing obsolete decade old results, while in fact state of art exceeds 99.8% (i.e. 0.2% error rate, which is five times lower than 1.0% error rate reported in this paper). I have seen MNIST given as a homework exercise for undergraduate students in ML/NN class, and getting 99.0% would indicate that your code has some serious bugs, a decent undergrad with no prior experience can get 99.7% on MNIST after a few lecture introduction to basics and a dozen hours of homework coding practice.