13 ms·

Deep Forest: Towards an Alternative to Deep Neural Networks

350 points by kercker 10y ago

bamboozled 10y ago
HN just won't be the same
DanielleMolloy 10y ago
So if this works well why is there no comparison on ImageNet?
- plafl 10y ago
  That's exactly what I was going to post, but you beat me. I found surprising that they claim competitive performance over neural networks just using a CPU. The advantage I think neural networks have over all other methods is how efficiently they run on GPUs. I would love to see other methods take advantage of modern hardware or parallelism and see if they can catch up with neural networks on image classification.
  - BickNowstrom 10y ago
    Random forests can be run on the GPU: https://github.com/EasonLiao/CudaTree https://github.com/EasonLiao/CudaTree
- kercker 10y ago
  In the conclusion section the authors said that : "If we had stronger computational facilities, we would like to try big data and deeper forest, which is left for future work." and that: "As a seminar study, we have only explored a little in this direction."
  - DanielleMolloy 10y ago
    If someone proposes a method as an alternative for a field, they need to test this method on the accepted benchmark dataset for that field. For object recognition in static images this dataset is the ImageNet competition. Computing power can be bought from AWS if no cluster is available. The lack of it can't be an argument. Not saying that the paper has no reason to exist, I think it is generally well written and decision trees certainly deserve attention. If they can do representation learning on high level this is certainly something to look into. But it shouldn't claim to be an alternative to state-of-the-art deep learning if there is no data for this comparison. Everyone can solve MNIST (or even CIFAR).
    - cgearhart 10y ago
      I agree their claim is a bit hyperbolic, but that's an unreasonably high bar to expect for the scope of this paper.
- levesque 10y ago
  It might be an alternative for deep neural networks, but I doubt it will be an alternative for deep convolutional neural networks. The key breakthrough for deep learning was the stacking of many convolutional layers and being able to train those (key elements include SGD for large scale, dropout or batch norm, etc.). Perhaps if they build some convolutional features for the decision tree they will be able to achieve comparable results, but it will start to look a lot like a deep convolutional neural net (i.e., the same thing with a different name).
- PeterisP 10y ago
  It doesn't do well - e.g. on the toy example of MNIST they report 98.96% which is, frankly, rather horrible. They claim that deep neural approaches get 98.75% or 99.05% by referencing obsolete decade old results, while in fact state of art exceeds 99.8% (i.e. 0.2% error rate, which is five times lower than 1.0% error rate reported in this paper). I have seen MNIST given as a homework exercise for undergraduate students in ML/NN class, and getting 99.0% would indicate that your code has some serious bugs, a decent undergrad with no prior experience can get 99.7% on MNIST after a few lecture introduction to basics and a dozen hours of homework coding practice.
jre 10y ago
I don't know about the others, but the two visions dataset they compare to (MNIST and the face recognition one) are small datasets and the CNN they compare to doesn't seem very state of the art. It also seems each layer of random forest just concatenates a class distribution to the original feature vector. So this doesn't seem to get the same "hierarchy of features" benefit that you get in large-scale CNN and DNN.
- aerioux 10y ago
  ^ The authors time and effort they observe it takes to create state-of-the-art CNNs, but their point-of-comparison CNNs look to be fairly simple -- I don't see an AlexNet or something for some of these tasks either just as a point of comparison even if not a fully relevant one
- jkbschwarz 10y ago
  To your point that they are comparing small datasets. I dont see that as a problem. If they achieve better results on small datasets that is a great achievement, as often the bottleneck is the size of the dataset rather than computation time.
  - krona 10y ago
    > often the bottleneck is the size of the dataset rather than computation time That's generally true for DNNs, which is a good place to be if you have lots of data. This typically isn't true for tree based approaches, which is why they fell out of fashion in some problem domains; they don't generalize as well. This paper doesn't seem to change what we already know in this respect.
  - argonaut 10y ago
    They achieve substantially worse results than state of the art on (the toy dataset) MNIST.
- curuinor 10y ago
  Comparable to SOTA circa 1998.
deleted 10y ago
[deleted]
FrozenVoid 10y ago
Related: Deep neural decision forests (ConvNets+Random Forests) http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kontschieder_Deep_Neural_Decision_ICCV_2015_paper.pdf http://www.cv-foundation.org/openaccess/content_iccv_2015/pa... http://matthewalunbrown.com/papers/1603.01250v1.pdf http://matthewalunbrown.com/papers/1603.01250v1.pdf https://topos-theory.github.io/deep-neural-decision-forests/ https://topos-theory.github.io/deep-neural-decision-forests/
deleted 10y ago
[deleted]
throwaway312780 10y ago
XGBoost also appears to have a GPU implementation.
- pilooch 10y ago
  Yes, as a plugin, and on its way to make it to the trunk AFAIK. I've integrated it into deepdetect recently because even as a beta it works well and complements GPU based DNNs fairly naturally. Deep learning practitioners are already well equipped with GPUs so having XGBoost run on them as well is a good bonus!
rkaplan 10y ago
"In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train." Hyperparameter tuning is not as much of an issue with deep neural networks anymore. Thanks to BatchNorm and more robust optimization algorithms, most of the time you can simply use Adam with a default learning rate of 0.001 and do pretty well. Dropout is not even necessary with many models that use BatchNorm nowadays, so generally tuning there is not an issue either. Many layers of 3x3 conv with stride 1 is still magical. Basically: deep NNs can work pretty well with little to no tuning these days. The defaults just work.
- computerex 10y ago
  I couldn't disagree more. The defaults don't just work, and the architecture of the network could also be considered a hyper parameter in which case what would be a reasonable default for all the types of problems ANN are used for?
  - ipunchghosts 10y ago
    Are you using batch normalization? If you are, an issue I see all the time is folks not setting the EMA filter coef correctly. In keras, it defaults to something like 0.99 which in my mind makes no sense. I use something around 0.6 and life is good. You want to get an overall good measurement of the statistics and in my mind the frequency cutoff when coef=0.99 is just way too high for most application. You usually want something that filters out just about everything except very close to DC.
    - kortex 10y ago
      When working with images, do you use mode 0, 1, or 2?
    - dirtyaura 10y ago
      Interesting, totally new concept for me: Where can I read more about EMA filter coefficient in Keras? My Google-fu is failing.
uptownfunk 10y ago
Would like to see three things coming out of this 1. R code implementation (could probably write this myself but would make things easier) 2. How to get feature importance? Otherwise difficult to implement in business context. 3. Better benchmarks
paulsutter 10y ago
No Free Lunch theorem refesher: "if an algorithm performs well on a certain class of problems then it necessarily pays for that with degraded performance on the set of all remaining problems" https://en.m.wikipedia.org/wiki/No_free_lunch_theorem https://en.m.wikipedia.org/wiki/No_free_lunch_theorem
- CuriouslyC 10y ago
  That doesn't apply to an ensemble of algorithms where the weights of a given member of the ensemble are adapted based on observations from the given domain. If it did, humans wouldn't be able to choose a good algorithm for specific cases, and obviously we can. Deep neural networks can be thought of as ensembles of smaller neural networks, though of course each member of the ensemble is going to share some degree of algorithmic bias. This suggests that perhaps deep neural networks with heterogeneous activation functions and branching structures will perform better than homogeneous networks.
  - paulsutter 10y ago
    The human brain is exactly such an ensemble of algorithms, and look at all the cognitive biases we have[1]. That our brain was developed for foraging, hunting, and reproducing could explain why the code we write is so buggy. Look I'm just kidding around. Every algorithm has its advocates, and the arguments get tedious after a while. [1] https://en.m.wikipedia.org/wiki/List_of_cognitive_biases https://en.m.wikipedia.org/wiki/List_of_cognitive_biases
  - IanCal 10y ago
    The No Free Lunch theorem doesn't really care how your algorithm does it, and an ensemble of algorithms is just another algorithm. It certainly applies. The reason it's not actually all that useful for real world applications is that problems we value are a subset of "all possible problems" for want of a better explanation. Problems are typically smooth, that is a small change in input variables results in a small change in 'fitness', which already limits how applicable the NFL theorem is. [disclaimer, based on lectures 10 years ago now]
  - 10y ago
ungzd 10y ago
Was about to joke about Deep Support Vector Machines, but found out they exist too: https://www.esat.kuleuven.be/sista/ROKS2013/files/presentations/DSVM_ROKS_2013_WIERING.pdf https://www.esat.kuleuven.be/sista/ROKS2013/files/presentati... http://deeplearning.net/wp-content/uploads/2013/03/dlsvm.pdf http://deeplearning.net/wp-content/uploads/2013/03/dlsvm.pdf
- soVeryTired 10y ago
  Deep linear regression?
  - eanzenberg 10y ago
    That's quite literally deep learning so made me lol.
    - soVeryTired 10y ago
      Can you elaborate on that? I feel like I've missed something.
      - mljoe 10y ago
        There is no "deep logistic regression" because such a construct is just called a "neural network". (It's not the only kind of neural network however.)
      - eanzenberg 10y ago
        Linear regression on top of linear regression with non-linearities in between is a neural network.
        deleted 10y ago
        [deleted]
      - davesque 10y ago
        Neural networks are just networks of logistic regressors.
KirinDave 10y ago
While optimizations to cost by ditching GPUs as a requirement are important (and presumably these systems benefit from GPU optimization as well, seems unclear from my skim of the paper), cheaper training is NOT just about saving your wallet. A real emerging area of opportunity is having systems train new systems. This has numerous applications, including assisting DSEs in the construction of new systems or allowing expert systems to learn more over time and even integrate new techniques into a currently deployed system. I'n not an expert here, but I'd like to be, so I'm definitely going to ask my expert friends more about this.
throw_away_777 10y ago
I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.
- deleted 10y ago
  [deleted]
- nojvek 10y ago
  Any good links you recommend learning xgboost? I've never quite figured out how they work.
  - deleted 10y ago
    [deleted]
  - BickNowstrom 10y ago
    http://xgboost.readthedocs.io/en/latest/model.html http://xgboost.readthedocs.io/en/latest/model.html http://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf http://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf https://www.youtube.com/watch?v=wPqtzj5VZus https://www.youtube.com/watch?v=wPqtzj5VZus Trevor Hastie - Gradient Boosting Machine Learning https://www.youtube.com/watch?v=sRktKszFmSk https://www.youtube.com/watch?v=sRktKszFmSk Ensembles (3): Gradient Boosting, Ihler
- photoJ 10y ago
  There are lots of reasons, principally among them that Kaggle comps problems represent only one tiny fraction of ML problems. On general purpose ML, with out much time rigging a optimal solution, RF/Xgboost will preform better. But in many problems, ie vision, DL is vastly superior. The other important point is that there many new opportunities for researchers regarding DL, where statistical ML approach on supervised problems is a much more well established field.
argonaut 10y ago
The method's performance on MNIST is relatively mediocre. You might think 98.96% is amazing, but it's about relative performance. It is a relatively easy exercise nowadays to get above 99% with neural nets. Even I can get that kind of performance with hand-written Python neural nets, on the CPU, with no convolutions. For the rest of the (non-image) datasets, it's already common knowledge that boosting methods are competitive with neural nets.
edixon 10y ago
None of these experiments actually do anything to show feature learning - if this is the claim, I would like to see a transfer learning experiment. I would be surprised if this works well, since they can't jointly optimize their layers (so you can't just use ImageNet to induce a good representation). Not quite clear why we should think that trees will turn out to be inherently cheaper that a DNN with similar accuracy, unless perhaps the model structure encodes a prior which matches the distribution of the problem?
DrNuke 10y ago
Progress in this field is astonishing and it really propagates to the masses in the form of easy-to-use black boxes with a pinch of undergraduate-level maths. Just wow!
Dim25 10y ago
Please note the CPUs they have used are pretty advanced: 2x Intel E5 2670 v3 CPU (24 cores) - approx. price $1.5k per unit (http://ark.intel.com/products/81709/Intel-Xeon-Processor-E5-2670-v3-30M-Cache-2_30-GHz http://ark.intel.com/products/81709/Intel-Xeon-Processor-E5-...). Looking forward to try the code (especially on CIFAR or ImageNet), Zhi-Hua Zhou, one of the authors, said they are going to publish it soon.

350 points by kercker 10y ago

bamboozled 10y ago

HN just won't be the same

DanielleMolloy 10y ago

So if this works well why is there no comparison on ImageNet?

plafl 10y ago
That's exactly what I was going to post, but you beat me. I found surprising that they claim competitive performance over neural networks just using a CPU. The advantage I think neural networks have over all other methods is how efficiently they run on GPUs. I would love to see other methods take advantage of modern hardware or parallelism and see if they can catch up with neural networks on image classification.
- BickNowstrom 10y ago
  Random forests can be run on the GPU: https://github.com/EasonLiao/CudaTree https://github.com/EasonLiao/CudaTree
kercker 10y ago
In the conclusion section the authors said that : "If we had stronger computational facilities, we would like to try big data and deeper forest, which is left for future work." and that: "As a seminar study, we have only explored a little in this direction."
- DanielleMolloy 10y ago
  If someone proposes a method as an alternative for a field, they need to test this method on the accepted benchmark dataset for that field. For object recognition in static images this dataset is the ImageNet competition. Computing power can be bought from AWS if no cluster is available. The lack of it can't be an argument. Not saying that the paper has no reason to exist, I think it is generally well written and decision trees certainly deserve attention. If they can do representation learning on high level this is certainly something to look into. But it shouldn't claim to be an alternative to state-of-the-art deep learning if there is no data for this comparison. Everyone can solve MNIST (or even CIFAR).
  - cgearhart 10y ago
    I agree their claim is a bit hyperbolic, but that's an unreasonably high bar to expect for the scope of this paper.
levesque 10y ago
It might be an alternative for deep neural networks, but I doubt it will be an alternative for deep convolutional neural networks. The key breakthrough for deep learning was the stacking of many convolutional layers and being able to train those (key elements include SGD for large scale, dropout or batch norm, etc.). Perhaps if they build some convolutional features for the decision tree they will be able to achieve comparable results, but it will start to look a lot like a deep convolutional neural net (i.e., the same thing with a different name).
PeterisP 10y ago
It doesn't do well - e.g. on the toy example of MNIST they report 98.96% which is, frankly, rather horrible. They claim that deep neural approaches get 98.75% or 99.05% by referencing obsolete decade old results, while in fact state of art exceeds 99.8% (i.e. 0.2% error rate, which is five times lower than 1.0% error rate reported in this paper). I have seen MNIST given as a homework exercise for undergraduate students in ML/NN class, and getting 99.0% would indicate that your code has some serious bugs, a decent undergrad with no prior experience can get 99.7% on MNIST after a few lecture introduction to basics and a dozen hours of homework coding practice.

jre 10y ago

I don't know about the others, but the two visions dataset they compare to (MNIST and the face recognition one) are small datasets and the CNN they compare to doesn't seem very state of the art. It also seems each layer of random forest just concatenates a class distribution to the original feature vector. So this doesn't seem to get the same "hierarchy of features" benefit that you get in large-scale CNN and DNN.

aerioux 10y ago
^ The authors time and effort they observe it takes to create state-of-the-art CNNs, but their point-of-comparison CNNs look to be fairly simple -- I don't see an AlexNet or something for some of these tasks either just as a point of comparison even if not a fully relevant one
jkbschwarz 10y ago
To your point that they are comparing small datasets. I dont see that as a problem. If they achieve better results on small datasets that is a great achievement, as often the bottleneck is the size of the dataset rather than computation time.
- krona 10y ago
  > often the bottleneck is the size of the dataset rather than computation time That's generally true for DNNs, which is a good place to be if you have lots of data. This typically isn't true for tree based approaches, which is why they fell out of fashion in some problem domains; they don't generalize as well. This paper doesn't seem to change what we already know in this respect.
- argonaut 10y ago
  They achieve substantially worse results than state of the art on (the toy dataset) MNIST.
curuinor 10y ago
Comparable to SOTA circa 1998.

deleted 10y ago

[deleted]

FrozenVoid 10y ago

Related: Deep neural decision forests (ConvNets+Random Forests) http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Kontschieder_Deep_Neural_Decision_ICCV_2015_paper.pdf http://www.cv-foundation.org/openaccess/content_iccv_2015/pa... http://matthewalunbrown.com/papers/1603.01250v1.pdf http://matthewalunbrown.com/papers/1603.01250v1.pdf https://topos-theory.github.io/deep-neural-decision-forests/ https://topos-theory.github.io/deep-neural-decision-forests/

deleted 10y ago

[deleted]

throwaway312780 10y ago

XGBoost also appears to have a GPU implementation.

pilooch 10y ago
Yes, as a plugin, and on its way to make it to the trunk AFAIK. I've integrated it into deepdetect recently because even as a beta it works well and complements GPU based DNNs fairly naturally. Deep learning practitioners are already well equipped with GPUs so having XGBoost run on them as well is a good bonus!

rkaplan 10y ago

"In contrast to deep neural networks which require great effort in hyper-parameter tuning, gcForest is much easier to train." Hyperparameter tuning is not as much of an issue with deep neural networks anymore. Thanks to BatchNorm and more robust optimization algorithms, most of the time you can simply use Adam with a default learning rate of 0.001 and do pretty well. Dropout is not even necessary with many models that use BatchNorm nowadays, so generally tuning there is not an issue either. Many layers of 3x3 conv with stride 1 is still magical. Basically: deep NNs can work pretty well with little to no tuning these days. The defaults just work.

computerex 10y ago
I couldn't disagree more. The defaults don't just work, and the architecture of the network could also be considered a hyper parameter in which case what would be a reasonable default for all the types of problems ANN are used for?
- ipunchghosts 10y ago
  Are you using batch normalization? If you are, an issue I see all the time is folks not setting the EMA filter coef correctly. In keras, it defaults to something like 0.99 which in my mind makes no sense. I use something around 0.6 and life is good. You want to get an overall good measurement of the statistics and in my mind the frequency cutoff when coef=0.99 is just way too high for most application. You usually want something that filters out just about everything except very close to DC.
  - kortex 10y ago
    When working with images, do you use mode 0, 1, or 2?
  - dirtyaura 10y ago
    Interesting, totally new concept for me: Where can I read more about EMA filter coefficient in Keras? My Google-fu is failing.

uptownfunk 10y ago

Would like to see three things coming out of this 1. R code implementation (could probably write this myself but would make things easier) 2. How to get feature importance? Otherwise difficult to implement in business context. 3. Better benchmarks

paulsutter 10y ago

No Free Lunch theorem refesher: "if an algorithm performs well on a certain class of problems then it necessarily pays for that with degraded performance on the set of all remaining problems" https://en.m.wikipedia.org/wiki/No_free_lunch_theorem https://en.m.wikipedia.org/wiki/No_free_lunch_theorem

CuriouslyC 10y ago
That doesn't apply to an ensemble of algorithms where the weights of a given member of the ensemble are adapted based on observations from the given domain. If it did, humans wouldn't be able to choose a good algorithm for specific cases, and obviously we can. Deep neural networks can be thought of as ensembles of smaller neural networks, though of course each member of the ensemble is going to share some degree of algorithmic bias. This suggests that perhaps deep neural networks with heterogeneous activation functions and branching structures will perform better than homogeneous networks.
- paulsutter 10y ago
  The human brain is exactly such an ensemble of algorithms, and look at all the cognitive biases we have[1]. That our brain was developed for foraging, hunting, and reproducing could explain why the code we write is so buggy. Look I'm just kidding around. Every algorithm has its advocates, and the arguments get tedious after a while. [1] https://en.m.wikipedia.org/wiki/List_of_cognitive_biases https://en.m.wikipedia.org/wiki/List_of_cognitive_biases
- IanCal 10y ago
  The No Free Lunch theorem doesn't really care how your algorithm does it, and an ensemble of algorithms is just another algorithm. It certainly applies. The reason it's not actually all that useful for real world applications is that problems we value are a subset of "all possible problems" for want of a better explanation. Problems are typically smooth, that is a small change in input variables results in a small change in 'fitness', which already limits how applicable the NFL theorem is. [disclaimer, based on lectures 10 years ago now]
- 10y ago

ungzd 10y ago

Was about to joke about Deep Support Vector Machines, but found out they exist too: https://www.esat.kuleuven.be/sista/ROKS2013/files/presentations/DSVM_ROKS_2013_WIERING.pdf https://www.esat.kuleuven.be/sista/ROKS2013/files/presentati... http://deeplearning.net/wp-content/uploads/2013/03/dlsvm.pdf http://deeplearning.net/wp-content/uploads/2013/03/dlsvm.pdf

soVeryTired 10y ago
Deep linear regression?
- eanzenberg 10y ago
  That's quite literally deep learning so made me lol.
  - soVeryTired 10y ago
    Can you elaborate on that? I feel like I've missed something.
    - mljoe 10y ago
      There is no "deep logistic regression" because such a construct is just called a "neural network". (It's not the only kind of neural network however.)
    - eanzenberg 10y ago
      Linear regression on top of linear regression with non-linearities in between is a neural network.
      - deleted 10y ago
        [deleted]
    - davesque 10y ago
      Neural networks are just networks of logistic regressors.

KirinDave 10y ago

While optimizations to cost by ditching GPUs as a requirement are important (and presumably these systems benefit from GPU optimization as well, seems unclear from my skim of the paper), cheaper training is NOT just about saving your wallet. A real emerging area of opportunity is having systems train new systems. This has numerous applications, including assisting DSEs in the construction of new systems or allowing expert systems to learn more over time and even integrate new techniques into a currently deployed system. I'n not an expert here, but I'd like to be, so I'm definitely going to ask my expert friends more about this.

throw_away_777 10y ago

I've always found it curious that Neural Networks get so much hype when xgboost (gradient boosted decision trees) is by far the most popular and accurate algorithm for most Kaggle competitions. While neural networks are better for image processing types of problems, there are a wide variety of machine learning problems where decision tree methods perform better and are much easier to implement.

deleted 10y ago
[deleted]
nojvek 10y ago
Any good links you recommend learning xgboost? I've never quite figured out how they work.
- deleted 10y ago
  [deleted]
- BickNowstrom 10y ago
  http://xgboost.readthedocs.io/en/latest/model.html http://xgboost.readthedocs.io/en/latest/model.html http://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf http://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf https://www.youtube.com/watch?v=wPqtzj5VZus https://www.youtube.com/watch?v=wPqtzj5VZus Trevor Hastie - Gradient Boosting Machine Learning https://www.youtube.com/watch?v=sRktKszFmSk https://www.youtube.com/watch?v=sRktKszFmSk Ensembles (3): Gradient Boosting, Ihler
photoJ 10y ago
There are lots of reasons, principally among them that Kaggle comps problems represent only one tiny fraction of ML problems. On general purpose ML, with out much time rigging a optimal solution, RF/Xgboost will preform better. But in many problems, ie vision, DL is vastly superior. The other important point is that there many new opportunities for researchers regarding DL, where statistical ML approach on supervised problems is a much more well established field.

argonaut 10y ago

The method's performance on MNIST is relatively mediocre. You might think 98.96% is amazing, but it's about relative performance. It is a relatively easy exercise nowadays to get above 99% with neural nets. Even I can get that kind of performance with hand-written Python neural nets, on the CPU, with no convolutions. For the rest of the (non-image) datasets, it's already common knowledge that boosting methods are competitive with neural nets.

edixon 10y ago

None of these experiments actually do anything to show feature learning - if this is the claim, I would like to see a transfer learning experiment. I would be surprised if this works well, since they can't jointly optimize their layers (so you can't just use ImageNet to induce a good representation). Not quite clear why we should think that trees will turn out to be inherently cheaper that a DNN with similar accuracy, unless perhaps the model structure encodes a prior which matches the distribution of the problem?

DrNuke 10y ago

Progress in this field is astonishing and it really propagates to the masses in the form of easy-to-use black boxes with a pinch of undergraduate-level maths. Just wow!

Dim25 10y ago

Please note the CPUs they have used are pretty advanced: 2x Intel E5 2670 v3 CPU (24 cores) - approx. price $1.5k per unit (http://ark.intel.com/products/81709/Intel-Xeon-Processor-E5-2670-v3-30M-Cache-2_30-GHz http://ark.intel.com/products/81709/Intel-Xeon-Processor-E5-...). Looking forward to try the code (especially on CIFAR or ImageNet), Zhi-Hua Zhou, one of the authors, said they are going to publish it soon.