I think there are a few additional important details. It's not so much that they stack a random forest on top of a neural network, as that they are trying to t

I think there are a few additional important details. It's not so much that they stack a random forest on top of a neural network, as that they are trying to train neural networks that have tree-like properties. Deep neural networks can be expensive to evaluate because every layer is needed to compute the resulting label. On the other hand, in a decision tree you only need to evaluate one of a nodes children (i.e., evaluation is linear in the depth of the tree). They combine these ideas by allowing some nodes in the network to branch (so for a given image, you choose one of two neural networks to evaluate). These nodes don't have to occur at the top level, but can occur in the middle of the network as well. The difficulty is how you train these nodes; that's where the differential technique for optimizing trees come in. Because you have a unified training algorithm, you can mix and match tree-like nodes and neural network like nodes and train the whole thing. All in all, the hope is that getting a label for a given node is much faster, so you can run these on smaller devices such as phones without sacrificing accuracy.