In my view, there's a big difference between successfully training, say, LSTM RNNs, versus successfully training "vanilla" MLPs. This work opens the door for u

In my view, there's a big difference between successfully training, say, LSTM RNNs, versus successfully training "vanilla" MLPs. This work opens the door for using new kinds of massively parallel "neuromorphic" hardware to implement orders of magnitude more layers and units, without requiring greater communications bandwidth between layers, because the model no longer needs to wait until gradients have back-propagated from the last to the first layer before moving on to the next sample. Scaling backpropagation to GPT-3 levels and beyond (think trillions of dense connections) is very hard -- it requires a lot of complicated plumbing and bookkeeping. Wouldn't you want to be able to throw 100x, 1000x, or even 1Mx more fully distributed computing power at problems? This work has paved a path pointing in that direction :-)