> RNNs have "lost the hardware lottery" by being structurally not that efficient to train on the cost-effective hardware that's available. Which suggests two o

> RNNs have "lost the hardware lottery" by being structurally not that efficient to train on the cost-effective hardware that's available. Which suggests two obvious paths forward: 1. Don't bother learning / using RNN's 2. Co-develop new hardware / new RNN architectures that work together to provide great performance per unit of price. Now of course nobody is saying (well, I am not saying) that (2) would be easy... or even necessarily possible. But somebody should at least be intrigued by the idea. And in the world we live in today where FPGA's and other devices make it easier than ever to experiment with custom hardware architectures... it might be worth taking a stab at it.