Neural network renaissance definitely renewed the interests to use neural networks as function approximators in many areas. For the one I am interested in rece

Neural network renaissance definitely renewed the interests to use neural networks as function approximators in many areas. For the one I am interested in recently (dynamics control), many networks are small feed-forward ones or two-layer LSTMs. The importance there is about balance robust controller v.s. update frequency. If you can run the same controller at higher frequency, it is basically free performance improvement (the robot will be more stable, more responsive etc.).