That's bad. There are gradient descent techniques that are feed-forward, and understand the domains where those are appropriate and the domains where backprop

That's bad. There are gradient descent techniques that are feed-forward, and understand the domains where those are appropriate and the domains where backprop is appropriate is I think important, especially as you try to do things like mix machine learning techniques with other differentiable programming strategies.