There's a bit of confusion here, because backpropagation is only used to train the network. After your network is trained, you don't want to change it, so you w

There's a bit of confusion here, because backpropagation is only used to train the network. After your network is trained, you don't want to change it, so you wouldn't run backpropagation in production. As for making training faster, second-order methods don't generally work as well as backpropagation for large networks. Some notes from Yann LeCunn! http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf