I think it would have helped if the people writing that paper had not confused the issue by introducing a new name for something that is a well known thing in o

I think it would have helped if the people writing that paper had not confused the issue by introducing a new name for something that is a well known thing in optimal control and had been invented even before neural networks, namely adjoint sensitivity analysis. There even appear multilayer networks of switching components in Pontryagin's book on the subject.