This is a valuable observation. I think the question, though, comes down to which of these two things should be mastered first or if they should be done in para

This is a valuable observation. I think the question, though, comes down to which of these two things should be mastered first or if they should be done in parallel. In general, the explosion of successes in DL in the last few years has been driven by AD and backprop rising to the occasion to make novel architectures explorable for the first time.