The frameworks can abstract away much of the math. You won't have to actually take the derivatives yourself, but understanding how they work actually matters to

The frameworks can abstract away much of the math. You won't have to actually take the derivatives yourself, but understanding how they work actually matters to network design. It's much harder to get meaningful insights out of papers for designing better networks if you can't read the math in them. And Neural Networks is an area where you will read research papers as part of your job in order to do it well, even in industry. When is RELU better than sigmoid? When should you use a bottleneck layer? When do you want fewer larger layers or more smaller layers? When is it appropriate to add some manual features in your problem? Or even when you get an error message back from your framework when is the error in your network code an when is it in your data design? It's very hard to develop good answers to the questions that come up if you don't have the math to read papers, don't have the math to double check expressions in the error code, or have problems going back and forth between expressions with Matrix and vectors and expressions in summation notation. There are a lot of people who can build a neural network when the architecture is done for them and nothing errors (using no math or stats), but training someone to make decisions on the architecture and handle the errors IS the job.