I understand the chain-rule. I understand using the derivative to minimize error. What I want to know is, what math is applied where to architect better models?

I understand the chain-rule. I understand using the derivative to minimize error. What I want to know is, what math is applied where to architect better models? Preferably with examples.