* understanding where loss functions and regularization terms come from allows you to reason about the right choice for your problem and possibly to extend/twea

* understanding where loss functions and regularization terms come from allows you to reason about the right choice for your problem and possibly to extend/tweak them to suit your needs. Are you working directly with probabilities? Then maybe you don’t want to use L2 regularization (Gaussian prior) but a beta prior or something with the right support. Are you modeling a Poisson rate (e.g. how many people buy my product for every dollar I spend on advertising), then use a Poisson likelihood (loss function would be negative log likelihood). * do you want to have your NN model your uncertainty as well as the mean? How do you incorporate that into the loss function? Hint: loss = (yhat- y)^2/sigma_hat^2 is missing a term but you wouldn’t know that if you don’t come from Bayesian stats. * the rabbit hole goes as deep as you want. Understanding Bayesian stats removes a lot of the “ad hoc” and intuitive guesswork that goes into ML when you don’t have a solid statistical foundation for what you’re doing.