We had to revisit linear regression multiple times in different courses for my undergrad classes. It's fascinating that optimality is provable using statistics

We had to revisit linear regression multiple times in different courses for my undergrad classes. It's fascinating that optimality is provable using statistics and probability theory, although given assumptions hold of course. For my cs phd I looked mostly at regression problems using deep learning models. I didn't look at this specifically but I still think it would be neat if there is some way to translate the rigid proofs and theorems for classical linear models to deep regression models.