I don't think they've missed the fact that for linear regression there's an algebraic solution, but machine learners typically treat linear regression a simple

I don't think they've missed the fact that for linear regression there's an algebraic solution, but machine learners typically treat linear regression a simple special case, and as soon as want to go a bit beyond you have to go with an iterative numerical optimizer anyway so why bother with the special case solution. My main criticism on the article would be that the nitty gritty section only makes sense to a reader that has already done a linear algebra / multivariate calculus course. In which case they've likely already covered least squares in greater depth (including the exact solutions) than this article. So I don't really see the purpose of the math section, except maybe to signal that the writer has a descent understanding of the algorithmic detail.