I think just historically it's interesting. Every statistician was using OLS before computers because they could solve it with pen and paper, so when computers

I think just historically it's interesting. Every statistician was using OLS before computers because they could solve it with pen and paper, so when computers came out it was ported over. But with computers you can minimize any loss function. However it is useful to have a closed form solution because it guarantees you actually minimized it. Other strategies to minimize functions don't guarantee that but they're still extremely useful.