Differential privacy can definitely do much fancier stuff. A big challenge is that you like to play around with your data and try lots of different analyses and

Differential privacy can definitely do much fancier stuff. A big challenge is that you like to play around with your data and try lots of different analyses and regressions, and DP tends to clash with that approach. It's more natural to pick a DP algorithm, run it once, and live with the results. But anyway, one can definitely do stuff like "differentially private linear regression".