2 ms·

i wouldn't dare to say its a 'serious drawback'. as soon as you add ridge to regression (something very common for statisticians to do), it's also no longer sc

i wouldn't dare to say its a 'serious drawback'. as soon as you add ridge to regression (something very common for statisticians to do), it's also no longer scale/unit-invariant. Edit: this is also not just true for ridge, but lasso as well

em500 6y ago
Personally I consider the lack of scale-invariance one of the main drawbacks of most common regularizers too. Again, not a big deal if all you're after are y-hat, a bit more concerning if you're interested in beta-hat.
- autokad 6y ago
  i try not to stick too religiously to coefficient interpretation unless its a very simple and known problem. one missing variable could change the coefficients sign.
  - zwaps 6y ago
    Depends on whether the missing variable is correlated with the beta hats you care about ;-)