4 ms·

Thanks for the kind words! I actually agree that L1 error is often more practical, and didn't mean the post to come off as prescriptive. (In fact, part of my mo

by benkuhn 11y ago

Thanks for the kind words! I actually agree that L1 error is often more practical, and didn't mean the post to come off as prescriptive. (In fact, part of my motivation for writing it was that L1 error seemed to have much better practical properties, and I was curious why people used L2 despite this!)

repsilat 11y ago
It's a good post. If you're interested, Nassim Taleb (shudders) wrote a counterpoint for Edge.org that seems to have disappeared. It exists on the Internet Archive, though: https://web.archive.org/web/20140606195420/http://edge.org/response-detail/25401 https://web.archive.org/web/20140606195420/http://edge.org/r... Well, that's not quite accurate -- Taleb doesn't argue that we should do away with squared error, he says we should do away with the standard deviation in favour of expected deviation from the mean. (Hrm, I'm going back and forth on whether there's a substantive difference between the concepts of variance/standard deviation as measures of a probability distribution and squared/RMS error measures... Obviously they coincide when the "prediction" is the mean of the data, but I don't think that's terribly convincing. I think it'd be perfectly reasonable to use squared error in your model fit and parametrise your Gaussian distributions by their average absolute deviation from the mean.) Discussed on HN https://news.ycombinator.com/item?id=7064435 https://news.ycombinator.com/item?id=7064435
- stdbrouw 11y ago
  When talking about deviation, you're describing the spread of a distribution. It's perfectly fine to describe your data using mean (or median) average deviation, but then fit a model on top of that minimizes the sum of squared errors. Taleb's point is that squared deviations make for a lousy descriptive statistic as it leads you to underestimate the average distance of observations from the mean.

by benkuhn 11y ago

repsilat 11y ago

It's a good post. If you're interested, Nassim Taleb (shudders) wrote a counterpoint for Edge.org that seems to have disappeared. It exists on the Internet Archive, though: https://web.archive.org/web/20140606195420/http://edge.org/response-detail/25401 https://web.archive.org/web/20140606195420/http://edge.org/r... Well, that's not quite accurate -- Taleb doesn't argue that we should do away with squared error, he says we should do away with the standard deviation in favour of expected deviation from the mean. (Hrm, I'm going back and forth on whether there's a substantive difference between the concepts of variance/standard deviation as measures of a probability distribution and squared/RMS error measures... Obviously they coincide when the "prediction" is the mean of the data, but I don't think that's terribly convincing. I think it'd be perfectly reasonable to use squared error in your model fit and parametrise your Gaussian distributions by their average absolute deviation from the mean.) Discussed on HN https://news.ycombinator.com/item?id=7064435 https://news.ycombinator.com/item?id=7064435

stdbrouw 11y ago
When talking about deviation, you're describing the spread of a distribution. It's perfectly fine to describe your data using mean (or median) average deviation, but then fit a model on top of that minimizes the sum of squared errors. Taleb's point is that squared deviations make for a lousy descriptive statistic as it leads you to underestimate the average distance of observations from the mean.