3 ms·
Had the author offered an alternative? Namely, can E^2 be calculated in practice?
by tgb 10y ago
Had the author offered an alternative? Namely, can E^2 be calculated in practice?
- a_bonobo 10y agoI don't think you can calculate E^2 without the "true model", which you practically never have. The code uses the "true model" too: https://github.com/johnmyleswhite/r_squared/blob/master/utils.R#L63 https://github.com/johnmyleswhite/r_squared/blob/master/util... I guess the post is similar to Anscombe's quartet [1]: a warning not to blindly trust summary statistics. [1] https://en.wikipedia.org/wiki/Anscombe%27s_quartet https://en.wikipedia.org/wiki/Anscombe%27s_quartet
- static_noise 10y agoSo the solution the author proposes is both absolutely correct and absolutely useless in practice?
- lottin 10y agoHe's not proposing a solution as far as I can tell. He's simply using the E^2 statistic in order to illustrate the problem.
- yiyus 10y agoThe author says that E2 is "generally unmeasurable". Indeed, the true model will be unknown in most cases. In my opinion, it is not about using E2 or other alternative measure to R2, but about being aware of the significance and validity of R2 and not blindly assuming that a higher value always means a better fit.
- data_scientist 10y agoMy main issue with R^2 is that it is an artificial indicator. It has no business meaning. The best alternative is to build a metric that is related to your business issue. How many $ do you loss if you're wrong by 1%? Or by 10 units? Do your business loss is linear in your error? Is it symmetrical? These are the kind of questions you have to ask yourself. Defining a metric is hard, and there is no good shortcut.