2 ms·
There is now a new definition for E^2 (one minus the ratio of the R^2 of the model and the "true" model) which doesn't solve the most obvious issue: getting neg
by kgwgk 10y ago
There is now a new definition for E^2 (one minus the ratio of the R^2 of the model and the "true" model) which doesn't solve the most obvious issue: getting negative values for a measure called "something squared". The values of E^2 in the first example are now -0.13 for the linear model and -0.30 for the quadratic model. In the second example, they are 0.24 and 0.10 respectively.
The graphical representation is a bit misleading. Leaving aside the fact that in the first example MSE_T is between MSE_M and MSE_C, this drawing make E^2 and R^2 seem more complementary than they really are. E^2 is the length of the blue bar as a fraction of the total length (blue+orange). R^2, however, is the length of the orange bar as a fraction of the distance from the end of the bar to the origin (not shown in the chart).
Edit: there is a new addition to the post, re-expressing E^2 in terms of a mean/variance decomposition. It should be kept in mind that the derivation presented is only asymptotically correct. In a small sample, the cross term does not vanish and the variance of the observations around the "true" value is not exactly sigma^2. In the second example, E^2 calculated using this new definition is quite similar (0.2373 and 0.0991 for the linear and quadratic models, compared to the previous values of 0.2382 and 0.0994). In the first example, however, the values we get from the new definition are far from the previous values: 0.0646 vs -0.129 for the linear model, 0.1528 vs -0.297 for the quadratic model.
Edit2: changed "approximation" to "new definition", "good" to "similar" and "exact" to "previous" in the previous paragraph. I'm not sure if he was suggesting to use this formula to calculate E^2 instead of the previous one. Anyway, it doesn't matter because this is not something that can be calculated at all unless the "true model" is known.