4 ms·
"simplicity is a guide to truth" is not a summary of Occam's Razor. It says that _all things being equal_ the simplest explanation is generally the best. If you
by boredguy8 18y ago
"simplicity is a guide to truth" is not a summary of Occam's Razor. It says that _all things being equal_ the simplest explanation is generally the best. If you have an elegant solution that doesn't account for everything, there's a problem[1]. That problem is either in the theory or in the data, but until that problem is resolved, you're not helping advance much of anything.
So, for instance, we can still (I am told by credible astrophysicists) posit a geocentric cosmology if we're willing to adapt a rather complex system. But rather than think this is true, we have an alternative hypothesis that accounts for the same data without introducing a plethora of adjustments.
But of course, gravity _alone_ doesn't count for the data, hence Einstein.
Gravity is a "simpler" explanation, but it doesn't _account_ for our observations, so it get axed.
Scientists doing philosophy without training is almost always a very scary thing to witness.
[1] Re: we don't make curves that fit every data point: those curves are not -predictive- (i.e. you can perfectly account for -known- data, but you blow when you try to predict -future- data) and hence don't provide a reliable account of phenomenon. So they're rejected on that basis.
- neilc 18y agoSo given an infinite set of possible theories to explain a data set, which theory should be preferred? Simple accuracy over the known data is frequently not sufficient, as you point out, because of overfitting. Given noisy data (which is almost always a given), you can't just use maximal accuracy as the sole criteria for which theory to prefer. One way forward is to combine accuracy with some measure of the complexity of the theory (e.g. minimum description length), which has much of the flavor of Occam's Razor.