3 ms·
There are many reasons to prefer simplicity, not just that prior to ubiquitous computing complicated models were intractable. Off the top of my head: A simpl
by clickok 9y ago
There are many reasons to prefer simplicity, not just that prior to ubiquitous computing
complicated models were intractable.
Off the top of my head:
A simple model can be "robust" to errors because the errors tend to cancel out.
More complex models are typically more sensitive to slight changes in the input (and therefore noise), so even if your model is exactly correct, small errors in the input can be amplified yielding large errors in the predictions.
Or it can be preferable because it tends to fail in certain characteristic ways.
For example, Newtonian mechanics breaks down at both the high and low mass/energy scales, but it's usually possible to determine whether you'll have to employ GR or QM (and perhaps despair if both are required simultaneously).
Also, it's much easier to test simple models (and typically also to train them if we're talking ML)[0].
Furthermore, if your first approximation is well-understood, then you can complicate it to address new data.
Simplicity has many virtues; just because we can employ more complicated methods now doesn't mean that we should, or that the sole reason previous generations of scholars opted for simple models is because they couldn't perform the calculations needed by more complex ones.
In general we can't expect to know the underlying truth with certainty, but maybe a question is so important and so well investigated that it merits a seriously convoluted answer.
For less well-studied or relatively unimportant problems, a model that is obviously wrong but easy to use is usually preferable.
It's called the bias-variance trade-off for a reason (although you should still try to get a good deal, which in this case means minimizing error or perhaps risk).
------
0. This point can be particularly subtle-- we generally have more computing power available than (quality) data.
I can try millions of variations on a given hypothesis for a small data set relatively quickly and find a handful of results with publishable p-values.
This is how you get replication crises.
Even if one of my hypotheses is invalidated, I can generate many more that are still consistent with the data.
Take string theory for example-- a naive approach to testing the various parameterizations/permutations of the basic idea would run out of time before the heat death of the universe, so absent theoretical work constraining the possibility space, it would be effectively unfalsifiable.