4 ms·
Obligatory reminder that there's an inherent tradeoff between bias and variance in any ML architecture. Try to create a "highly regular" model that gives predi
by zozbot234 5y ago
Obligatory reminder that there's an inherent tradeoff between bias and variance in any ML architecture. Try to create a "highly regular" model that gives predictable, less noisy results even with low-quality data, you'll just end up strengthening some implied prior and introducing bias. Try to remove inbuilt bias, you'll just make the model more data-dependent, and noisier with any given amount of data. You just can't win.
- jstx1 5y agoAlso important to point out that the word bias has a very specific meaning in the context of statistics and machine learning and it isn't just a synonym for discrimination or something of that sort - https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff
- commandlinefan 5y ago... which seems not to be the definition the author of the linked article is using.
- rualca 5y ago> Obligatory reminder that there's an inherent tradeoff between bias and variance in any ML architecture. It should also be stressed that in this context "bias" is a kin to interpolating between data points where resolution in the training set is relatively low, thus leads to a simpler model that does not exactly match the test set. What this implies is that this "bias" does not reflect a preference, let alone an intention to favour one group over another. It just means that a combination of the test set being too sparse or with too few dimensions, the choice of model being too inflexible, and the problem domain having too much uncertainty at that resolution. Critics may single out the model, but the core of the problem actually lies on the data and the problem domain.