4 ms·
Junk data can’t improve accuracy, that’s the flaw with Bayesian thinking. If I hand you a normal looking coin and ask what’s the odds for heads, tails, or edge
by Retric 7y ago
Junk data can’t improve accuracy, that’s the flaw with Bayesian thinking. If I hand you a normal looking coin and ask what’s the odds for heads, tails, or edge to +\- 0.0001%, the absolute minimal trials is going to be based on zero priors.
It’s valuable when you have a combination of useful prior knowledge and insufficient data to review. But, having useful prior knowledge can be difficult to distinguish from useless prior knowledge. Incorrect belief that something is safe means more people are harmed before you update that belief.
This means you can’t say if Bayesian think is a net positive or a net negative abstractly.
- joshuamorton 7y agoI'm confused, in cases where you have truly no prior knowledge, such as your thought experiment, a bayesian can reasonably select a uniform prior. But often in real world statistical experiments, we have priors we can believe in.
- Retric 7y agoYou likely have fairly accurate priors about that coin flip I just mentioned. They are just not accurate enough to be useful. The math for designing sample sizes may be unfamiliar, so you can just try running the same simulation a few times. Then try a few priors on each data set and compare. This is effectively pure math so there is nothing to disagree or get upset about. What’s up for debate is how accurate your priors are in the real world.
- nightski 7y agoYour comments apply to statistics and machine learning as a whole, not just Bayes statistics. At the end of the day all you have is your prior knowledge and what you learn from the data. If both of those are suspect, it does not matter what method you use. I just don't see how this can be considered a problem specific to Bayesian statistics.
- Retric 7y agoThat’s a qualitative assessment of a quantitative process. Measuring how each system works in specific situations allows you to compare them, which considering the subject matter seems appropriate.
- nightski 7y agoAre you measuring the process or the inputs? Because in your post the inputs are what you criticized (the prior and the data), not the process itself. Every machine learning algorithm takes these inputs. So I was just saying that it seems unfair to penalize Bayesian statistics for this and not any other method.