4 ms·
A really small datasets shouldn’t be used for statistics. But is it small, considering there are millions of flights registered?
by bjelkeman-again 3y ago
A really small datasets shouldn’t be used for statistics. But is it small, considering there are millions of flights registered?
- Someone1234 3y agoMillions of flights, but less than ten crashes for most, and it was pointing out that even some of those are incorrectly categorized. When you're dividing A by B to get ratio C, you cannot ignore that A is bad data because B is good, it still makes for a C that is useless. All this dataset provides is that with small enough samples it is too noisy to draw reliable conclusions.
- lolinder 3y agoYou need a larger dataset if you're dealing with very small probabilities. If you measured millions of events and only saw 1 anomalous event, you have enough data to say that the probability of such an anomaly is extremely rare, but you don't have enough data to compare it to other very rare events. More concretely: if you flip an unfair coin 1 million times and get a single heads, you know that the odds of getting that heads are extremely low, but you can't yet say what the odds are. It's possible that the odds are one in a million, but it's also possible that you got very lucky or very unlucky relative to the actual odds. You have to have a lot of data in both buckets before you can distinguish.