2 ms·
Oh I see. You appear to be unfamiliar with how to evaluate failure rates. You appear to be under the mistaken assumption that your sample size needs to be in p
by Veserv 2y ago
Oh I see. You appear to be unfamiliar with how to evaluate failure rates.
You appear to be under the mistaken assumption that your sample size needs to be in proportion to the failure rate you hope to achieve to draw any conclusions. That is untrue. Your sample size only needs to be in proportion to the actual failure rate to draw conclusions.
If your self-driving car drives one mile with zero crashes, that provides almost no evidence to support a claim that your self-driving car has a crash rate of 1 per billion miles. However, if your self-driving car drives one mile and crashes 10 times, that provides tremendous amounts of evidence against the claim that your self-driving car has a crash rate of 1 per billion miles. This is despite the fact that both instances only have a sample of a single mile.
Proving, failure to prove, and disproving are not symmetrical at all. But do not worry, that is a common mistake made by people with no training in scientific or statistical analysis; just take this as a learning experience.
- fragmede 2y agoYou’re making a valid point about the asymmetry in proving vs. disproving failure rates, but the condescending tone is unnecessary. That might be acceptable behavior back on Reddit or Kiwi farms or wherever you come from, but we strive for a higher level of discourse here. I never said a sample size needs to match the target failure rate—only that proving an extremely low failure rate requires significant data. Yes, a high failure rate can be detected quickly, but that doesn’t mean a small sample is enough to confirm an ultra-low failure rate. Just because a car doesn’t crash in one mile doesn’t mean it won’t in the next billion. You’re oversimplifying the problem while assuming I don’t understand statistical inference.