7 ms·
>At 95% certainty, you have 19 people saying “yes” and 1 person saying “no.” >At 99% certainty, you have 99 people saying “yes” and 1 person saying “no.” The
by cyborgx7 7y ago
>At 95% certainty, you have 19 people saying “yes” and 1 person saying “no.”
>At 99% certainty, you have 99 people saying “yes” and 1 person saying “no.”
The article was worth the read to me just for this part. I thought I had a pretty decent intuitive understanding of probabilities but this put things in new perspective for me.
The rest of the article is marketing gobbledygook to me, though.
- computerex 7y agoI didn't understand this part. Why is the author changing scales? Shouldn't it be that at: At 95% certainty, you have 95 people saying "yes" and 5 people saying "no"? So it's easier to make an apples to apples comparison? What point is the author trying to make with changing the scale?
- tempguy9999 7y agoThough obviously equivalent, I think as a percentage 95/5 and as a reduced fraction 19/1, they both have value in different ways to the non-mathematician. Giving 2 intuitive handles on a thing instead of one is helpful.
- meigwilym 7y agoEspecially as he goes on to state: > It feels like a difference of four people when, in reality, it’s a difference of 80. That’s a much bigger difference than we expect. I had to stop reading here.
- Marazan 7y agoBut that's the point. Ratios are really hard to reason about. Your mind sees the 5% going to 1% (or the 95% going to 99%) and thinks it's a small difference. When in actuality it's a big change.
- UserIsUnused 7y agoDepends on how you frame it. There is 80 of difference when you think about "how many users say yes for each No" So, what do you want to know, "yes for every 100" or "yes for every no" ? It matters.
- Ensorceled 7y agoWhen you thought about it some more, did you come back?
- arnvald 7y agoThe point is to show that with 95% for every 1 person saying "no" you have 19 people saying "yes". With 99% for every 1 person saying "no" you have 99 people saying "yes". The common denominator is one "no".
- ashelmire 7y agoThat’s not how this works though. The same number of people are landing on your site. You aren’t getting 80 more signups as suggested.
- kryogen1c 7y ago> That’s not how this works though. What youre struggling with is the counterintuitive nature of applied statistics vs pure math, and this is the point TFA was trying to make. > You aren’t getting 80 more signups as suggested TFA isnt saying you "get" more, but just illustrating how different 95 and 99% actually are. Its restating the potato paradox, linked elsewhere in the thread https://en.m.wikipedia.org/wiki/Potato_paradox https://en.m.wikipedia.org/wiki/Potato_paradox
- bobwaycott 7y agoGenuinely curious aside—you used TFA twice. It means “the fucking article” here, right? When you use TFA, your otherwise helpful comment reads as if it’s angry, exasperated, or some otherwise negative feeling that stands in opposition to the rest of your comment—which reads as a genuine attempt to be helpful and explain. Why do use TFA?
- lixtra 7y agoTFA can also stand for The Fine Article. By HN rules you should assume the best; in this case for you, he was talking about the fine article.
- bobwaycott 7y ago
- samastur 7y agoI'm not the author of the article, but I sometimes use the same approach because in my experience saying 5 people out of 100 does not have the same effect as 1 in 20 even though mathematically they are same. It provides a mental check if you are still ok with your original perception.
- dkersten 7y agoComparing A:B against C:D is much less intuitive for us puny humans than comparing A:B against C:B.
- mytailorisrich 7y agoHe is not changing scale. He is just illustrating what the probability means in terms of the size of the set you need to expect the event to occur once, which has a more practical value than a probability.
- Ensorceled 7y ago> So it's easier to make an apples to apples comparison? That's exactly the point. How many people said yes for every one who said no: 50% -> 1 yes for every no 95% -> 19 yeses for every no 99% -> 99 yeses for every no 99.9% -> 999 yeses for every no
- bigiain 7y agoIn the context of A/B Testing, if you've decided to stop the test at "95% significance" then you'll stop at the 19 yes, 1 no spot (or maybe 38:2). If you're testing for 99% significance, you won't stop unless you get to 99:1 (or more likely, you'll need to test way past that because "no" number 2 will arrive before "yes" number 99, so you'll need to test to 198:2 or 297:3 or further. This is why you should never stop an A/B test once you've "hit your statistical significance". Always choose the number of tests you'd need to prove the significance before you start, and let it run even if it's "obviously winning" (or losing).
- krumpet 7y agoCan you elaborate on this? When an A/B test has hit the desired significance, what is the value obtained in testing further?
- brianberns 7y agoHere's a little puzzle that really puts this in perspective: I put 100 lb. of potatoes in the sun. The potatoes were 99% water by weight. After drying a while, the potatoes were 98% water. How much did they weigh at that point? https://en.wikipedia.org/wiki/Potato_paradox https://en.wikipedia.org/wiki/Potato_paradox
- vishnugupta 7y agoIn my college days, we used a similar trick question to harass (or ragging as known in India) juniors. Beginning with $100 how much will you be left with if you lose 50% and gain 50%?
- maneesh 7y agoIt's $75, right? Don't leave us in suspense !
- cityzen 7y agoThat’s how they rag on juniors, they never tell them the answer!
- vishnugupta 7y agoHaha! It's indeed $75 ️.
- Normal_gaussian 7y agoThis is a technique often used in gambling games (poker apps minigames etc.) They offer a series of actions that let you lose or gain x% n many times in a row, with an equal chance of each. The fact that +x% doesn't cancel out a -x% compounds on the difference in lay person expectations to produce horrific odds for microtransactions.
- sundbry 7y ago51 pounds?
- ncphillips 7y agoIt’s actually nonsense. Certainty has nothing to do with how many people said yes or no, it’s about how many people are in the experiment. If you sample size of 2, and both like the new UI, then you can’t say you have 100% certainty. I.e. 3/4 people isn’t the same certainty as 75/100 I think the point he’s trying to make is it’s easier to get the statement “95% of people prefer X” from a study then to get “99% of people”. The first requires >20 people, the later >100. But he’s tying together the concepts of sample size, confidence, and the outcome of the experiment in a way hat doesn’t make sense
- Nimitz14 7y agoThat's not at all what he's saying. Maybe understand the article first before calling it nonsense. Kind of funny how all you people are making his point for him!
- 127 7y agoTo make it match with human brain, this would be more helpful indeed: >At 5% uncertainty >At 1% uncertainty
- benschulz 7y agoI actually thought it was the worst part of the article because it fails to put the statements into context. Let me quote from "0 And 1 Are Not Probabilities"[1]. > In probabilities, 0.9999 and 0.99999 seem to be only 0.00009 apart, so that 0.502 is much further away from 0.503 than 0.9999 is from 0.99999. To get to probability 1 from probability 0.99999, it seems like you should need to travel a distance of merely 0.00001. > But when you transform to odds ratios, 0.502 and 0.503 go to 1.008 and 1.012, and 0.9999 and 0.99999 go to 9,999 and 99,999. And when you transform to log odds, 0.502 and 0.503 go to 0.03 decibels and 0.05 decibels, but 0.9999 and 0.99999 go to 40 decibels and 50 decibels. > When you work in log odds, the distance between any two degrees of uncertainty equals the amount of evidence you would need to go from one to the other. That is, the log odds gives us a natural measure of spacing among degrees of confidence. [1]: https://www.lesswrong.com/posts/QGkYCwyC7wTDyt3yT/0-and-1-are-not-probabilities https://www.lesswrong.com/posts/QGkYCwyC7wTDyt3yT/0-and-1-ar... Edit: formatting
- mattkrause 7y agoThe quoted part is sensible, but the title is goofy. Zero and one have to be probabilities. A lot of the underlying math doesn't work out otherwise, but you can also believe something always/never happens (if you're a Bayesian), or observe the same thing happening over and over again (for the frequentists). Your example shows that (non-linear) transformations aren't linear, and that our gut feels about likelihood aren't either, which is true enough. However, you wouldn't say "101 ˚C isn't a temperature" because it takes much less energy to heat a wet thing from 97 to 99˚ than it does to bring it from 99 to 101˚ because of the phase change.
- marcosdumay 7y agoYou can not ever discover in a statistical test that some probability is 0 or 1. Those are impossible values. You can get arbitrarily close to them, but you can never reach those values. Of course, they are valid values for the set of probabilities, just like 0K is a valid temperature, and c is a valid speed for massive things. You just will never see any of those.