10 ms·
Paradoxes of probability and other statistical strangeness
- pmoriarty 10y agoMy favorite probability paradox has always been the Monty Hall problem[1]: Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice? [1] - https://en.wikipedia.org/wiki/Monty_Hall_problem https://en.wikipedia.org/wiki/Monty_Hall_problem
- haddr 10y agoI love this one, but I guess it must be less common to have such a problem in the real life. Or I'm wrong?
- justinpombrio 10y agoAs the Wikipedia article points out, that problem as stated is ambiguous. You need to know under what circumstances the host would have opened that door. The standard assumption is that the host always reveals a door with a goat behind it. However, another possibility is that: (i) if both of the remaining doors contain goats then the host reveals one of them; but (ii) if one of the remaining doors contains the car then the host doesn't open a door at all.
- kharms 10y agoThat variation doesn't make much sense as a paradox. If (i) you know that you should stay 100% success. If (ii) you know you should switch, 50% success.
- deleted 10y ago[deleted]
- casta 10y agoIf you see this problem as a special case of another problem the answer becomes pretty intuitive. Suppose there are N (let's pick 1000) doors: Behind one door is a car; behind the others N-1 (999), goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens N-2 (998) doors, excluding No. 1, which have have goats. Now there are only two doors closed, the one you picked No. 1, and one that was part of N-1 (999) doors. He then says to you, "Do you want to pick door No. 1 or the other one?" Is it to your advantage to switch your choice?
- std_throwaway 10y agoSuppose there's two doors. You can't decide. The host opens one. There is no goat behind that one. Now you know for certain.
- p1esk 10y agoIt did not get any more intuitive for me.
- bmay 10y agoSay there are a trillion doors and the host closes 999,999,999,998 of them. Which scenario is more plausible: 1) Your 1-in-a-trillion chance original choice just happened to be correct 2) The omniscient host picked the correct door
- nerdponx 10y agoAgreed. But here's the intuition: because the host never opens a door with a car behind it, the fact that he didn't open a particular door suggests that the door in question is the winner. Compare that to the initial door, which was selected blindly.
- p1esk 10y agoWhy doesn't it also suggest that my door is now more likely to be the winner?
- YCode 10y agoThis one is uniquely impressive in that often when first described to someone they will be all but certain it doesn't matter. Many I told were just about ready to argue their view to death and asserted the switch your choice conclusion is wrong despite them having no formal learning on the subject. This and the birthday problem effectively illustrate how intuitively bad we are at probability and statistics in general.
- j7ake 10y agoProperly applying Bayesian inference gives you the correct answer. Also the answer becomes intuitive once this scales to more than 3 doors (someone below already mentioned).
- denzil_correa 10y agoThe best explanation on the solution for this problem is from Numberphile. https://www.youtube.com/watch?v=4Lb-6rxZxx0 https://www.youtube.com/watch?v=4Lb-6rxZxx0
- pmoriarty 10y agoOne of the things that interests me about this paradox is that when you're left with just two doors, the odds of either of them being the one with the car is 50%, so what does it matter if the one you happened to pick was picked before the host opened some other doors? How could that possibly affect the probability? It seems like choosing to switch would be admitting that the host's action has some kind of reverse-causality. The other thing I find interesting about this puzzle is wondering whether the people who find the arguments in favor of switching convincing are actually reasoning better than I am. If so, how are they doing this? What is it that makes those (allegedly superior) arguments convincing to them? The thing that kind of convinced me that the pro-switch arguments must be correct is that people have run simulations of this problem on computers and the switching strategy turns out to win in line with the pro-switch argument probabilities. But the tricky thing about probabilities and randomness is that such successes could be "just a matter of luck", as it were. Maybe they were wrong, but just got lucky. It's improbable, but still possible. Only mathematical proof independent of experimentation should be rock-solid correct. But mathematical proof seems to be a matter of being convincing enough to mathematicians, and no so independent after all, and that seems disappointing somehow. There's a feeling that mathematical proof should be above mere argumentative skill and the audience's susceptibility to being convinced.
- rndmio 10y ago> One of the things that interests me about this paradox is that when you're left with just two doors, the odds of either of them being the one with the car is 50%. Well, no, that's the point of this, it's not 50%. Before you pick there is a 1/3 chance of the car being behind any of the doors, so there is a 2/3 chance of it cumulatively being behind the other two. But there is only one car, so the set of doors you didn't pick must include at least one non-car. The host knows the location of the car so can always open a non-car door from this set. But still as a set they have a 2/3 chance of containing the car. By swapping you can effectively think of it as choosing both other doors. An easy way to get your head around this is increase the number of doors, say to 100. You pick 1 of those 100, the host opens 98 other doors, do you swap?
- kevhito 10y agoAlternate reality rules of the game are: nobody speaks, you pick any two doors you like, and you win the prize behind both of them. The odds of wining this game is obviously 2/3. Monty will gesture at you to point to a door. You point at the one you don't like. Monty will then show you one of your prizes behind one of your two chosen doors. Monty will then gesture for you to point to a door. You point at the remaining one you liked. Monty then gives you your second prize behind that door. Notice that all of the externally visible actions are identical to the original game.
- babyrainbow 10y ago>You pick a door, say No. 1, and the host, who knows what's behind the doors... I don't get this. Why does't the player insist that they open door no 1. Or is it part of the game that the host will open a different door other than the one the player picked... If the host knows what is behind the doors, and you are only given two chances, and the host can open a different door from what you picked, isn't your chance of winning zero? I don't get this...
- Retra 10y agoThe host's job is to create suspense by opening a door and offering a chance to switch. If they opened the door you picked first, there would be no suspense, and if they didn't know what was behind the door, they could show you that you lose, which also removes suspense. Turns out the host just ends up giving you more information that you can use to make a better decision.
- babyrainbow 10y agoCan you please explain this.. >If the host knows what is behind the doors, and you are only given two chances, and the host can open a different door from what you picked, isn't your chance of winning zero?
- z3t4 10y agoWhen we discussed this in school I quickly wrote a program that simulated it ...
- danbruc 10y agoNo need to look at fancy paradoxes, just think about the following. What does it mean that tossing a fair coin has a 50 % probability of showing heads? If you think you know the answer, you are probably wrong. EDIT: Instead of just voting this down, try to give an answer. If you think it is easy, you have not thought about it careful enough.
- yellowstuff 10y agoThe frequentist answer is that you could get the proportion of tosses that come up heads arbitrarily close to 50% in a large enough sample. The Bayesian answer is that given all the evidence available to me I would be willing to bet $1 to win $1 if the next toss comes up heads. (Since I work in finance I'll add that this assumes I'm risk neutral, so losing $1 is exactly as bad for me as winning $1 is good.) Except for the risk-neutrality detail this is all Probability 101, right? Or are you thinking of something else?
- danbruc 10y agoThat the relative frequency converges to 50 % is obviously not true. You would have to be exceptionally lucky, but you could get heads, heads, tails repeated for ever and therefore the relative frequency of heads would fluctuate increasingly tiny amounts around 66.(6) %. This of course has probability zero, but it is not impossible. And there are of course many other sequences of outcomes for which the relative frequency does not converge to 50 %. So at best you could say that the relative frequency converges to 50 % with high probability, but now you have a circular definition because you make use of probabilities while defining probabilities. The Bayesian view is problematic for several reasons. Why do I need someone with believes about the coin, we are talking about intrinsic properties of tossing a coin after all. And if that is not enough, we also throw some betting in. Tossing a coin does certainly not depends on the invention of money and gambling, at least ignoring that coins are usually money. Last but not least you have to explain where your believe about a 50 % probability for heads comes from and how it is justified. I could certainly believe that the probability for heads is 25 %, that would not be a good believe.
- naftaliharris 10y agoA less popular but perhaps more influential phenomenon is Stein's Paradox [1]. Here's a provocative example often given to illustrate it: Say you have a baseball player, soccer player, and football player, and you wish to estimate the true mean number of home runs, goals, and touchdowns each scores per year. If you have their last ten seasons worth of data for each, then the obvious thing to do, for each player, is to estimate the true yearly mean score for each player by their average yearly score from the last ten years. (E.g., the baseball player hits an average of 20 home runs each year, so let's estimate their true mean yearly home runs by 20). Stein's Paradox says that you can actually do a lot better than this. Even more crazy, the James-Stein Estimator which does this actually uses data about the football player and soccer player to make predictions about the baseball player, (and vice-versa). This is deeply unintuitive to most people since the players aren't related to each other at all. The phenomenon only holds with at least three players; it doesn't work for two. (More generally, Stein's Paradox is the fact that if you have p >= 3 independent Gaussians with a known variance, you can do better in estimating their p-dimensional mean than just using their sample means). I've spent a bunch of time trying to understand why this actually works [2]; to be honest I still don't deeply understand. But nonetheless the consensus is that the same shrinkage phenomenon is what causes improved performance for a variety of high-dimensional estimators, (lasso or ridge regression, e.g.), making the paradox very very influential. [1] https://en.wikipedia.org/wiki/James%E2%80%93Stein_estimator https://en.wikipedia.org/wiki/James%E2%80%93Stein_estimator [2] https://www.naftaliharris.com/blog/steinviz/ https://www.naftaliharris.com/blog/steinviz/
- projectorlochsa 10y agoWell this does make a little bit of sense if after estimating things get worse for a particular case. If MSEs were 5, 5, 100, and after they are 10, 10, 80, the total got better but the prediction for the other two got worse.
- mafuyu 10y agoYeah, reading through the Wikipedia, it looks like this reduces the total error of the combined estimator, but the error compared to an estimator of any one single parameter could be worse. So you can combine whatever crazy parameters you want, but it's only really relevant when you have things that are associated with each other somehow, and you want to reduce the total error of estimating all of them.
- haddr 10y agoI think there is a whole class of statistical "strangeness" with using p values for hypothesis testing. For instance, p = 0.05 means that we have ~30% chance that our hypothesis is a false positive [1], which is far from what intuition tells us. [1] http://www.nature.com/news/scientific-method-statistical-errors-1.14700 http://www.nature.com/news/scientific-method-statistical-err...
- Houshalter 10y agoBy far the most unintuitive paradox for me personally is the one presented here: https://youtu.be/go3xtDdsNQM?t=3m27s https://youtu.be/go3xtDdsNQM?t=3m27s "Mr. Jones has 2 children. What is the probability he has a girl if he has a boy born on Tuesday?" Somehow knowing the day of the week the boy was born changes the result. It's completely bizarre.
- prolways 10y agoIs there a more rigorous explanation of why they count the probability space how they do? Watching that video I feel like the ordering of the kids and the striking of one of the "b2b2" entries seems wrong to me. If we care which kid was first... which doesn't seem to matter... then the first b2b2 and the second b2b2 seem like they're different and shouldn't "cancel." Then again... it took me multiple explanations to understand the Monty Hall problem.
- kevinwang 10y agoYeah, it definitely seems wrong to me. B2B2 has twice the probability of any other event listed in that table. edit: I guess it's more nuanced than that. The explanation and interpretation sections on this blog post [1] and on wikipedia talk about the controversy. [1]: https://jakubmarian.com/the-day-of-the-week-boy-or-girl-paradox-explained/ https://jakubmarian.com/the-day-of-the-week-boy-or-girl-para... [2]: https://en.wikipedia.org/wiki/Boy_or_Girl_paradox https://en.wikipedia.org/wiki/Boy_or_Girl_paradox
- Houshalter 10y agoEven if that's the case it still doesn't make sense. That changes the probability to 50% girl instead of 52%. But it should be 2/3rds.
- kevinwang 10y agodammit, you're right.
- 10y ago
- deleted 10y ago[deleted]
- mrcactu5 10y agomy favorite data science paradox is the "curse of dimensionality" https://en.wikipedia.org/wiki/Curse_of_dimensionality https://en.wikipedia.org/wiki/Curse_of_dimensionality
- z3t4 10y agoIf you throw a six sided dice two times, there's 1/6*1/6=~3% chance to hit six both times. But if you throw one six, there's now ~17% change to hit six again ...
- prmph 10y agoHere is another seemingly basic question that leads down the rabbit hole: What does it mean to say two things are the same?