9 ms·

Reading through this article is like reading a description of the Monty-Hall problem. [0] It's as through the conclusion seems to defy common sense, yet is pro

by MR4D 2y ago

Reading through this article is like reading a description of the Monty-Hall problem. [0] It's as through the conclusion seems to defy common sense, yet is provable. [1] [0] - https://priceonomics.com/the-time-everyone-corrected-the-worlds-smartest/ https://priceonomics.com/the-time-everyone-corrected-the-wor... [1] - 2nd to the last paragraph: "The fact that you can achieve a constant average query time, regardless of the hash table’s fullness, was wholly unexpected — even to the authors themselves."

darknavi 2y ago
I always really enjoyed the Numb3rs lecture on Monty-Hall https://www.youtube.com/watch?v=P9WFKmLK0dc https://www.youtube.com/watch?v=P9WFKmLK0dc
ryao 2y ago
> “Our brains are just not wired to do probability problems very well, so I’m not surprised there were mistakes,” Stanford stats professor Persi Diaconis told a reporter, years ago. “[But] the strict argument would be that the question cannot be answered without knowing the motivation of the host.” This is wrong. Let’s label the goats A and B to simplify things (so we do not need to consider the positions of the doors). There are 3 cases: 1. You pick the right door. The other two doors have goats. The host may only choose a goat. Whether it is A or B does not matter. 2. You pick the door with goat A. The host may only choose goat B. 3. You pick the door with goat B. The host may only choose goat A. The host’s intentions are irrelevant as far as the probability is concerned (unless the host is allowed to tell the contestant which door is correct, but I am not aware of that ever being the case). 2/3 of the time, you pick the wrong door. In each of those cases, the remaining door is correct. The most strict argument is yet another statistics professor got basic statistics wrong.
- CrazyStat 2y ago
  I can assure you Diaconis didn’t get it wrong. > "The problem is not well-formed," Mr. Gardner said, "unless it makes clear that the host must always open an empty door and offer the switch. Otherwise, if the host is malevolent, he may open another door only when it's to his advantage to let the player switch, and the probability of being right by switching could be as low as zero." Mr. Gardner said the ambiguity could be eliminated if the host promised ahead of time to open another door and then offer a switch. The hosts’s intentions absolutely do matter, because the problem (as originally stated) doesn’t specify that the host always opens a door and offers a switch. Maybe he only offers a trade when you initially picked the good door.
  - thadt 2y ago
    > Maybe he only offers a trade when you initially picked the good door. That would be a rather convenient signal to the player.
    - CrazyStat 2y ago
      Indeed. But the hosts machinations can be arbitrarily more complex; maybe he offers the switch to contestants he finds attractive only when they’ve picked a goat, and contestants he finds unattractive when they’ve picked the car.
      - the_af 2y ago
        You're introducing bizarre ad hoc hypotheses. This works as a logic puzzle. Assuming the host offers different doors depending on contestant attractiveness makes absolutely no sense. It's a bizarre assumption. Maybe the goats can wander from door to door, or maybe there is no car, or maybe behind all of the doors there are tigers. Which would be absurd and unrelated to this puzzle.
        roenxi 2y ago
        The hosts' strategy is the core of the puzzle. The question is what information he has just conveyed to the player by opening the door and that depends entirely on his mental state. If he was always going to open a door then the player should switch. If he is opening a door only if the player has picked the car then they should not switch. If he has bizarre ad hoc motivations then the correct decision depends on bizarre ad hoc considerations. And, as CrazyStat has correctly pointed out, as stated in the linked article the hosts' strategy is an unknown. It could be bizarre. Although I'd still rather say vos Savant was correct in her reasoning; since the answer is interesting it seems fairer to blame the person posing the question for getting a detail wrong.
        ryao 2y ago
        The source material for the article says otherwise: > So let’s look at it again, remembering that the original answer defines certain conditions, the most significant of which is that the host always opens a losing door on purpose. (There’s no way he can always open a losing door by chance!) Anything else is a different question. https://web.archive.org/web/20130121183432/http://marilynvossavant.com/game-show-problem/ https://web.archive.org/web/20130121183432/http://marilynvos...
- ksenzee 2y ago
  Did you not read the entire article on how scads of intelligent people got this wrong? And the explanation of why they got it wrong? It’s like following a map that carefully routes you around a sinkhole, and then stepping right into the sinkhole.
  - ryao 2y ago
    I read the entire article. They all had defective reasoning. The player picked an option with a 1/3 chance of being right and a 2/3 chance of being wrong. The host’s action did not change that. However, the host’s action did make the remaining door have a 2/3 chance of being right and a 1/3 chance of being wrong.
    - Izkata 2y ago
      This is downvoted, but correct. Which door is right and which door is wrong doesn't get reshuffled when the host removes a wrong door, so even though there's only 2 doors left the chance isn't 50:50, it's still 33:67 - with the player having most likely chosen a wrong door.
      - alexey-salmin 2y ago
        It's not correct. P(A|B)=P(A) only if A and B are independent. Requiring independence in this case literally means "the host opens the door regardless of the player making the right or wrong choice first time". It's a core assumption in your calculations, without it the math is not correct.
- shkkmo 2y ago
  I don't think "motivation of the host" is a great way to accurately describe the issue that Diaconis is calling out, it is rather intended to be more intuitive. In a precise way, the reason the question is underspecified is because it doesn't say if the probability of the host offering you a chance to choose again is dependent on which choice you make. If the host offers the choice more twice as often when your pick right and when you pick wrong, then changing you pick is the incorrect choice. Now, colloquially, it can makes sense to assume the host always offers the choice, but practically, if we're looking at how to use statistics in a real world situation, that isn't a safe to always assume that probabilities are independent.
  - ryao 2y ago
    The question as stated does not permit such a choice by the host since if it were a choice, it had already been made. This is like being presented with a nearly completed game of chess, asked if the loser can lose in 1 move and then arguing that the answer is more nuanced because there might have been other moves taken that produced a different end games rather than the ones that produced this particular end game. We do not care about those other end games, since we are only considering this particular one.
    - shkkmo 2y ago
      > The question as stated does not permit such a choice by the host since if it were a choice, it had already been made. Whether the choice was already made by the host makes no difference, what matters is what information about the hidden state can be derived from that choice. Let's say the rules of the game are modified to sat that the host never offers a re-selection when you already have selected a door with a goat. Then if the host has offered you a re-selection you should definitely not take it because you already have the good prize. You know this because the re-selection offer provides information about what is behind the door you selected. In fact, any time your choice of door has amyy statistical effect on whether a re-selection is offered, then a re-selection offer (or lack) provides a small amount of information that modifies the expected value of choosing a new door. > This is like being presented with a nearly completed game of chess, asked if the loser can lose in 1 move and then arguing that the answer is more nuanced because there might have been other moves taken that produced a different end games It is absolutely nothing like that. That is not a question about statistics or probability.
- deleted 2y ago
  [deleted]
- tzs 2y ago
  This was the problem as stated in the Marilyn vos Savant column that started the controversy: > Suppose you’re on a game show, and you’re given the choice of three doors. Behind one door is a car, behind the others, goats. You pick a door, say #1, and the host, who knows what’s behind the doors, opens another door, say #3, which has a goat. He says to you, “Do you want to pick door #2?” Is it to your advantage to switch your choice of doors? Diaconis is in fact correct that given just that information the problem cannot be solved. What is missing is a statement that the host will always reveal a goat and always offer you a chance to switch doors. If the host can chose whether or not to make the offer then if you you happen to receive the offer when you are on the show you cannot say anything about whether or not switching is to your advantage. For instance suppose the show has given away a lot of cars earlier in the season and the producers ask the host to try to reduce the number of cars given away during the rest of the season. The host might then only offer switching when he knows the contestant has picked the car door. He will still open a goat door first because that's more dramatic. He just won't offer to let you switch before going on to open either your door or the remaining door.
- jeremysalwen 2y ago
  Following your arguments throughout this thread, I think the piece that is confusing you is the framing of the problem as a game-show host, which primes you to think of the host being "fair" by default. To understand how the framing might change how you interpret the problem, consider the following scenario: You are in a game of poker, and you have a flush with king high. Your opponent reveals all but one card from their hand, which shows they have 4 hearts, and they also reveal that their last card is an ace, but they don't reveal its suit. It's your turn to bet. Do you bet, or do you fold? Now you could treat this as a simple statistics problem -- there are four possible aces they could have in their hand, and only one is a heart, so only a 1/4 chance they will beat you. But is the solution to this problem that there is a 3/4 chance of winning the pot? In the problem text, we haven't specified under what conditions your opponent will reveal which cards in their hands. But somehow, by saying it's a game of poker makes you think that they probably are more likely to reveal their hand if they are bluffing, so the true probability is not 3/4. We are primed by this description of this person as your "opponent" to think about them making the decision adversarially. What if instead we say that that game of poker is part of a game show and your opponent is the host of the game show? Depending on the assumptions you make about your opponent's motivations, you must calculate the odds differently, and simply saying "3/4" is not unambiguously correct.
default-kramer 2y ago
Not this again... https://duckduckgo.com/?q=monty+hall+site%3Anews.ycombinator.com https://duckduckgo.com/?q=monty+hall+site%3Anews.ycombinator... I need to write a blog post or something convincing everyone we need to stop talking about the Monty Hall problem and replace it with a new problem with all the ambiguities removed. (Unless ambiguity is the point, then Monty Hall is fine.)
- the_af 2y ago
  There are no ambiguities in the Monty Hall problem. It's usually people who skim read and make assumptions about the challenge. No new problem is going to stop people from skim reading. For example, going by that ddg search, one result is making a fuss about not knowing whether Monty opens a door at random and happens to show a goat, or purposefully opens a door with a goat behind it. But we do know: it's always on purpose, Monty never opened a door with a car behind it, thus prematurely ending the bet. So there's no ambiguity. The problem is cool because the right answer doesn't seem intuitively right, even though it can be formally shown to be right.
  - chikere232 2y ago
    I imagine one reason people have a hard time with the monty hall problem is that they have learnt a rule that seems to fit but really doesn't. A person not trained at all in math might do better as they haven't learnt that rule. There's probably a name for that cognitive bias, but I don't know it.
  - default-kramer 2y ago
    > But we do know: it's always on purpose, Monty never opened a door with a car behind it We only know that if the problem tells us. Sometimes it doesn't. > There are no ambiguities in the Monty Hall problem The problem has been written up thousands of times. I'm sure that some writeups are sufficiently unambiguous, but many are not. For example, consider the two "variants" described by this comment https://news.ycombinator.com/item?id=8664550 https://news.ycombinator.com/item?id=8664550 > The host selects one of the doors with a goat from the remaining two doors, and opens it. > The host chooses one of the remaining two doors at random and opens it, showing a goat. This commenter was trying hard for semantic precision, and yet, I think if you encountered the first variant in isolation it would be perfectly reasonable to interpret it as "The host [randomly] selects one of the doors with a goat [although he might have selected the prize]" even though this is clearly not what the commenter was attempting. If you disagree, that only proves my point: this problem is prone to silly and wasteful semantic debate, rather than the interesting probability result it should be focused on.
sdenton4 2y ago
guys it's 2025, let's have a throw-down fight about the monty hall problem.
- IncreasePosts 2y ago
  Monty Hall is solved. We need to fight over whether .999 repeating == 1.000 repeating
  - azinman2 2y ago
    Forgive my mathematical ignorance, but why would it be? Isn’t it just asymptotically close but not actually equal? What does the 0’s repeating give you that 1 does not?
    - jon_richards 2y ago
      My favorite version: x = 0.999... x - x/10 = 0.9 x = 1
    - kevlened 2y ago
      This is likely oversimplified, but an intuitive approach is: 1/3 = 0.333 repeating 3/3 = 0.999 repeating 1 = 0.999 repeating
      - deleted 2y ago
        [deleted]
      - xlbuttplug2 2y ago
        yeah but 1/3 = 0.333 recurring is an equivalent problem to the parent
        IncreasePosts 2y ago
        X = .999r 10x= 9.999r 10x - x = 9.999r - .999r 9x = 9 x = 1 .999r = 1
    - 2y ago

by MR4D 2y ago

darknavi 2y ago

I always really enjoyed the Numb3rs lecture on Monty-Hall https://www.youtube.com/watch?v=P9WFKmLK0dc https://www.youtube.com/watch?v=P9WFKmLK0dc

ryao 2y ago

> “Our brains are just not wired to do probability problems very well, so I’m not surprised there were mistakes,” Stanford stats professor Persi Diaconis told a reporter, years ago. “[But] the strict argument would be that the question cannot be answered without knowing the motivation of the host.” This is wrong. Let’s label the goats A and B to simplify things (so we do not need to consider the positions of the doors). There are 3 cases: 1. You pick the right door. The other two doors have goats. The host may only choose a goat. Whether it is A or B does not matter. 2. You pick the door with goat A. The host may only choose goat B. 3. You pick the door with goat B. The host may only choose goat A. The host’s intentions are irrelevant as far as the probability is concerned (unless the host is allowed to tell the contestant which door is correct, but I am not aware of that ever being the case). 2/3 of the time, you pick the wrong door. In each of those cases, the remaining door is correct. The most strict argument is yet another statistics professor got basic statistics wrong.

CrazyStat 2y ago
I can assure you Diaconis didn’t get it wrong. > "The problem is not well-formed," Mr. Gardner said, "unless it makes clear that the host must always open an empty door and offer the switch. Otherwise, if the host is malevolent, he may open another door only when it's to his advantage to let the player switch, and the probability of being right by switching could be as low as zero." Mr. Gardner said the ambiguity could be eliminated if the host promised ahead of time to open another door and then offer a switch. The hosts’s intentions absolutely do matter, because the problem (as originally stated) doesn’t specify that the host always opens a door and offers a switch. Maybe he only offers a trade when you initially picked the good door.
- thadt 2y ago
  > Maybe he only offers a trade when you initially picked the good door. That would be a rather convenient signal to the player.
  - CrazyStat 2y ago
    Indeed. But the hosts machinations can be arbitrarily more complex; maybe he offers the switch to contestants he finds attractive only when they’ve picked a goat, and contestants he finds unattractive when they’ve picked the car.
    - the_af 2y ago
      You're introducing bizarre ad hoc hypotheses. This works as a logic puzzle. Assuming the host offers different doors depending on contestant attractiveness makes absolutely no sense. It's a bizarre assumption. Maybe the goats can wander from door to door, or maybe there is no car, or maybe behind all of the doors there are tigers. Which would be absurd and unrelated to this puzzle.
      - roenxi 2y ago
        The hosts' strategy is the core of the puzzle. The question is what information he has just conveyed to the player by opening the door and that depends entirely on his mental state. If he was always going to open a door then the player should switch. If he is opening a door only if the player has picked the car then they should not switch. If he has bizarre ad hoc motivations then the correct decision depends on bizarre ad hoc considerations. And, as CrazyStat has correctly pointed out, as stated in the linked article the hosts' strategy is an unknown. It could be bizarre. Although I'd still rather say vos Savant was correct in her reasoning; since the answer is interesting it seems fairer to blame the person posing the question for getting a detail wrong.
        ryao 2y ago
        The source material for the article says otherwise: > So let’s look at it again, remembering that the original answer defines certain conditions, the most significant of which is that the host always opens a losing door on purpose. (There’s no way he can always open a losing door by chance!) Anything else is a different question. https://web.archive.org/web/20130121183432/http://marilynvossavant.com/game-show-problem/ https://web.archive.org/web/20130121183432/http://marilynvos...
ksenzee 2y ago
Did you not read the entire article on how scads of intelligent people got this wrong? And the explanation of why they got it wrong? It’s like following a map that carefully routes you around a sinkhole, and then stepping right into the sinkhole.
- ryao 2y ago
  I read the entire article. They all had defective reasoning. The player picked an option with a 1/3 chance of being right and a 2/3 chance of being wrong. The host’s action did not change that. However, the host’s action did make the remaining door have a 2/3 chance of being right and a 1/3 chance of being wrong.
  - Izkata 2y ago
    This is downvoted, but correct. Which door is right and which door is wrong doesn't get reshuffled when the host removes a wrong door, so even though there's only 2 doors left the chance isn't 50:50, it's still 33:67 - with the player having most likely chosen a wrong door.
    - alexey-salmin 2y ago
      It's not correct. P(A|B)=P(A) only if A and B are independent. Requiring independence in this case literally means "the host opens the door regardless of the player making the right or wrong choice first time". It's a core assumption in your calculations, without it the math is not correct.
shkkmo 2y ago
I don't think "motivation of the host" is a great way to accurately describe the issue that Diaconis is calling out, it is rather intended to be more intuitive. In a precise way, the reason the question is underspecified is because it doesn't say if the probability of the host offering you a chance to choose again is dependent on which choice you make. If the host offers the choice more twice as often when your pick right and when you pick wrong, then changing you pick is the incorrect choice. Now, colloquially, it can makes sense to assume the host always offers the choice, but practically, if we're looking at how to use statistics in a real world situation, that isn't a safe to always assume that probabilities are independent.
- ryao 2y ago
  The question as stated does not permit such a choice by the host since if it were a choice, it had already been made. This is like being presented with a nearly completed game of chess, asked if the loser can lose in 1 move and then arguing that the answer is more nuanced because there might have been other moves taken that produced a different end games rather than the ones that produced this particular end game. We do not care about those other end games, since we are only considering this particular one.
  - shkkmo 2y ago
    > The question as stated does not permit such a choice by the host since if it were a choice, it had already been made. Whether the choice was already made by the host makes no difference, what matters is what information about the hidden state can be derived from that choice. Let's say the rules of the game are modified to sat that the host never offers a re-selection when you already have selected a door with a goat. Then if the host has offered you a re-selection you should definitely not take it because you already have the good prize. You know this because the re-selection offer provides information about what is behind the door you selected. In fact, any time your choice of door has amyy statistical effect on whether a re-selection is offered, then a re-selection offer (or lack) provides a small amount of information that modifies the expected value of choosing a new door. > This is like being presented with a nearly completed game of chess, asked if the loser can lose in 1 move and then arguing that the answer is more nuanced because there might have been other moves taken that produced a different end games It is absolutely nothing like that. That is not a question about statistics or probability.
deleted 2y ago
[deleted]
tzs 2y ago
This was the problem as stated in the Marilyn vos Savant column that started the controversy: > Suppose you’re on a game show, and you’re given the choice of three doors. Behind one door is a car, behind the others, goats. You pick a door, say #1, and the host, who knows what’s behind the doors, opens another door, say #3, which has a goat. He says to you, “Do you want to pick door #2?” Is it to your advantage to switch your choice of doors? Diaconis is in fact correct that given just that information the problem cannot be solved. What is missing is a statement that the host will always reveal a goat and always offer you a chance to switch doors. If the host can chose whether or not to make the offer then if you you happen to receive the offer when you are on the show you cannot say anything about whether or not switching is to your advantage. For instance suppose the show has given away a lot of cars earlier in the season and the producers ask the host to try to reduce the number of cars given away during the rest of the season. The host might then only offer switching when he knows the contestant has picked the car door. He will still open a goat door first because that's more dramatic. He just won't offer to let you switch before going on to open either your door or the remaining door.
jeremysalwen 2y ago
Following your arguments throughout this thread, I think the piece that is confusing you is the framing of the problem as a game-show host, which primes you to think of the host being "fair" by default. To understand how the framing might change how you interpret the problem, consider the following scenario: You are in a game of poker, and you have a flush with king high. Your opponent reveals all but one card from their hand, which shows they have 4 hearts, and they also reveal that their last card is an ace, but they don't reveal its suit. It's your turn to bet. Do you bet, or do you fold? Now you could treat this as a simple statistics problem -- there are four possible aces they could have in their hand, and only one is a heart, so only a 1/4 chance they will beat you. But is the solution to this problem that there is a 3/4 chance of winning the pot? In the problem text, we haven't specified under what conditions your opponent will reveal which cards in their hands. But somehow, by saying it's a game of poker makes you think that they probably are more likely to reveal their hand if they are bluffing, so the true probability is not 3/4. We are primed by this description of this person as your "opponent" to think about them making the decision adversarially. What if instead we say that that game of poker is part of a game show and your opponent is the host of the game show? Depending on the assumptions you make about your opponent's motivations, you must calculate the odds differently, and simply saying "3/4" is not unambiguously correct.

default-kramer 2y ago

Not this again... https://duckduckgo.com/?q=monty+hall+site%3Anews.ycombinator.com https://duckduckgo.com/?q=monty+hall+site%3Anews.ycombinator... I need to write a blog post or something convincing everyone we need to stop talking about the Monty Hall problem and replace it with a new problem with all the ambiguities removed. (Unless ambiguity is the point, then Monty Hall is fine.)

the_af 2y ago
There are no ambiguities in the Monty Hall problem. It's usually people who skim read and make assumptions about the challenge. No new problem is going to stop people from skim reading. For example, going by that ddg search, one result is making a fuss about not knowing whether Monty opens a door at random and happens to show a goat, or purposefully opens a door with a goat behind it. But we do know: it's always on purpose, Monty never opened a door with a car behind it, thus prematurely ending the bet. So there's no ambiguity. The problem is cool because the right answer doesn't seem intuitively right, even though it can be formally shown to be right.
- chikere232 2y ago
  I imagine one reason people have a hard time with the monty hall problem is that they have learnt a rule that seems to fit but really doesn't. A person not trained at all in math might do better as they haven't learnt that rule. There's probably a name for that cognitive bias, but I don't know it.
- default-kramer 2y ago
  > But we do know: it's always on purpose, Monty never opened a door with a car behind it We only know that if the problem tells us. Sometimes it doesn't. > There are no ambiguities in the Monty Hall problem The problem has been written up thousands of times. I'm sure that some writeups are sufficiently unambiguous, but many are not. For example, consider the two "variants" described by this comment https://news.ycombinator.com/item?id=8664550 https://news.ycombinator.com/item?id=8664550 > The host selects one of the doors with a goat from the remaining two doors, and opens it. > The host chooses one of the remaining two doors at random and opens it, showing a goat. This commenter was trying hard for semantic precision, and yet, I think if you encountered the first variant in isolation it would be perfectly reasonable to interpret it as "The host [randomly] selects one of the doors with a goat [although he might have selected the prize]" even though this is clearly not what the commenter was attempting. If you disagree, that only proves my point: this problem is prone to silly and wasteful semantic debate, rather than the interesting probability result it should be focused on.

sdenton4 2y ago

guys it's 2025, let's have a throw-down fight about the monty hall problem.

IncreasePosts 2y ago
Monty Hall is solved. We need to fight over whether .999 repeating == 1.000 repeating
- azinman2 2y ago
  Forgive my mathematical ignorance, but why would it be? Isn’t it just asymptotically close but not actually equal? What does the 0’s repeating give you that 1 does not?
  - jon_richards 2y ago
    My favorite version: x = 0.999... x - x/10 = 0.9 x = 1
  - kevlened 2y ago
    This is likely oversimplified, but an intuitive approach is: 1/3 = 0.333 repeating 3/3 = 0.999 repeating 1 = 0.999 repeating
    - deleted 2y ago
      [deleted]
    - xlbuttplug2 2y ago
      yeah but 1/3 = 0.333 recurring is an equivalent problem to the parent
      - IncreasePosts 2y ago
        X = .999r 10x= 9.999r 10x - x = 9.999r - .999r 9x = 9 x = 1 .999r = 1
  - 2y ago