3 ms·

Isn't the first example extremely bullshit? > A third party asks them to "assume that a coin is fair, i.e., has an equal probability of coming up heads or tail

Isn't the first example extremely bullshit? > A third party asks them to "assume that a coin is fair, i.e., has an equal probability of coming up heads or tails when flipped. I flip it ninety-nine times and get heads each time. What are the odds of my getting tails on my next throw?" Istm that Fat Tony here gets the right answer to the wrong question. Whether we should assume this is not part of the question. That this is a useful thing to ask IRL doesn't change the fact that if you posed this to me IRL and smugly quoted the ludic fallacy at me for correctly answering your question I'd be strongly tempted to deck you. https://xkcd.com/169/ https://xkcd.com/169/ comes to mind.

baxtr 6y ago
That’s actually exactly what I was going to say. Of course you should update your priors continuously and thus evolve your model. A model is never perfect but it’s the best approximation to reality we have. In the end, Fat Tony has a model in his head, too.
- FeepingCreature 6y ago
  Right - "assume at the start that the coin is fair - you are free to update the assumption as you go" is a different thing than they said. Usually "assume that" assumptions are meant to hold throughout the question. As Randall says, "communicating badly and then acting smug when you're misunderstood is not cleverness."
  - marvin 6y ago
    I think the central point here is that «assume that» is fine when working with a math problem, but can kill you in real life. If you’ve just lost two billion dollars, or killed 200,000 people in a preventable war, no one cares that the assumptions were reasonable.
    - FeepingCreature 6y ago
      Right, but the study has nothing to do with this, because the question was not in real life, but about a fictional scenario. "Assume you jump off a bridge. Would you be dead?" "No, because I'd change my mind. What, you think I'd jump off a bridge because someone tells me to?" "No, but I'd rather expect you'd assume it because I tell you, in the context of a hypothetical."
      - basdftrewq 6y ago
        I think the point would be that expertise in those fictional scenarios doesn't correlate with expertise in real scenarios, and many experts only have been trained in fictional scenarios.
      - Ambol 6y ago
        But this is experts we are talking about. There's supposed to be some degree of translation between models and real life. The point of the fallacy is that this translation can be very debatable and restricted in many fields, and that a good observer with decent intuition can beat those models easily. If this was about a purely intellectual roleplaying game with internal rules there would be no discussion in the first place.
        FeepingCreature 6y ago
        > If this was about a purely intellectual roleplaying game with internal rules there would be no discussion in the first place. Which is what makes the discussion bullshit. Because if you're starting your experiment with "assume that", that's what you're signalling you're doing.
  - basdftrewq 6y ago
    The point is that in real life problems it's more important to throw away ridiculous "assume that"s than having very neat models.
- gwd 6y ago
  I think the Ludic Fallacy would be better described as, "A tendency to be overconfident that you know what the game is." If you read the Black Swan, what Taleb is mainly railing against are people who are completely confident that because they have a model, that it must therefore be correct, and refuse to remember all the caveats, and what could happen if their model is wrong.
l0b0 6y ago
Yeah, the only thing it proves is that when a scientist says "assume X" to another scientist the common understanding is that it means "in a situation where it is given that X is true, no matter how ridiculous that might be in real life." As for more reasons why this is bullshit, see the gambler's fallacy and the Monty Hall problem – intuition is often terrible at arriving at the right prediction.
TheOtherHobbes 6y ago
But the assumption is the point. When you have enough data to suggest your model is wrong, you should throw away the model and the assumptions it's based on, not the data. The fact that your model can be justified as technically correct is irrelevant. When the data doesn't fit the model any more, the supposed absolute correctness of the textbook model blocks your ability to understand what's happening. This isn't even about games, but about whether it's possible to be open-minded enough to consider that simple textbook "solutions" are not guaranteed to be a good fit for real problems - especially if you don't understand their limitations or the assumptions they're based on. And it doesn't just apply to statistics.
- deleted 6y ago
  [deleted]
- Hitton 6y ago
  I remember reading some kind of fantasy anthology of Sherlock Holmes pastiches. In one story there was interesting twist on the famous quote "Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.". It was changed to the effect that "once whatever remains is ridiculously improbable, you should probably reconsider what is actually impossible".
- leereeves 6y ago
  But the example is poorly communicated. It's not clear whether the entire quoted portion is: A) a hypothetical introduced by the idiom "Assume that..." (which introduces a statement of fact in the hypothetical situation), and we are discussing a scenario in which the coin is in fact fair, in which case Dr. John is correct and Fat Tony is committing the gambler's fallacy, or B) a scenario in which a third-party tells someone that the coin is fair, but that is simply an assumption that should be updated by new evidence, in which case Dr. John would be committing the ludic fallacy. The example isn't instructive because anyone reading this and thinking of scenario A isn't disputing the idea that, if this happened in reality, you should throw away the model. They're just interpreting the phrase "assume that" differently.
- moring 6y ago
  > When you have enough data to suggest your model is wrong, you should throw away the model and the assumptions it's based on, not the data. In the example, you agreed to assume the coin is fair. That is, you agreed to stick to the model even if it turns out to be wrong. Throwing the model away, however suggested by the evidence, is just violating the rules you agreed to.

baxtr 6y ago

That’s actually exactly what I was going to say. Of course you should update your priors continuously and thus evolve your model. A model is never perfect but it’s the best approximation to reality we have. In the end, Fat Tony has a model in his head, too.

FeepingCreature 6y ago
Right - "assume at the start that the coin is fair - you are free to update the assumption as you go" is a different thing than they said. Usually "assume that" assumptions are meant to hold throughout the question. As Randall says, "communicating badly and then acting smug when you're misunderstood is not cleverness."
- marvin 6y ago
  I think the central point here is that «assume that» is fine when working with a math problem, but can kill you in real life. If you’ve just lost two billion dollars, or killed 200,000 people in a preventable war, no one cares that the assumptions were reasonable.
  - FeepingCreature 6y ago
    Right, but the study has nothing to do with this, because the question was not in real life, but about a fictional scenario. "Assume you jump off a bridge. Would you be dead?" "No, because I'd change my mind. What, you think I'd jump off a bridge because someone tells me to?" "No, but I'd rather expect you'd assume it because I tell you, in the context of a hypothetical."
    - basdftrewq 6y ago
      I think the point would be that expertise in those fictional scenarios doesn't correlate with expertise in real scenarios, and many experts only have been trained in fictional scenarios.
    - Ambol 6y ago
      But this is experts we are talking about. There's supposed to be some degree of translation between models and real life. The point of the fallacy is that this translation can be very debatable and restricted in many fields, and that a good observer with decent intuition can beat those models easily. If this was about a purely intellectual roleplaying game with internal rules there would be no discussion in the first place.
      - FeepingCreature 6y ago
        > If this was about a purely intellectual roleplaying game with internal rules there would be no discussion in the first place. Which is what makes the discussion bullshit. Because if you're starting your experiment with "assume that", that's what you're signalling you're doing.
- basdftrewq 6y ago
  The point is that in real life problems it's more important to throw away ridiculous "assume that"s than having very neat models.
gwd 6y ago
I think the Ludic Fallacy would be better described as, "A tendency to be overconfident that you know what the game is." If you read the Black Swan, what Taleb is mainly railing against are people who are completely confident that because they have a model, that it must therefore be correct, and refuse to remember all the caveats, and what could happen if their model is wrong.

l0b0 6y ago

Yeah, the only thing it proves is that when a scientist says "assume X" to another scientist the common understanding is that it means "in a situation where it is given that X is true, no matter how ridiculous that might be in real life." As for more reasons why this is bullshit, see the gambler's fallacy and the Monty Hall problem – intuition is often terrible at arriving at the right prediction.

TheOtherHobbes 6y ago

But the assumption is the point. When you have enough data to suggest your model is wrong, you should throw away the model and the assumptions it's based on, not the data. The fact that your model can be justified as technically correct is irrelevant. When the data doesn't fit the model any more, the supposed absolute correctness of the textbook model blocks your ability to understand what's happening. This isn't even about games, but about whether it's possible to be open-minded enough to consider that simple textbook "solutions" are not guaranteed to be a good fit for real problems - especially if you don't understand their limitations or the assumptions they're based on. And it doesn't just apply to statistics.

deleted 6y ago
[deleted]
Hitton 6y ago
I remember reading some kind of fantasy anthology of Sherlock Holmes pastiches. In one story there was interesting twist on the famous quote "Once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth.". It was changed to the effect that "once whatever remains is ridiculously improbable, you should probably reconsider what is actually impossible".
leereeves 6y ago
But the example is poorly communicated. It's not clear whether the entire quoted portion is: A) a hypothetical introduced by the idiom "Assume that..." (which introduces a statement of fact in the hypothetical situation), and we are discussing a scenario in which the coin is in fact fair, in which case Dr. John is correct and Fat Tony is committing the gambler's fallacy, or B) a scenario in which a third-party tells someone that the coin is fair, but that is simply an assumption that should be updated by new evidence, in which case Dr. John would be committing the ludic fallacy. The example isn't instructive because anyone reading this and thinking of scenario A isn't disputing the idea that, if this happened in reality, you should throw away the model. They're just interpreting the phrase "assume that" differently.
moring 6y ago
> When you have enough data to suggest your model is wrong, you should throw away the model and the assumptions it's based on, not the data. In the example, you agreed to assume the coin is fair. That is, you agreed to stick to the model even if it turns out to be wrong. Throwing the model away, however suggested by the evidence, is just violating the rules you agreed to.