3 ms·

> would you then be justified in concluding that a news article about Zanzibar was more likely to be fake because it contained two zees? I completely agree wit

by nequo 4y ago

> would you then be justified in concluding that a news article about Zanzibar was more likely to be fake because it contained two zees? I completely agree with you about hidden assumptions. But I don’t think that causal thinking is one of the problems here. Bayes’ rule applies whether or not the relationship between X and Y is causal. Bayes’ rule is a predictive model and prediction does not need causation. The problem in the Zanzibar example is that the model is misspecified. Or at least the model specification does not represent how you as a human being with cultural knowledge think about the problem. You look at Zanzibar and see an island in Tanzania. Your model looks at Zanzibar and sees two a’s and two z’s and a few other letters. So it is the wrong implicit assumption about the underlying model that’s giving you “non-sensical” results.

lisper 4y ago
> the model is misspecified... it does not represent how you as a human being with cultural knowledge think about the problem Yes, that is exactly my point. The way "you as a human with cultural knowledge think about the problem" is a major component of Bayesian analysis. The math is almost incidental. By the time you have chosen what features to do the math on, most of the heavy lifting has already been done. This matters because if you just crunch the Bayesian numbers on an arbitrary data set you will almost certainly find features that appear predictive but are not. The more numbers you crunch, the more likely this is to happen. A lot of people lose money in the stock market this way.
- nequo 4y ago
  Much agreed. Data analysis needs a good theory.
- harry8 4y ago
  > A lot of people lose money in the stock market this way. This sounds extremely plausible to me. I'd like to use it as an example in something, any data to back it up?
  - kadoban 4y ago
    This is referred to as "p-hacking" if that helps you find some. Or maybe that term is more general, I'm not 100% sure, but it's close.
    - harry8 4y ago
      No it isn't. p-hacking is something much more general and is in no way evidence of people losing money on the stock market. Plausible unsubstantiated anecdote is not data, etc etc, no matter how well the story reads. Try to imagine that there's quite a few experts around HN ...
      - kadoban 4y ago
        P-hacking is sifting through _many_ possible relationships in a dataset and picking one that happens to look correlated. This is what I was responding to: > This matters because if you just crunch the Bayesian numbers on an arbitrary data set you will almost certainly find features that appear predictive but are not. The more numbers you crunch, the more likely this is to happen. A lot of people lose money in the stock market this way. Please explain how this is not exactly the same concept, expect with different motivations for looking through the data. > Try to imagine that there's quite a few experts around HN Why be a jerk though? I gave a reasonably well thought out comment related to one I found interesting, and made sure not to overly imply expertise.
        harry8 4y ago
        Forking paths. [1], overfitting, fitting to the holdout set, etc etc. "P-hacking" applicable to vastly more than stock data and is not evidence of anything, evidence is what was asked for. Your response implies I'm ignorant and can't google p-hacking without being told the name of that one way of finding false correlation applicable to many, many fields. Now you're getting a bit cross about it that I've quietly pointed out that it isn't useful, and you're not actually providing expertise here which you thought you were. It's kind to provide expertise and I appreciate the kindness. You misread, fine. Best. edit: Oh wait > Or maybe that term is more general, I'm not 100% sure, but it's close. That is an edit addition to the parent after my reply, so I guess you did get the point. Best. [1] http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf http://www.stat.columbia.edu/~gelman/research/unpublished/p_...

by nequo 4y ago

lisper 4y ago

> the model is misspecified... it does not represent how you as a human being with cultural knowledge think about the problem Yes, that is exactly my point. The way "you as a human with cultural knowledge think about the problem" is a major component of Bayesian analysis. The math is almost incidental. By the time you have chosen what features to do the math on, most of the heavy lifting has already been done. This matters because if you just crunch the Bayesian numbers on an arbitrary data set you will almost certainly find features that appear predictive but are not. The more numbers you crunch, the more likely this is to happen. A lot of people lose money in the stock market this way.

nequo 4y ago
Much agreed. Data analysis needs a good theory.
harry8 4y ago
> A lot of people lose money in the stock market this way. This sounds extremely plausible to me. I'd like to use it as an example in something, any data to back it up?
- kadoban 4y ago
  This is referred to as "p-hacking" if that helps you find some. Or maybe that term is more general, I'm not 100% sure, but it's close.
  - harry8 4y ago
    No it isn't. p-hacking is something much more general and is in no way evidence of people losing money on the stock market. Plausible unsubstantiated anecdote is not data, etc etc, no matter how well the story reads. Try to imagine that there's quite a few experts around HN ...
    - kadoban 4y ago
      P-hacking is sifting through _many_ possible relationships in a dataset and picking one that happens to look correlated. This is what I was responding to: > This matters because if you just crunch the Bayesian numbers on an arbitrary data set you will almost certainly find features that appear predictive but are not. The more numbers you crunch, the more likely this is to happen. A lot of people lose money in the stock market this way. Please explain how this is not exactly the same concept, expect with different motivations for looking through the data. > Try to imagine that there's quite a few experts around HN Why be a jerk though? I gave a reasonably well thought out comment related to one I found interesting, and made sure not to overly imply expertise.
      - harry8 4y ago
        Forking paths. [1], overfitting, fitting to the holdout set, etc etc. "P-hacking" applicable to vastly more than stock data and is not evidence of anything, evidence is what was asked for. Your response implies I'm ignorant and can't google p-hacking without being told the name of that one way of finding false correlation applicable to many, many fields. Now you're getting a bit cross about it that I've quietly pointed out that it isn't useful, and you're not actually providing expertise here which you thought you were. It's kind to provide expertise and I appreciate the kindness. You misread, fine. Best. edit: Oh wait > Or maybe that term is more general, I'm not 100% sure, but it's close. That is an edit addition to the parent after my reply, so I guess you did get the point. Best. [1] http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf http://www.stat.columbia.edu/~gelman/research/unpublished/p_...