5 ms·

No, you aren't the only one. It has become even worse now that Twitter will not render without JavaScript enabled. Unfortunately, I still do not know what Ber

No, you aren't the only one. It has become even worse now that Twitter will not render without JavaScript enabled. Unfortunately, I still do not know what Berkson's Paradox is because I will not enable JavaScript for Twitter.

anonymousiam 6y ago
Okay, I googled it. A non-hostile site hosts a definition here: https://en.wikipedia.org/wiki/Berkson%27s_paradox https://en.wikipedia.org/wiki/Berkson%27s_paradox
- junippor 6y ago
  Thank you. Does anyone understand the difference between this and Simpson's paradox?
  - kgwgk 6y ago
    The latter appears when analyzing subgroups gives a different result than analyzing the pooled data. The former is about correlations that appear in samples which are not representative of the general population, due to the way that those samples are selected.
    - junippor 6y ago
      > The latter appears when analyzing subgroups gives a different result than analyzing the pooled data. > The former is about correlations that appear in samples which are not representative of the general population, due to the way that those samples are selected. You just said the same thing twice. Think about it. For one you used terms like "subgroups" and "pooled data" and for the other "samples" and "general population". Those are the same things. Then you used "[the effect] appears in" and in the other "correlations". Well, Simpsons paradox can also manifest itself in correlations. So you just said the same thing twice.
      - doubleunplussed 6y ago
        Eh. There is intentional splitting into subgroups, and there is accidental selection bias. I think that's the difference.
      - caddemon 6y ago
        I think Berkson's paradox is more specific than just correlations arising from non-random sample selection. Correlations that are not representative of the general population could still be useful, if it's a meaningful correlation within some subgroup of interest. The problem is when the features you are correlating relate too closely to the features that were used for sample selection - then you can end up with a trivial result. I've always learned of Simpson's paradox as relating more to different sample sizes when partitioning data, which can happen entirely arbitrarily - for example a baseball player getting injured part way through the season. The fact that one player's at bats get partitioned differently than another's is not caused by the on field performance, so there's no "double dipping" going on like I would imagine with Berkson's. Conversely I'm having trouble fitting a Berkson's example into the framework of Simpson's paradox, since there's no reason the poorly-selected subpopulation can't theoretically be exactly half of the general population. And if all of the samples are of equal size Simpson's paradox doesn't exist anymore (because with equal bin sizes the mean of means is equivalent to the overall mean).
        kgwgk 6y ago
        > I've always learned of Simpson's paradox as relating more to different sample sizes when partitioning data When you look at proportions based on binary outcomes it may be related to imbalanced groups but it's more general than that. In the context discussed here of correlations between continous variables the groups can be of similar size. See for example the chart here: https://towardsdatascience.com/simpsons-paradox-d2f4d8f08d42 https://towardsdatascience.com/simpsons-paradox-d2f4d8f08d42
        caddemon 6y ago
        Interesting, I only ever heard of Simpson's paradox in the context of comparing overall averages versus subgroup averages. I guess this paradox could then be thought of as a special case of Simpson's paradox? Since the out group will exclude people with both traits there should also be a negative correlation there, which disappears in the overall population. But in Berkson's case it seems they're implying the subgroup correlation is spurious whereas with Simpson's it could go either way.
1vuio0pswjnm7 6y ago
You do not need to enable Javascript, you only need to change your User-Agent header to one that is acceptable.

anonymousiam 6y ago

Okay, I googled it. A non-hostile site hosts a definition here: https://en.wikipedia.org/wiki/Berkson%27s_paradox https://en.wikipedia.org/wiki/Berkson%27s_paradox

junippor 6y ago
Thank you. Does anyone understand the difference between this and Simpson's paradox?
- kgwgk 6y ago
  The latter appears when analyzing subgroups gives a different result than analyzing the pooled data. The former is about correlations that appear in samples which are not representative of the general population, due to the way that those samples are selected.
  - junippor 6y ago
    > The latter appears when analyzing subgroups gives a different result than analyzing the pooled data. > The former is about correlations that appear in samples which are not representative of the general population, due to the way that those samples are selected. You just said the same thing twice. Think about it. For one you used terms like "subgroups" and "pooled data" and for the other "samples" and "general population". Those are the same things. Then you used "[the effect] appears in" and in the other "correlations". Well, Simpsons paradox can also manifest itself in correlations. So you just said the same thing twice.
    - doubleunplussed 6y ago
      Eh. There is intentional splitting into subgroups, and there is accidental selection bias. I think that's the difference.
    - caddemon 6y ago
      I think Berkson's paradox is more specific than just correlations arising from non-random sample selection. Correlations that are not representative of the general population could still be useful, if it's a meaningful correlation within some subgroup of interest. The problem is when the features you are correlating relate too closely to the features that were used for sample selection - then you can end up with a trivial result. I've always learned of Simpson's paradox as relating more to different sample sizes when partitioning data, which can happen entirely arbitrarily - for example a baseball player getting injured part way through the season. The fact that one player's at bats get partitioned differently than another's is not caused by the on field performance, so there's no "double dipping" going on like I would imagine with Berkson's. Conversely I'm having trouble fitting a Berkson's example into the framework of Simpson's paradox, since there's no reason the poorly-selected subpopulation can't theoretically be exactly half of the general population. And if all of the samples are of equal size Simpson's paradox doesn't exist anymore (because with equal bin sizes the mean of means is equivalent to the overall mean).
      - kgwgk 6y ago
        > I've always learned of Simpson's paradox as relating more to different sample sizes when partitioning data When you look at proportions based on binary outcomes it may be related to imbalanced groups but it's more general than that. In the context discussed here of correlations between continous variables the groups can be of similar size. See for example the chart here: https://towardsdatascience.com/simpsons-paradox-d2f4d8f08d42 https://towardsdatascience.com/simpsons-paradox-d2f4d8f08d42
        caddemon 6y ago
        Interesting, I only ever heard of Simpson's paradox in the context of comparing overall averages versus subgroup averages. I guess this paradox could then be thought of as a special case of Simpson's paradox? Since the out group will exclude people with both traits there should also be a negative correlation there, which disappears in the overall population. But in Berkson's case it seems they're implying the subgroup correlation is spurious whereas with Simpson's it could go either way.

1vuio0pswjnm7 6y ago

You do not need to enable Javascript, you only need to change your User-Agent header to one that is acceptable.