4 ms·

Berkson's Paradox

101 points by luu 4y ago

ouid 4y ago
for whom is this unintuitive? pick two random numbers whose sum is 0.
- nerdponx 4y ago
  > pick two random numbers whose sum is 0. It's unintuitive if you don't realize that the numbers sum to 0. This stuff can creep up on you in real-world data without realizing it.
  - bmacho 4y ago
    Is that still happening? I'd guess that statisticians figured it out by now? Is it hard, or impossible maybe? But I agree with GP, how is this surprising? The example in the article is stupid, and I am not convinced whether there are any real surprising or counter-intuitive examples.
    - mxwsn 4y ago
      This can happen all the time in data science. Just add a few layers of bureaucracy and department separations between data gathering, cleaning/filtering, and analysis and it's easy to make the wrong conclusions on observed correlations without carefully thinking through all the filtering steps that might induce Berkson's paradox.
    - ZephyrBlu 4y ago
      It’s possible to implicitly filter your population, like in the hospital example given on the wiki page. Any situation where you make assumptions about the population could cause Berkson’s paradox. The Ellenberg example on the wiki page is a classic situation. Less attractive people are filtered out of the dating pool due to standards. This is reasonable, but it creates Berkson’s paradox. It’s often hard to realize this is happening.
rahimnathwani 4y ago
Related: * https://twitter.com/page_eco/status/1373266475230789633 https://twitter.com/page_eco/status/1373266475230789633 (discussion: https://news.ycombinator.com/item?id=26566810 https://news.ycombinator.com/item?id=26566810) * https://erikbern.com/2020/01/13/how-to-hire-smarter-than-the-market-a-toy-model.html https://erikbern.com/2020/01/13/how-to-hire-smarter-than-the...
mjburgess 4y ago
The whole family of statistical paradoxes like these arise, imv (and perhaps that of Pearl's) due to a failure to distinguish between statistical and explanatory (ie., causal) analysis. The whole machinery of statistics is insensitive to whether X causes Y, Y causes X, neither cause each other, or there is an intermediate cause. To see this, take any graph with any statistical model, and flip the axes. The analysis is the same, but the direction of causation implicitly reversed. Indeed, take any of these paradoxes (esp. simpson's). Nevertheless for much of the recent history of applied statistics these distinctions have not been made (esp. because it has been developed within the experimental contexts of dubious fields). It is important to state however, that no technique or quantity of statistics is ever sufficient to credit any association with being an explanation. To credit any association with any scientific theory, and hence neither with any properties. No quantity of associative statistical research into IQ thereby implies such a thing exists (cf. the reification fallacy). To transcend statistics into science one needs experiments, those which can control variables, and hence provide a non-statistical interpretive framework which demonstrates these variables are explanatory, and hence gives credence to the theories which explain them.
- perfecthjrjth 4y ago
  This. Very interesting take, indeed.
- achrono 4y ago
  I wonder if it would help mitigate the false reading of causality from a correlation X-Y chart if we lay these charts out on a 'V' layout instead of the usual 'L'. It might (1) distinguish these clearly from X-Y charts that do actually intend to show causality (which can remain on 'L') (2) do a better job at conveying that the 2 variables go together rather than one impacting the other.
dang 4y ago
Recent and related: Brilliant jerks, crazy hotties, and other artifacts of range restriction (2019) - https://news.ycombinator.com/item?id=33676648 https://news.ycombinator.com/item?id=33676648 - Nov 2022 (73 comments) Also: Berkson's Paradox - https://news.ycombinator.com/item?id=26566810 https://news.ycombinator.com/item?id=26566810 - March 2021 (39 comments) Berkson's Paradox - https://news.ycombinator.com/item?id=18667423 https://news.ycombinator.com/item?id=18667423 - Dec 2018 (21 comments) Berkson's Paradox - https://news.ycombinator.com/item?id=8264252 https://news.ycombinator.com/item?id=8264252 - Sept 2014 (20 comments)

Berkson's Paradox

101 points by luu 4y ago

ouid 4y ago

for whom is this unintuitive? pick two random numbers whose sum is 0.

nerdponx 4y ago
> pick two random numbers whose sum is 0. It's unintuitive if you don't realize that the numbers sum to 0. This stuff can creep up on you in real-world data without realizing it.
- bmacho 4y ago
  Is that still happening? I'd guess that statisticians figured it out by now? Is it hard, or impossible maybe? But I agree with GP, how is this surprising? The example in the article is stupid, and I am not convinced whether there are any real surprising or counter-intuitive examples.
  - mxwsn 4y ago
    This can happen all the time in data science. Just add a few layers of bureaucracy and department separations between data gathering, cleaning/filtering, and analysis and it's easy to make the wrong conclusions on observed correlations without carefully thinking through all the filtering steps that might induce Berkson's paradox.
  - ZephyrBlu 4y ago
    It’s possible to implicitly filter your population, like in the hospital example given on the wiki page. Any situation where you make assumptions about the population could cause Berkson’s paradox. The Ellenberg example on the wiki page is a classic situation. Less attractive people are filtered out of the dating pool due to standards. This is reasonable, but it creates Berkson’s paradox. It’s often hard to realize this is happening.

rahimnathwani 4y ago

Related: * https://twitter.com/page_eco/status/1373266475230789633 https://twitter.com/page_eco/status/1373266475230789633 (discussion: https://news.ycombinator.com/item?id=26566810 https://news.ycombinator.com/item?id=26566810) * https://erikbern.com/2020/01/13/how-to-hire-smarter-than-the-market-a-toy-model.html https://erikbern.com/2020/01/13/how-to-hire-smarter-than-the...

mjburgess 4y ago

The whole family of statistical paradoxes like these arise, imv (and perhaps that of Pearl's) due to a failure to distinguish between statistical and explanatory (ie., causal) analysis. The whole machinery of statistics is insensitive to whether X causes Y, Y causes X, neither cause each other, or there is an intermediate cause. To see this, take any graph with any statistical model, and flip the axes. The analysis is the same, but the direction of causation implicitly reversed. Indeed, take any of these paradoxes (esp. simpson's). Nevertheless for much of the recent history of applied statistics these distinctions have not been made (esp. because it has been developed within the experimental contexts of dubious fields). It is important to state however, that no technique or quantity of statistics is ever sufficient to credit any association with being an explanation. To credit any association with any scientific theory, and hence neither with any properties. No quantity of associative statistical research into IQ thereby implies such a thing exists (cf. the reification fallacy). To transcend statistics into science one needs experiments, those which can control variables, and hence provide a non-statistical interpretive framework which demonstrates these variables are explanatory, and hence gives credence to the theories which explain them.

perfecthjrjth 4y ago
This. Very interesting take, indeed.
achrono 4y ago
I wonder if it would help mitigate the false reading of causality from a correlation X-Y chart if we lay these charts out on a 'V' layout instead of the usual 'L'. It might (1) distinguish these clearly from X-Y charts that do actually intend to show causality (which can remain on 'L') (2) do a better job at conveying that the 2 variables go together rather than one impacting the other.

dang 4y ago

Recent and related: Brilliant jerks, crazy hotties, and other artifacts of range restriction (2019) - https://news.ycombinator.com/item?id=33676648 https://news.ycombinator.com/item?id=33676648 - Nov 2022 (73 comments) Also: Berkson's Paradox - https://news.ycombinator.com/item?id=26566810 https://news.ycombinator.com/item?id=26566810 - March 2021 (39 comments) Berkson's Paradox - https://news.ycombinator.com/item?id=18667423 https://news.ycombinator.com/item?id=18667423 - Dec 2018 (21 comments) Berkson's Paradox - https://news.ycombinator.com/item?id=8264252 https://news.ycombinator.com/item?id=8264252 - Sept 2014 (20 comments)