7 ms·

Statistical challenges and misreadings of literature create unreplicable science [pdf]

67 points by luu 2y ago

blackeyeblitzar 2y ago
On page 11 there is a mention of taking the result of self reporting (surveys) at their word. I’ve wondered about this issue not just in science but other situations. For example political polling, data point in time surveys, census, etc. Without verification, what good is the data? And yet you often see such self reported data quoted by articles or papers as if it were factual.
- fithisux 2y ago
  It is a proxy.
- tsimionescu 2y ago
  For many types of data you'd like, there is no alternative. For example for political polls, especially early ones where you're asking people who they would vote for if so and so were running, what other tool could you use? Or how would you verify the results? Of course, given the weakness of the data, you have to remember to place very little trust in the results. Even so, "X% of people say they would vote for Y" can be an interesting information in itself, even if those people were lying.
  - deleted 2y ago
    [deleted]
  - graemep 2y ago
    Polls may or may not be carried out in a neutral way. A good many are deliberately biased, even more have known flaws that are glossed over. A humorous take on this I love (from the 80s BBC comedy Yes Minister, which is often scarily realistic): https://www.youtube.com/watch?v=G0ZZJXw4MTA https://www.youtube.com/watch?v=G0ZZJXw4MTA
    - tsimionescu 2y ago
      Oh, absolutely, even in more subtle, unintentional ways (order of responses, whether you offer a neutral option or not, etc). But that doesn't mean polling is entirely useless, or that you need to verify the responses in the way that the original seemed to suggest.
- culebron21 2y ago
  Pollsters know about this, and the rule of thumb is that you can reliably ask people of what they currently do routinely, almost daily. E.g. if you ask them who they voted 10 years ago, they might remember it wrong. If you ask what they're gonna do, this is completely unreliable. In political polls, this is the only thing you can do. I guess, they have corrections for the real attendance of social/age strata. In deep interviews, the sociologists are more interested in how people explain their world picture and what they point at, and how react to questions, than to what they tell about themselves.
- michaelt 2y ago
  > For example political polling, data point in time surveys, census, etc. Without verification, what good is the data? Political polls are regularly wrong https://www.bbc.co.uk/news/articles/cj4ve004llxo https://www.bbc.co.uk/news/articles/cj4ve004llxo In my country, supporters of one political party seem a lot more likely to tell polling companies they're "undecided" - so the pollsters try to adjust their predictions, based on what they've seen in previous elections. But the problem is getting the adjustment right is very difficult. The political landscape can change a great deal between elections. If John McCain and Donald Trump appeal to very different sets of voters, why should voter behaviour observed with McCain hold true with Trump?
- HPsquared 2y ago
  It's like the uncertainty principle: even the act of asking someone a question will likely change their views.
- jjk166 2y ago
  There are multiple levels to data. If 57% of respondents to a self reported survey say they are going to do X, it doesn't mean that 57% of people are actually going to do X. But if you do the same survey again and now only 43% of respondents say they will do X, then that is clear evidence something has clearly changed, even if you don't know exactly what effect that something will have on X. That is very useful. The problem is when people only look at self reporting. For example if in the previous scenario 57% responded X but only 54% actually did X, then someone might naively assume that after 43% respond X that 40% will actually do X. Or just as naively they could say 54% will still do X. Or they could apply any of an infinite number of other models. There exists a model that will spit out any given answer for any given input, so without that followup work to actually verify and understand the underlying mechanism, models are worthless.
mjburgess 2y ago
We're increasingly aware today of how the media operates cycles of self-referential and self-justifying citations: a TV show will quote an article that reports "some people" taking an issue, which ends up being a quote from someone interviewed for another newspaper article.. and so on. This "legitimacy laundering" is rampant, and we're now getting towards media literacy levels which expose it for many people. However, most concerning: this is how academia has always worked. It's the great absurdity of peer review, and of citation. This is how entire fields can sustain themselves with little or no scientific validity (esp. see, psychometrics). We are no where near the equivalent "academic literacy" for generally informed members of the public to understand this problem. Entire fields can be sustained with zero "empirical pressure" to close down. So long as one can cite another who can cite another... and somewhere some government body will take these citations as prima facie evidence of a research programme, then funding will be given and more papers published.
- littlestymaar 2y ago
  Economics works exactly like this, but at a fractal level: there are many different subfields living in the grand family of economists which barely communicate with one another, often have dramatically different theories that lead to opposing conclusion, none of which even trying to give an accurate description of the world but instead promote one particular ideological bias. Every once is a while someone from one chapel is going to actually do science by refuting one other chapel's bullshit, but it's almost never seen as a good thing by anyone, and the Nobel committee sometimes ends up giving the price together to people from different chapels even when their work contradict each other (Fama being empirically being proven wrong by Schiller and both being given the price in 2013 to appease the feud). Economic behaviors are so complex that studying economics seriously requires gigantic efforts for little results, and people who attempt to do that stand no chance of survival in the publish-or-perish competition against the grandiose bullshitters, and that's how we get where we are.
  - light_hue_1 2y ago
    This happens everywhere. Let me tell you a story about who studies language and how. Linguistics studies language right? It has a few hundred year history of doing so seriously. But once AI people wanted to also have a look linguists became unhappy. They didn't like the idea of someone else saying something about their problem with a different set of tools. Those tools don't provide explanations that they approve of. Even the thought of measuring things over a dataset to check if your linguistic theories work made them bristle with Chomsky famously rejecting the idea of evidence at scale. So AI people created natural language processing. Their own field to study the same problem. One in a while someone tried to do something to bridge the gap which led to the famous "every time I fire a linguist my performance goes up". Many people in NLP see linguists as outdated fossils. Most would probably vote to just shut down linguistics departments I suspect. Then something funny happened. The LLM people did to NLP what they did to linguistics. And the NLP community rejected them. Because LLMs don't provide an explanation that they approve of. Recently just like in linguistics the President of ACL, the main NLP organization, said recently that NLP is not about machine learning! The absurdity of that statement just boggles the mind. So the LLM people now have their own conference and field. A lot of people cannot accept that things must change over time. They want to do research the way they did it 10 or 40 years ago. So they create an immune response. It's human nature. As someone who doesn't feel this way and enjoys moving forward, I'm really sad about what's happened to NLP in the past few years. It would be nice if we could stop this rift. But it won't happen with the current hate filled President. I stopped sending any work there until they reform. Just the cycle of academic life. I wonder if science would move faster if we could select for people who are willing to change with the times.
ykonstant 2y ago
Besides the sociological problems listed, we must always be conscious of how counterintuitive and difficult statistical inference itself can be. Good things to search for are statistical fallacies, probabilistic paradoxes and books like Counterexamples in Probability. And it is not sufficient to read about them once or twice; researchers who use statistical inference regularly must revisit these caveats at least as regularly. Myself, I have taught Probability and Statistics many times, discussed and dispelled many misconceptions by students. Would I be 100% sure I will not be caught up in a fallacy while informally thinking about probability? I wouldn't even be 10% sure; any intuition I conjure up, I would triple check as rigorously as possible.

67 points by luu 2y ago

On page 11 there is a mention of taking the result of self reporting (surveys) at their word. I’ve wondered about this issue not just in science but other situations. For example political polling, data point in time surveys, census, etc. Without verification, what good is the data? And yet you often see such self reported data quoted by articles or papers as if it were factual.

fithisux 2y ago
It is a proxy.
tsimionescu 2y ago
For many types of data you'd like, there is no alternative. For example for political polls, especially early ones where you're asking people who they would vote for if so and so were running, what other tool could you use? Or how would you verify the results? Of course, given the weakness of the data, you have to remember to place very little trust in the results. Even so, "X% of people say they would vote for Y" can be an interesting information in itself, even if those people were lying.
- deleted 2y ago
  [deleted]
- graemep 2y ago
  Polls may or may not be carried out in a neutral way. A good many are deliberately biased, even more have known flaws that are glossed over. A humorous take on this I love (from the 80s BBC comedy Yes Minister, which is often scarily realistic): https://www.youtube.com/watch?v=G0ZZJXw4MTA https://www.youtube.com/watch?v=G0ZZJXw4MTA
  - tsimionescu 2y ago
    Oh, absolutely, even in more subtle, unintentional ways (order of responses, whether you offer a neutral option or not, etc). But that doesn't mean polling is entirely useless, or that you need to verify the responses in the way that the original seemed to suggest.
culebron21 2y ago
Pollsters know about this, and the rule of thumb is that you can reliably ask people of what they currently do routinely, almost daily. E.g. if you ask them who they voted 10 years ago, they might remember it wrong. If you ask what they're gonna do, this is completely unreliable. In political polls, this is the only thing you can do. I guess, they have corrections for the real attendance of social/age strata. In deep interviews, the sociologists are more interested in how people explain their world picture and what they point at, and how react to questions, than to what they tell about themselves.
michaelt 2y ago
> For example political polling, data point in time surveys, census, etc. Without verification, what good is the data? Political polls are regularly wrong https://www.bbc.co.uk/news/articles/cj4ve004llxo https://www.bbc.co.uk/news/articles/cj4ve004llxo In my country, supporters of one political party seem a lot more likely to tell polling companies they're "undecided" - so the pollsters try to adjust their predictions, based on what they've seen in previous elections. But the problem is getting the adjustment right is very difficult. The political landscape can change a great deal between elections. If John McCain and Donald Trump appeal to very different sets of voters, why should voter behaviour observed with McCain hold true with Trump?
HPsquared 2y ago
It's like the uncertainty principle: even the act of asking someone a question will likely change their views.
jjk166 2y ago
There are multiple levels to data. If 57% of respondents to a self reported survey say they are going to do X, it doesn't mean that 57% of people are actually going to do X. But if you do the same survey again and now only 43% of respondents say they will do X, then that is clear evidence something has clearly changed, even if you don't know exactly what effect that something will have on X. That is very useful. The problem is when people only look at self reporting. For example if in the previous scenario 57% responded X but only 54% actually did X, then someone might naively assume that after 43% respond X that 40% will actually do X. Or just as naively they could say 54% will still do X. Or they could apply any of an infinite number of other models. There exists a model that will spit out any given answer for any given input, so without that followup work to actually verify and understand the underlying mechanism, models are worthless.

mjburgess 2y ago

We're increasingly aware today of how the media operates cycles of self-referential and self-justifying citations: a TV show will quote an article that reports "some people" taking an issue, which ends up being a quote from someone interviewed for another newspaper article.. and so on. This "legitimacy laundering" is rampant, and we're now getting towards media literacy levels which expose it for many people. However, most concerning: this is how academia has always worked. It's the great absurdity of peer review, and of citation. This is how entire fields can sustain themselves with little or no scientific validity (esp. see, psychometrics). We are no where near the equivalent "academic literacy" for generally informed members of the public to understand this problem. Entire fields can be sustained with zero "empirical pressure" to close down. So long as one can cite another who can cite another... and somewhere some government body will take these citations as prima facie evidence of a research programme, then funding will be given and more papers published.

littlestymaar 2y ago
Economics works exactly like this, but at a fractal level: there are many different subfields living in the grand family of economists which barely communicate with one another, often have dramatically different theories that lead to opposing conclusion, none of which even trying to give an accurate description of the world but instead promote one particular ideological bias. Every once is a while someone from one chapel is going to actually do science by refuting one other chapel's bullshit, but it's almost never seen as a good thing by anyone, and the Nobel committee sometimes ends up giving the price together to people from different chapels even when their work contradict each other (Fama being empirically being proven wrong by Schiller and both being given the price in 2013 to appease the feud). Economic behaviors are so complex that studying economics seriously requires gigantic efforts for little results, and people who attempt to do that stand no chance of survival in the publish-or-perish competition against the grandiose bullshitters, and that's how we get where we are.
- light_hue_1 2y ago
  This happens everywhere. Let me tell you a story about who studies language and how. Linguistics studies language right? It has a few hundred year history of doing so seriously. But once AI people wanted to also have a look linguists became unhappy. They didn't like the idea of someone else saying something about their problem with a different set of tools. Those tools don't provide explanations that they approve of. Even the thought of measuring things over a dataset to check if your linguistic theories work made them bristle with Chomsky famously rejecting the idea of evidence at scale. So AI people created natural language processing. Their own field to study the same problem. One in a while someone tried to do something to bridge the gap which led to the famous "every time I fire a linguist my performance goes up". Many people in NLP see linguists as outdated fossils. Most would probably vote to just shut down linguistics departments I suspect. Then something funny happened. The LLM people did to NLP what they did to linguistics. And the NLP community rejected them. Because LLMs don't provide an explanation that they approve of. Recently just like in linguistics the President of ACL, the main NLP organization, said recently that NLP is not about machine learning! The absurdity of that statement just boggles the mind. So the LLM people now have their own conference and field. A lot of people cannot accept that things must change over time. They want to do research the way they did it 10 or 40 years ago. So they create an immune response. It's human nature. As someone who doesn't feel this way and enjoys moving forward, I'm really sad about what's happened to NLP in the past few years. It would be nice if we could stop this rift. But it won't happen with the current hate filled President. I stopped sending any work there until they reform. Just the cycle of academic life. I wonder if science would move faster if we could select for people who are willing to change with the times.

ykonstant 2y ago

Besides the sociological problems listed, we must always be conscious of how counterintuitive and difficult statistical inference itself can be. Good things to search for are statistical fallacies, probabilistic paradoxes and books like Counterexamples in Probability. And it is not sufficient to read about them once or twice; researchers who use statistical inference regularly must revisit these caveats at least as regularly. Myself, I have taught Probability and Statistics many times, discussed and dispelled many misconceptions by students. Would I be 100% sure I will not be caught up in a fallacy while informally thinking about probability? I wouldn't even be 10% sure; any intuition I conjure up, I would triple check as rigorously as possible.