5 ms·

Scientists used the same data, but their politics predicted the results

30 points by geox 9mo ago

jjk166 9mo ago
This is why the goal of experiments needs to be falsifying hypotheses. If you can make a model that fits the raw data without absurd assumptions which contradicts your hypothesis, then the data doesn't support your hypothesis. If you're checking whether a coin is fair and you toss out significantly more tails than heads because you didn't feel they were proper tosses, of course you're going to reject the null hypothesis that the coin is fair. Even if your criteria for what was a proper toss is objective and reasonable, you're no longer testing whether the coin is fair, you're only testing whether your criteria for counting a toss is fair assuming the coin isn't. When you construct your experiments to reject a hypothesis - this particular model can not be right if we see this - then you can make real progress towards truth.
- pepinator 9mo ago
  how do you falsify a hypothesis in social sciences?
  - jjk166 9mo ago
    Same way as the hard sciences - you make a prediction for something that you would never observe if the hypothesis is true, and then you go look for it. If you find it, then the hypothesis must be rejected. I might suspect that hunter gatherer tribes don't go to war, and I might observe many such tribes which don't, but that doesn't prove my hypothesis right. On the other hand, if I can find just one tribe which does go to war, then the hypothesis has been falsified.
    - pepinator 9mo ago
      The problem is that there are no two equal situations in social sciences, so you won't ever have the same set of initial conditions. I don't know why they call them sciences, but the scientific method is intrinsically incompatible with social phenomena.
      - jjk166 9mo ago
        You won't ever have the exact same set of initial conditions in any experiment. You drop two balls from the tower of pisa and do it again you have different air conditions, the planet has moved, the tower may have leaned slightly more, etc. You either control for these variables or assume they don't matter. The value of science is that even if you drop two balls on the moon, you're still going to get the same result after controlling for differences - the results are broadly applicable. It's exactly the same in the social sciences. If you're looking at something real, it will show up consistently in repeated experiments with proper controls despite variation in irrelevant initial conditions. The phenomena that social sciences try to understand may have many more variables, but there's nothing inherently special about humanity that makes us impossible to describe.
        pepinator 9mo ago
        I'm not saying humanity is impossible to describe (on the contrary!). your physics example is too different to what happens in social sciences, it's not a good analogy. you can largely control the conditions under which you throw the balls, while if you're, for instance, studying the recent socio political events in the USA, things are given and then you need to rely on similar past event to understand current ones. even in fields like psychology it's too difficult to even measure initial conditions and outputs. that's why you rely on qualitative analysis.
- chickenimprint 9mo ago
  Then people will simply falsify the logical inverse of their current hypotheses. Preregistration is a more promising approach.
  - jjk166 9mo ago
    Falsifying the logical inverse is fine. The key though is that showing the inverse of what you believe can not be true is much harder than showing that what you believe might be true. Preregistration only helps with a select set of researcher bias, specifically where they retroactively change parts of their model to get the intended result. However it does nothing to protect against bias baked into the model ahead of time. Presumably the same reasons that researchers are strongly politically opinionated are the reasons they emphasize certain things over others in their models long before they see any data.
    - chickenimprint 9mo ago
      Falsifying the logical inverse of X is identical to verifying X. There's nothing about negation that does anything here. You're making the same mistake people make when claiming "You can't prove a negative". > The key though is that showing the inverse of X can not be true is much harder than showing that what X might be true. This is nonsense modal logic. You're saying ¬◻¬X, which if necessity and possibility are duals, is equivalent to ◇X, and otherwise an irrelevant statement. The inverse of X is ¬X. ¬¬X is logically equivalent to X.
      - jjk166 9mo ago
        > Falsifying the logical inverse of X is identical to verifying X. That's exactly the point. The issue with positivist hypotheses is that you can find evidence that supports, but does not actually verify, the claim. This seems convincing until you try to flip it around. So for example if a prediction of my model is that the sun will rise tomorrow, and the sun does rise tomorrow, that seems like it supports my model. But if my model can be wrong and the sun would still rise tomorrow, then looking for the sun rise was never going to answer the question. If you are doing a test that will actually verify X then bias doesn't matter.
wtcactus 9mo ago
No, they aren’t scientists, they are social “scientists”. Physicists, geologists and other kinds of real scientists’ personal politics don’t determine the outcome of their research. It’s high time we denounce and publicly defund social “sciences” for that they really are: astrology at best and political propaganda at worst.
- gus_massa 9mo ago
  Assuming "zero" bias for physicist is too optimistic, let's say it's quite low.
  - wtcactus 9mo ago
    I’m not saying they don’t have bias. But their bias is not political. We all have bias. From our supervisors, from our department, from the teachers we had. They have their own pet theories and they do imprint a part of them in their students. But they aren’t dictated by political ideologies.
- whattheheckheck 9mo ago
  Science only worked because of the religion/politics ecosystem
  - bigbadfeline 9mo ago
    > Science only worked because of the religion/politics ecosystem In many cases what worked did so despite the religion/politics ecosystem. Moreover, some science theories worked while others didn't. Some helped the ecosystem, others damaged it. At their current state of development, the utility of social sciences is between useless and severely damaging, depending on case. And let's not try to hide faulty science by wrapping it in the generic term "science" which is riding on the good name of sciences with much better track records. What's faulty has to be fixed, not covered up.
- pepinator 9mo ago
  saying that social science is astrology at best is quite extremist. serious people work in those fields
xve 9mo ago
The foundation of science was built on a certain "political" view (the enlightenment). It's interesting that maybe science isn't exactly science without those values.
- Voultapher 9mo ago
  Science has a massive blind spot, one it can't fathom exists. For many today, especially on HN science is closer to a religion than what they themselves view it as. This is not a particularly popular believe since it contradicts a lot of nicely build up self-perceptions. Science can't figure out what is worthwhile pursuing and what isn't _without_ biased input at the very beginning of that chain. Science can't reason about the limits to it's power since it assumes that everything can be analyzed and broken into smaller problems. The fact that science is a tool, one profoundly incompatible with certain types of very real properties of our world does not fit into the religion of science. "Science can solve all problems and if it hasn't we just haven't tried hard enough". Science is a hammer that insists it is the right tool for every problem and if it doesn't work well you're just holding it wrong. I was born into and shaped by a science and enlightenment religion world-view and lack proper words to describe the issues with it, but I feel them.
  - xve 9mo ago
    I sympathize with your feeling. All these questions are not going away, science can't do it, religion doesn't have those answers either. The religion of science offers the worst of both worlds. We are struggling to see something beyond these two options but I think there must be some other approach.
ratelimitsteve 9mo ago
when i took stats in high school our final had one set of data and two questions. one was to use that set of data to prove a hypothesis, and the other to use that same set of data to disprove it. the trick was manipulating confidence, and the real trick was realizing that this was almost certainly happening any time we allow someone else to make the critical decisions for how to crunch any particular dataset. there seems to be an unspoken translation layer: you ask a question in english, then use statistics as the process of translating that question from english to math, then apply the translated question to the dataset and get an answer. if translation were mechanical that would be fine, but often the english has to go through interpretation. How confident do we have to be in a "yes" answer? 99%? 95%? 50/50? What constitutes an outlier? Sometimes a very slight shift in the line between good and bad data will include or exclude a single data point that can shift the calculations. The problem seems to be standardization of this translation layer as much as anything else, which is to say that everyone is ostensibly trying to answer the same question but under the hood they're asking very different questions of the data.

30 points by geox 9mo ago

This is why the goal of experiments needs to be falsifying hypotheses. If you can make a model that fits the raw data without absurd assumptions which contradicts your hypothesis, then the data doesn't support your hypothesis. If you're checking whether a coin is fair and you toss out significantly more tails than heads because you didn't feel they were proper tosses, of course you're going to reject the null hypothesis that the coin is fair. Even if your criteria for what was a proper toss is objective and reasonable, you're no longer testing whether the coin is fair, you're only testing whether your criteria for counting a toss is fair assuming the coin isn't. When you construct your experiments to reject a hypothesis - this particular model can not be right if we see this - then you can make real progress towards truth.

pepinator 9mo ago
how do you falsify a hypothesis in social sciences?
- jjk166 9mo ago
  Same way as the hard sciences - you make a prediction for something that you would never observe if the hypothesis is true, and then you go look for it. If you find it, then the hypothesis must be rejected. I might suspect that hunter gatherer tribes don't go to war, and I might observe many such tribes which don't, but that doesn't prove my hypothesis right. On the other hand, if I can find just one tribe which does go to war, then the hypothesis has been falsified.
  - pepinator 9mo ago
    The problem is that there are no two equal situations in social sciences, so you won't ever have the same set of initial conditions. I don't know why they call them sciences, but the scientific method is intrinsically incompatible with social phenomena.
    - jjk166 9mo ago
      You won't ever have the exact same set of initial conditions in any experiment. You drop two balls from the tower of pisa and do it again you have different air conditions, the planet has moved, the tower may have leaned slightly more, etc. You either control for these variables or assume they don't matter. The value of science is that even if you drop two balls on the moon, you're still going to get the same result after controlling for differences - the results are broadly applicable. It's exactly the same in the social sciences. If you're looking at something real, it will show up consistently in repeated experiments with proper controls despite variation in irrelevant initial conditions. The phenomena that social sciences try to understand may have many more variables, but there's nothing inherently special about humanity that makes us impossible to describe.
      - pepinator 9mo ago
        I'm not saying humanity is impossible to describe (on the contrary!). your physics example is too different to what happens in social sciences, it's not a good analogy. you can largely control the conditions under which you throw the balls, while if you're, for instance, studying the recent socio political events in the USA, things are given and then you need to rely on similar past event to understand current ones. even in fields like psychology it's too difficult to even measure initial conditions and outputs. that's why you rely on qualitative analysis.
chickenimprint 9mo ago
Then people will simply falsify the logical inverse of their current hypotheses. Preregistration is a more promising approach.
- jjk166 9mo ago
  Falsifying the logical inverse is fine. The key though is that showing the inverse of what you believe can not be true is much harder than showing that what you believe might be true. Preregistration only helps with a select set of researcher bias, specifically where they retroactively change parts of their model to get the intended result. However it does nothing to protect against bias baked into the model ahead of time. Presumably the same reasons that researchers are strongly politically opinionated are the reasons they emphasize certain things over others in their models long before they see any data.
  - chickenimprint 9mo ago
    Falsifying the logical inverse of X is identical to verifying X. There's nothing about negation that does anything here. You're making the same mistake people make when claiming "You can't prove a negative". > The key though is that showing the inverse of X can not be true is much harder than showing that what X might be true. This is nonsense modal logic. You're saying ¬◻¬X, which if necessity and possibility are duals, is equivalent to ◇X, and otherwise an irrelevant statement. The inverse of X is ¬X. ¬¬X is logically equivalent to X.
    - jjk166 9mo ago
      > Falsifying the logical inverse of X is identical to verifying X. That's exactly the point. The issue with positivist hypotheses is that you can find evidence that supports, but does not actually verify, the claim. This seems convincing until you try to flip it around. So for example if a prediction of my model is that the sun will rise tomorrow, and the sun does rise tomorrow, that seems like it supports my model. But if my model can be wrong and the sun would still rise tomorrow, then looking for the sun rise was never going to answer the question. If you are doing a test that will actually verify X then bias doesn't matter.

wtcactus 9mo ago

No, they aren’t scientists, they are social “scientists”. Physicists, geologists and other kinds of real scientists’ personal politics don’t determine the outcome of their research. It’s high time we denounce and publicly defund social “sciences” for that they really are: astrology at best and political propaganda at worst.

gus_massa 9mo ago
Assuming "zero" bias for physicist is too optimistic, let's say it's quite low.
- wtcactus 9mo ago
  I’m not saying they don’t have bias. But their bias is not political. We all have bias. From our supervisors, from our department, from the teachers we had. They have their own pet theories and they do imprint a part of them in their students. But they aren’t dictated by political ideologies.
whattheheckheck 9mo ago
Science only worked because of the religion/politics ecosystem
- bigbadfeline 9mo ago
  > Science only worked because of the religion/politics ecosystem In many cases what worked did so despite the religion/politics ecosystem. Moreover, some science theories worked while others didn't. Some helped the ecosystem, others damaged it. At their current state of development, the utility of social sciences is between useless and severely damaging, depending on case. And let's not try to hide faulty science by wrapping it in the generic term "science" which is riding on the good name of sciences with much better track records. What's faulty has to be fixed, not covered up.
pepinator 9mo ago
saying that social science is astrology at best is quite extremist. serious people work in those fields

xve 9mo ago

The foundation of science was built on a certain "political" view (the enlightenment). It's interesting that maybe science isn't exactly science without those values.

Voultapher 9mo ago
Science has a massive blind spot, one it can't fathom exists. For many today, especially on HN science is closer to a religion than what they themselves view it as. This is not a particularly popular believe since it contradicts a lot of nicely build up self-perceptions. Science can't figure out what is worthwhile pursuing and what isn't _without_ biased input at the very beginning of that chain. Science can't reason about the limits to it's power since it assumes that everything can be analyzed and broken into smaller problems. The fact that science is a tool, one profoundly incompatible with certain types of very real properties of our world does not fit into the religion of science. "Science can solve all problems and if it hasn't we just haven't tried hard enough". Science is a hammer that insists it is the right tool for every problem and if it doesn't work well you're just holding it wrong. I was born into and shaped by a science and enlightenment religion world-view and lack proper words to describe the issues with it, but I feel them.
- xve 9mo ago
  I sympathize with your feeling. All these questions are not going away, science can't do it, religion doesn't have those answers either. The religion of science offers the worst of both worlds. We are struggling to see something beyond these two options but I think there must be some other approach.

ratelimitsteve 9mo ago

when i took stats in high school our final had one set of data and two questions. one was to use that set of data to prove a hypothesis, and the other to use that same set of data to disprove it. the trick was manipulating confidence, and the real trick was realizing that this was almost certainly happening any time we allow someone else to make the critical decisions for how to crunch any particular dataset. there seems to be an unspoken translation layer: you ask a question in english, then use statistics as the process of translating that question from english to math, then apply the translated question to the dataset and get an answer. if translation were mechanical that would be fine, but often the english has to go through interpretation. How confident do we have to be in a "yes" answer? 99%? 95%? 50/50? What constitutes an outlier? Sometimes a very slight shift in the line between good and bad data will include or exclude a single data point that can shift the calculations. The problem seems to be standardization of this translation layer as much as anything else, which is to say that everyone is ostensibly trying to answer the same question but under the hood they're asking very different questions of the data.