6 ms·

I've seen the term 'evidence-based policy' pop up all over the place. I take issue with people who use this phrase because most of the time their definition of

I've seen the term 'evidence-based policy' pop up all over the place. I take issue with people who use this phrase because most of the time their definition of evidence is not very well developed, and when the evidence is something they don't like they start to stretch their definition of evidence so they can dismiss implementing a certain policy. If evidence is only restricted to, and dictated by, academic papers, you leave room for companies and news media to start selectively reporting and funding studies they want to see or want people to see. And where are the scientific papers dedicated to studying this problem? No one is going to fund them. Now you have an entire society believing that corporate influence is not real because there is seemingly no evidence for it. If you only make decisions based on science you can pretend problems aren't real without a scientist's approval, and you can pretend solutions to those problems are bad if there is even a tiny bit of counter-evidence that supports your viewpoint.

mindcrime 6y ago
If you only make decisions based on science you can pretend problems aren't real without a scientist's approval, and you can pretend solutions to those problems are bad if there is even a tiny bit of counter-evidence that supports your viewpoint. And if you make decisions based on anything other than science, is the final outcome going to be any better? I admit that something that we might call "misuse of science" is a real thing, and a problem. But I don't see any solution that in any way involves de-emphasizing the importance of science in decision making.
- arotical 6y ago
  Here's a scenario: You are tasked with dealing with a problem where there is little research and no scientific consensus. There is plenty of video evidence and other related pieces of non-so-scientific evidence, but most people don't count that in evidence-based policy. What do you do? Sometimes, I think, we have to make decisions when there's no clear solution, and it's better than just dismissing a very obvious problem as non-existent. So yes, include science in every decision, but it's not the be all end all of society, it never has been, and it's not good to teach people that you can't tackle issues without a known best solution.
  - acqq 6y ago
    > a problem where there is little research and no scientific consensus. > There is plenty of video evidence I can't even what you talk about. "Video evidence"? Edit: if I'd guess it's about "UFOs" for these there practically exists scientific consensus: no aliens proved, ever. "Video evidence" isn't the evidence of aliens.
    - domnomnom 6y ago
      Lmao, turtles all the way down my friend.
  - Retric 6y ago
    In the real world problems are never completely novel. Video game addiction for example clearly only showed up after video games, but it’s also closely related to other forms of addiction. So, simply using related models and then adjusting to new information as it shows up works just fine. If government happens to ban loot boxes for a decade it’s not that important if they get it wrong and that’s plenty of time to do real research and adjust policy. PS: If you disagree try and come up with a counter example.
- SpicyLemonZest 6y ago
  We certainly shouldn't de-emphasize the importance of science. While not all scientific studies are correct, if someone makes a habit of saying "bah who cares what scientists think", they're gonna make bad decisions. But it's important to keep in mind that science is usually a tool to inform decisions, not a verdict to decide them. I see a lot of people declare we have to implement suchandsuch policy because that's what the science says, and almost all of them are adding implicit assumptions about values and goals to the studies they cite.
- 0d9eooo 6y ago
  It depends on what you mean by "science." Science has become so political and subject to hype and fads, at least in biomedicine, that everyone should be deeply skeptical of every claim that is made. That doesn't mean that you should distrust everything completely, just that critical skepticism should be the norm. If by "science" you mean that "rigorous, logical empirical study" should be part of the decision-making, yes, I agree. If by "science" you mean that the statements of scientists with prestigious academic appointments should be given priority over other considerations, I say no. Outsider criticism should be welcome, and I think nonscientific considerations are sometimes required. One of the reasons society is in the mess it is in (at least in the US) is because of a failure to realize that certain segments, like the biomedical-scientific community, are just as driven by human failures as any other.
feralimal 6y ago
Even evidence might not be showing you what you think. Eg if you fund 10 studies in support of, er, say cats being dangerous to humans. 2 might find that to be true, 2 find that false, and 6 remain neutral. You can leave the 8 you that don't support what you want to show unpublished. You can then fund more studies that build on the 2 you like. Rinse and repeat. Pretty soon, driven by funding alone, you will have lots of 'evidence' that caters to your viewpoint. Is that science? Unfortunately, funding determines scientific reality.
- ceejayoz 6y ago
  XKCD has a great comic demonstrating how this is done. https://xkcd.com/882/ https://xkcd.com/882/
Nursie 6y ago
I don't believe that all policy should 100% have to be formed from scientific evidence, but there are places where the evidence is well researched and well understood, yet politicians legislate based on their own prejudices, directly flying in the face of both experts and evidence. In the face of this sort of behaviour, is it not reasonable to ask for more evidence-based policy? Or at least that real evidence be considered where it is available?
- arotical 6y ago
  yeah this is fair
DoreenMichele 6y ago
Social policy is inherently hard to get objective evidence for. Social experiments mostly don't work because the fact that it's an experiment alters the behavior of participants. This is part of why I don't think there is any value in UBI experiments. Our mental models influence the kinds of results we get. This is always true, but it's a bigger problem for some things than for others.
nitwit005 6y ago
It depends how strictly you take the idea. Often people just want some sort of demonstration that people at least looked into things and thought about the alternatives.

mindcrime 6y ago

If you only make decisions based on science you can pretend problems aren't real without a scientist's approval, and you can pretend solutions to those problems are bad if there is even a tiny bit of counter-evidence that supports your viewpoint. And if you make decisions based on anything other than science, is the final outcome going to be any better? I admit that something that we might call "misuse of science" is a real thing, and a problem. But I don't see any solution that in any way involves de-emphasizing the importance of science in decision making.

arotical 6y ago
Here's a scenario: You are tasked with dealing with a problem where there is little research and no scientific consensus. There is plenty of video evidence and other related pieces of non-so-scientific evidence, but most people don't count that in evidence-based policy. What do you do? Sometimes, I think, we have to make decisions when there's no clear solution, and it's better than just dismissing a very obvious problem as non-existent. So yes, include science in every decision, but it's not the be all end all of society, it never has been, and it's not good to teach people that you can't tackle issues without a known best solution.
- acqq 6y ago
  > a problem where there is little research and no scientific consensus. > There is plenty of video evidence I can't even what you talk about. "Video evidence"? Edit: if I'd guess it's about "UFOs" for these there practically exists scientific consensus: no aliens proved, ever. "Video evidence" isn't the evidence of aliens.
  - domnomnom 6y ago
    Lmao, turtles all the way down my friend.
- Retric 6y ago
  In the real world problems are never completely novel. Video game addiction for example clearly only showed up after video games, but it’s also closely related to other forms of addiction. So, simply using related models and then adjusting to new information as it shows up works just fine. If government happens to ban loot boxes for a decade it’s not that important if they get it wrong and that’s plenty of time to do real research and adjust policy. PS: If you disagree try and come up with a counter example.
SpicyLemonZest 6y ago
We certainly shouldn't de-emphasize the importance of science. While not all scientific studies are correct, if someone makes a habit of saying "bah who cares what scientists think", they're gonna make bad decisions. But it's important to keep in mind that science is usually a tool to inform decisions, not a verdict to decide them. I see a lot of people declare we have to implement suchandsuch policy because that's what the science says, and almost all of them are adding implicit assumptions about values and goals to the studies they cite.
0d9eooo 6y ago
It depends on what you mean by "science." Science has become so political and subject to hype and fads, at least in biomedicine, that everyone should be deeply skeptical of every claim that is made. That doesn't mean that you should distrust everything completely, just that critical skepticism should be the norm. If by "science" you mean that "rigorous, logical empirical study" should be part of the decision-making, yes, I agree. If by "science" you mean that the statements of scientists with prestigious academic appointments should be given priority over other considerations, I say no. Outsider criticism should be welcome, and I think nonscientific considerations are sometimes required. One of the reasons society is in the mess it is in (at least in the US) is because of a failure to realize that certain segments, like the biomedical-scientific community, are just as driven by human failures as any other.

feralimal 6y ago

Even evidence might not be showing you what you think. Eg if you fund 10 studies in support of, er, say cats being dangerous to humans. 2 might find that to be true, 2 find that false, and 6 remain neutral. You can leave the 8 you that don't support what you want to show unpublished. You can then fund more studies that build on the 2 you like. Rinse and repeat. Pretty soon, driven by funding alone, you will have lots of 'evidence' that caters to your viewpoint. Is that science? Unfortunately, funding determines scientific reality.

ceejayoz 6y ago
XKCD has a great comic demonstrating how this is done. https://xkcd.com/882/ https://xkcd.com/882/

Nursie 6y ago

I don't believe that all policy should 100% have to be formed from scientific evidence, but there are places where the evidence is well researched and well understood, yet politicians legislate based on their own prejudices, directly flying in the face of both experts and evidence. In the face of this sort of behaviour, is it not reasonable to ask for more evidence-based policy? Or at least that real evidence be considered where it is available?

arotical 6y ago
yeah this is fair

DoreenMichele 6y ago

Social policy is inherently hard to get objective evidence for. Social experiments mostly don't work because the fact that it's an experiment alters the behavior of participants. This is part of why I don't think there is any value in UBI experiments. Our mental models influence the kinds of results we get. This is always true, but it's a bigger problem for some things than for others.

nitwit005 6y ago

It depends how strictly you take the idea. Often people just want some sort of demonstration that people at least looked into things and thought about the alternatives.