13 ms·

Want more policies based on evidence?

luckydata 8y ago
Coming from the Chicago School of Economics, specialized in voodoo theories that don't pan out in the real world this is pretty funny.
- rsj_hn 8y ago
  Pretty much. The Chicago school was the one pushing for financial deregulation, IIRC. I think the problem here is that we elect governments to solve collective problems, not personal problems. Collective problems arise from the interactions of large groups of people and they are generally intractable. Even if an economist tries to accurately model human behavior, it's really hard to do as you start adding more rational agents, and then you need a theory of production and a theory of expectations formation and all of a sudden what you are doing is just a toy to be tractable. So the zone where there is clear unambiguous science is very small, and it's generally not the zone of contention in politics. Stuff like "don't swallow arsenic" -- it doesn't really help us to govern or solve any of our political questions. So if we can't use science to show us how to organize ourselves, the debate turns to meta-approaches which are basically philosophies of how we approach problems. For example, a liberal might start with a Rousseau-type blank slate view of human nature and a various declaration of rights, both of which are basically statements of faith, and then try to think of some way that government can secure those rights through a period of debate, and then adopt whatever seems right to the people doing the debating, even if it means overturning long established customs. A conservative might go -- woah, there is hidden information in customs and traditions, which we may not appreciate (it's hidden) but, as an article of faith, the fact that these customs have lasted so long means that they are probably useful for _something_, and I doubt your ability to think of every unintended consequence of your policy, so I'm going to oppose it, and instead suggest you do something much smaller, as a pilot, over here for a generation, and then see what happens. And a libertarian might be led to say, "Your proposal infringes on my personal liberties, and thus, as an article of faith, must lead to bad outcomes." Etc. So the real debates we have are actually methodological debates given that we are working under imperfect information and limited knowledge about how complex interacting social systems work. Saying "we should use more science" is vacuous B.S. What we need is a better functioning public policy discourse that supports more course corrections, allows more people to air their views, and eliminates certain known distortions, like the influence of big money groups on campaigns. That's a hell of a lot more useful than pretending that scientists can be politicians or that politicians should be scientists.
  - fein 8y ago
    We need more people like yourself sowing discourse on HN. Good read, and great take on the current stream of consciousness.
  - maxxxxx 8y ago
    I agree that a lot of policy decision are value judgements and can't be fully explained by scientific approaches. But I definitely would like to see more hard numbers in political discourse. For example, what was the effect of previous tax cuts that increased deficits? Just put out the numbers: government revenue, business investment and so on. At least people (maybe) could have a rational foundation for their opinions. But I guess I am asking too much...
    - rsj_hn 8y ago
      I was horrified at the business tax cuts that we recently passed, and I was sure they wouldn't result in any real benefits for investment. But, to me, that's more of an example of money in politics and money having a big loudspeaker rather than "more science needed". Moreover, I don't think that people who voted for Trump, voted for him because they favored corporate tax cuts, but I think it was more of a protest vote. Of course, I could be wrong, but the problem is the outsized influence of lobbyists and the wealthy in our political system -- that was a Koch brothers job. I don't think conservatives love the Koch brothers, to be honest. Let's remember "science" was how the economists pushed through all the banking de-regulation and liberalization of international capital flows which I am so opposed to, as well as other disastrous reforms. For example, the sadism against Greece in the EU was led by PhDs, and many PhDs pushed for more austerity. So, for me, "more science" isn't the solution, but something like public financing of campaigns might be a better solution. I think the skepticism of technocrats is quite healthy.
      - maxxxxx 8y ago
        I guess I would like to see more data and less opinion. Especially in economics.
- refurb 8y ago
  Compared to what? Other economic schools of theory that worked out exactly as predicted?
fein 8y ago
> Who wouldn’t want science and empirical evidence to guide policy decisions. One look at crime stats and this will not be a favorable idea.
- rpedela 8y ago
  Care to elaborate?
  - sattoshi 8y ago
    I assume this is a reference to the crime per race per capita numbers.
  - deleted 8y ago
    [deleted]
  - Forge36 8y ago
    I think what OP alluded too is not understanding preexisting bias. "Where should police patrol" ->where crime is highest. Where is crime highest ->existing reports point to X neighborhood having the most arrests. Without asking "why" it may ignore That neighborhood is black, so historically racist decisions lead to extra patrolling (but no evidence of increased crime rates) which led to more arrests.
    - fein 8y ago
      I don't think there's any pre existing bias influencing Chicago crime stats. There aren't any racist decisions that created that situation. Edit: As of posting this was at -2. HN, sometimes you amaze me. For the absolute leftist bias this site attempts to maintain, you REALLY don't want to approach the question of why black on black crime is so high, while in other comparable economic situations with different ethnicities the intra-racial crime stats are much lower. It's an issue that should be discussed and not dismissed with downvotes.
      - Dylan16807 8y ago
        The claim that no racist decisions have been a cause toward crime stats in Chicago is laughable. They've had extensive effects throughout the country.
ejlangev 8y ago
This idea always struck me as fairly misguided. It of course makes sense to try to muster evidence and data when making decisions but it's not some sort of panacea. The book The Tyranny of Metrics (https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller/dp/0691174954 https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller/dp/069...) goes into the successes and failures of various attempts to use data for decision making in some detail. Found it to be an interesting read. In terms of public policy you have to decide what you're optimizing for and that decision can't be made with data alone because it does not help resolve questions of value and fairness.
- bpchaps 8y ago
  There's a middle ground. Practically no systemic analysis is done within government. At least in Chicago. Government systems are compartmentalized in ways that makes interfacing with them impossible for any worthwhile analysis. Example: the only analysis that Chicago's finance department had done on its parking tickets is a single very high level spreadsheet. Some analysis is better than no analysis.
  - ejlangev 8y ago
    > Some analysis is better than no analysis. I don't agree with that. Doing some analysis on a limited and possibly skewed data set can lull you into a false sense of understanding. It makes your ideas seems objective when they can in fact be completely baseless.
    - alexanderdmitri 8y ago
      How is no analysis just as good? It's hard to believe that the percentage of successes with no analysis is GTE the percentage of successes with some analysis.
      - ACow_Adonis 8y ago
        I presume because faulty analysis or bad data can actually get you more off track than when its just a "back of the napkin" based guess or hunch. And additionally, you now have the pride/confidence thing in your even worse results because you did "analysis"...
lacker 8y ago
I didn't understand the article's point at first because to me, medicine is not a great example of evidence-driven policies. The American health insurance system does not appear to be optimized by any sort of metric, much less to be driven by evidence. Many key decisions, like how many doctors should be given medical degrees each year, are not subject to any evidence at all. And just at a high level in terms of performance for cost something is fundamentally "not working" in medicine over the past couple decades. It's true that for the micro decisions, like does drug X treat condition Y, medicine appears to be much more evidence-based than, say, sociology. But the overall dysfunction of medical policymaking makes me dubious of using medicine as a model for other fields of decisionmaking.
- jacobush 8y ago
  It's optimised for efficient transfer of tax payer money to the industry with as little distraction as possible, such as wasting money on actual care. As with everything the USA sets out to do, it excels at it.
- rayiner 8y ago
  Even at the micro level: many childbearing related dictates (don’t drink, do breastfeed) are based on extremly weak evidence. Emotion trumps data in many cases even in medicine.
  - chaosite 8y ago
    What do you mean by "don't drink" being based on extremely weak evidence? Do you mean after the pregnancy, or while trying to conceive? Because the harms of drinking during pregnancy are well known. Fetal Alcohol Syndrome is a real thing, as are other Fetal Alcohol Spectrum disorders.
    - fosco 8y ago
      I was aware of Fetal Alcohol Syndrome but not Fetal Alcohol Spectrum Disorders [0]. Thank you for sharing. From what I ahve read the dont drink and do breast feed have fairly strong research behind them so I am a bit confused by the weak correlation comment as well. Anecdotally, while in Europe I do see it extremely commonplace for women to have a few wines while pregnant. [0] https://en.wikipedia.org/wiki/Fetal_alcohol_spectrum_disorder https://en.wikipedia.org/wiki/Fetal_alcohol_spectrum_disorde...
PeakHackerNews 8y ago
No, I don't.
openloop 8y ago
Yes.
aidenn0 8y ago
Too many people know that evidence will indicate that a policy that they don't want changed should be changed. This makes evidence gathering much, much harder.
Proven 8y ago
No, I want less policies based on common sense that government is the lease competent spender and decider.
Dowwie 8y ago
Government tends to conduct studies prior to engaging in an effort but it's questionable how much evidence from the studies actually gets used in decision making.
ada1981 8y ago
I’m concerned the medical field is what is being aspired too. The number of examples of where standard of Care deviates from empirical research is staggering. We still lose hundreds of thousands of lives a year in the US to preventable medical errors. Many hospitals still refuse to use things like simple checklists that are proven to save lives.
- avgDev 8y ago
  So this is an area of my interest. I actually hope to one day be able to improve diagnosis of certain diseases through apps. I have widespread tendinitis caused by a reaction to a medication. I have seen many physicians and ALL OF THEM are just terrible with dealing with stuff they don't have expertise in. Also, my cousin recently got diagnosed with EDS she suffered for over 20 years but they just made the diagnosis now. I cannot comprehend why doctors aren't just open when they have not diagnosed a disease with some symptoms and ask to schedule another appointment. They go type some terms into google and at least come up with a checklist or a plan to test for diseases. The worst thing is when you have done some ACTUAL RESEARCH and can even provide certain studies but they straight up don't have the will to look at it. I think many doctors are just getting burned out by the current state of the medical system in the US, which is definitely not helping. As a SE I often say, I have never done that before but I'm sure with a little time I can get it done. Medicine is supposed to be based on science but some doctors do not trust drug companies and studies anymore, as that is another shit show. There is pressure on researchers to obtain positive results.
nonbel 8y ago
"Evidence-based" seems to end up meaning practices like NHST where there is evidence involved, but the analysis is so messed up you can conclude anything you want from it. And its really funny they use "evidence-based medicine" as a positive example. Basically this is what we should expect to happen: >'“Evidence-based medicine has drifted in recent years from investigating and managing established disease to detecting and intervening in non-diseases,” Greenhalgh and colleagues wrote in BMJ in June. Furthermore, the “evidence” favoring various treatments typically comes from trials in which companies decide which drugs to test, at what doses, on how many people. Often the experimental statistical “evidence” in such studies establishes a benefit for a drug that is of little practical value; the supposed benefit, while perhaps real in a mathematical sense, is so slight as to be meaningless for real patients. In other cases, perfectly sound evidence regarding a particular disease is rendered irrelevant in patients afflicted with more than one disorder (patients who are commonly seen in medical practice, but typically excluded from the trials that produced the evidence). On top of all that, Greenhalgh and colleagues point out, the sheer volume of medical evidence makes assessing it all intelligently essentially impossible. Even the guidelines summarizing the evidence are too voluminous to be useful to doctors. “The number of [evidence based] clinical guidelines is now both unmanageable and unfathomable,” Greenhalgh and coauthors note. In one 24-hour period in 2005, for instance, a hospital in the United Kingdom admitted 18 patients with 44 diagnoses. The relevant U.K. national guidelines for those patients totaled 3,679 pages. Estimated reading time: 122 hours.'" https://www.sciencenews.org/blog/context/evidence-based-medicine-actually-isnt https://www.sciencenews.org/blog/context/evidence-based-medi...
- 08-15 8y ago
  Medicine and life sciences adopted bad statistics. Every single paper claims something along the lines of "what we observed is so incredibly unlikely (<5%) assuming complete randomness, it must mean our theory is true, our new drug works, the bad pollutant causes cancer, etc." Papers that should claim "what we observed is completely ordinary and there is nothing to see here" never get published. Even if this was done correctly (it isn't, as John Ioannidis keeps pointing out), you'd get lots of false positive results (Ioannides again). The moment you look at a collection of statistics, the only available conclusion is "They all contradict each other!!1" Until they ask physicists for help, medicine, life sciences, and many other fields will not be evidence based. We can call them "argument based", but that's not the same...
mbrodersen 8y ago
Politics is mostly about power and ideology. Not truth/evidence/practicality. Even if people agree on the evidence they will still disagree on what to do about it.

Coming from the Chicago School of Economics, specialized in voodoo theories that don't pan out in the real world this is pretty funny.

rsj_hn 8y ago
Pretty much. The Chicago school was the one pushing for financial deregulation, IIRC. I think the problem here is that we elect governments to solve collective problems, not personal problems. Collective problems arise from the interactions of large groups of people and they are generally intractable. Even if an economist tries to accurately model human behavior, it's really hard to do as you start adding more rational agents, and then you need a theory of production and a theory of expectations formation and all of a sudden what you are doing is just a toy to be tractable. So the zone where there is clear unambiguous science is very small, and it's generally not the zone of contention in politics. Stuff like "don't swallow arsenic" -- it doesn't really help us to govern or solve any of our political questions. So if we can't use science to show us how to organize ourselves, the debate turns to meta-approaches which are basically philosophies of how we approach problems. For example, a liberal might start with a Rousseau-type blank slate view of human nature and a various declaration of rights, both of which are basically statements of faith, and then try to think of some way that government can secure those rights through a period of debate, and then adopt whatever seems right to the people doing the debating, even if it means overturning long established customs. A conservative might go -- woah, there is hidden information in customs and traditions, which we may not appreciate (it's hidden) but, as an article of faith, the fact that these customs have lasted so long means that they are probably useful for _something_, and I doubt your ability to think of every unintended consequence of your policy, so I'm going to oppose it, and instead suggest you do something much smaller, as a pilot, over here for a generation, and then see what happens. And a libertarian might be led to say, "Your proposal infringes on my personal liberties, and thus, as an article of faith, must lead to bad outcomes." Etc. So the real debates we have are actually methodological debates given that we are working under imperfect information and limited knowledge about how complex interacting social systems work. Saying "we should use more science" is vacuous B.S. What we need is a better functioning public policy discourse that supports more course corrections, allows more people to air their views, and eliminates certain known distortions, like the influence of big money groups on campaigns. That's a hell of a lot more useful than pretending that scientists can be politicians or that politicians should be scientists.
- fein 8y ago
  We need more people like yourself sowing discourse on HN. Good read, and great take on the current stream of consciousness.
- maxxxxx 8y ago
  I agree that a lot of policy decision are value judgements and can't be fully explained by scientific approaches. But I definitely would like to see more hard numbers in political discourse. For example, what was the effect of previous tax cuts that increased deficits? Just put out the numbers: government revenue, business investment and so on. At least people (maybe) could have a rational foundation for their opinions. But I guess I am asking too much...
  - rsj_hn 8y ago
    I was horrified at the business tax cuts that we recently passed, and I was sure they wouldn't result in any real benefits for investment. But, to me, that's more of an example of money in politics and money having a big loudspeaker rather than "more science needed". Moreover, I don't think that people who voted for Trump, voted for him because they favored corporate tax cuts, but I think it was more of a protest vote. Of course, I could be wrong, but the problem is the outsized influence of lobbyists and the wealthy in our political system -- that was a Koch brothers job. I don't think conservatives love the Koch brothers, to be honest. Let's remember "science" was how the economists pushed through all the banking de-regulation and liberalization of international capital flows which I am so opposed to, as well as other disastrous reforms. For example, the sadism against Greece in the EU was led by PhDs, and many PhDs pushed for more austerity. So, for me, "more science" isn't the solution, but something like public financing of campaigns might be a better solution. I think the skepticism of technocrats is quite healthy.
    - maxxxxx 8y ago
      I guess I would like to see more data and less opinion. Especially in economics.
refurb 8y ago
Compared to what? Other economic schools of theory that worked out exactly as predicted?

fein 8y ago

> Who wouldn’t want science and empirical evidence to guide policy decisions. One look at crime stats and this will not be a favorable idea.

rpedela 8y ago
Care to elaborate?
- sattoshi 8y ago
  I assume this is a reference to the crime per race per capita numbers.
- deleted 8y ago
  [deleted]
- Forge36 8y ago
  I think what OP alluded too is not understanding preexisting bias. "Where should police patrol" ->where crime is highest. Where is crime highest ->existing reports point to X neighborhood having the most arrests. Without asking "why" it may ignore That neighborhood is black, so historically racist decisions lead to extra patrolling (but no evidence of increased crime rates) which led to more arrests.
  - fein 8y ago
    I don't think there's any pre existing bias influencing Chicago crime stats. There aren't any racist decisions that created that situation. Edit: As of posting this was at -2. HN, sometimes you amaze me. For the absolute leftist bias this site attempts to maintain, you REALLY don't want to approach the question of why black on black crime is so high, while in other comparable economic situations with different ethnicities the intra-racial crime stats are much lower. It's an issue that should be discussed and not dismissed with downvotes.
    - Dylan16807 8y ago
      The claim that no racist decisions have been a cause toward crime stats in Chicago is laughable. They've had extensive effects throughout the country.

ejlangev 8y ago

This idea always struck me as fairly misguided. It of course makes sense to try to muster evidence and data when making decisions but it's not some sort of panacea. The book The Tyranny of Metrics (https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller/dp/0691174954 https://www.amazon.com/Tyranny-Metrics-Jerry-Z-Muller/dp/069...) goes into the successes and failures of various attempts to use data for decision making in some detail. Found it to be an interesting read. In terms of public policy you have to decide what you're optimizing for and that decision can't be made with data alone because it does not help resolve questions of value and fairness.

bpchaps 8y ago
There's a middle ground. Practically no systemic analysis is done within government. At least in Chicago. Government systems are compartmentalized in ways that makes interfacing with them impossible for any worthwhile analysis. Example: the only analysis that Chicago's finance department had done on its parking tickets is a single very high level spreadsheet. Some analysis is better than no analysis.
- ejlangev 8y ago
  > Some analysis is better than no analysis. I don't agree with that. Doing some analysis on a limited and possibly skewed data set can lull you into a false sense of understanding. It makes your ideas seems objective when they can in fact be completely baseless.
  - alexanderdmitri 8y ago
    How is no analysis just as good? It's hard to believe that the percentage of successes with no analysis is GTE the percentage of successes with some analysis.
    - ACow_Adonis 8y ago
      I presume because faulty analysis or bad data can actually get you more off track than when its just a "back of the napkin" based guess or hunch. And additionally, you now have the pride/confidence thing in your even worse results because you did "analysis"...

lacker 8y ago

I didn't understand the article's point at first because to me, medicine is not a great example of evidence-driven policies. The American health insurance system does not appear to be optimized by any sort of metric, much less to be driven by evidence. Many key decisions, like how many doctors should be given medical degrees each year, are not subject to any evidence at all. And just at a high level in terms of performance for cost something is fundamentally "not working" in medicine over the past couple decades. It's true that for the micro decisions, like does drug X treat condition Y, medicine appears to be much more evidence-based than, say, sociology. But the overall dysfunction of medical policymaking makes me dubious of using medicine as a model for other fields of decisionmaking.

jacobush 8y ago
It's optimised for efficient transfer of tax payer money to the industry with as little distraction as possible, such as wasting money on actual care. As with everything the USA sets out to do, it excels at it.
rayiner 8y ago
Even at the micro level: many childbearing related dictates (don’t drink, do breastfeed) are based on extremly weak evidence. Emotion trumps data in many cases even in medicine.
- chaosite 8y ago
  What do you mean by "don't drink" being based on extremely weak evidence? Do you mean after the pregnancy, or while trying to conceive? Because the harms of drinking during pregnancy are well known. Fetal Alcohol Syndrome is a real thing, as are other Fetal Alcohol Spectrum disorders.
  - fosco 8y ago
    I was aware of Fetal Alcohol Syndrome but not Fetal Alcohol Spectrum Disorders [0]. Thank you for sharing. From what I ahve read the dont drink and do breast feed have fairly strong research behind them so I am a bit confused by the weak correlation comment as well. Anecdotally, while in Europe I do see it extremely commonplace for women to have a few wines while pregnant. [0] https://en.wikipedia.org/wiki/Fetal_alcohol_spectrum_disorder https://en.wikipedia.org/wiki/Fetal_alcohol_spectrum_disorde...

PeakHackerNews 8y ago

No, I don't.

openloop 8y ago

Yes.

aidenn0 8y ago

Too many people know that evidence will indicate that a policy that they don't want changed should be changed. This makes evidence gathering much, much harder.

Proven 8y ago

No, I want less policies based on common sense that government is the lease competent spender and decider.

Dowwie 8y ago

Government tends to conduct studies prior to engaging in an effort but it's questionable how much evidence from the studies actually gets used in decision making.

ada1981 8y ago

I’m concerned the medical field is what is being aspired too. The number of examples of where standard of Care deviates from empirical research is staggering. We still lose hundreds of thousands of lives a year in the US to preventable medical errors. Many hospitals still refuse to use things like simple checklists that are proven to save lives.

avgDev 8y ago
So this is an area of my interest. I actually hope to one day be able to improve diagnosis of certain diseases through apps. I have widespread tendinitis caused by a reaction to a medication. I have seen many physicians and ALL OF THEM are just terrible with dealing with stuff they don't have expertise in. Also, my cousin recently got diagnosed with EDS she suffered for over 20 years but they just made the diagnosis now. I cannot comprehend why doctors aren't just open when they have not diagnosed a disease with some symptoms and ask to schedule another appointment. They go type some terms into google and at least come up with a checklist or a plan to test for diseases. The worst thing is when you have done some ACTUAL RESEARCH and can even provide certain studies but they straight up don't have the will to look at it. I think many doctors are just getting burned out by the current state of the medical system in the US, which is definitely not helping. As a SE I often say, I have never done that before but I'm sure with a little time I can get it done. Medicine is supposed to be based on science but some doctors do not trust drug companies and studies anymore, as that is another shit show. There is pressure on researchers to obtain positive results.

nonbel 8y ago

"Evidence-based" seems to end up meaning practices like NHST where there is evidence involved, but the analysis is so messed up you can conclude anything you want from it. And its really funny they use "evidence-based medicine" as a positive example. Basically this is what we should expect to happen: >'“Evidence-based medicine has drifted in recent years from investigating and managing established disease to detecting and intervening in non-diseases,” Greenhalgh and colleagues wrote in BMJ in June. Furthermore, the “evidence” favoring various treatments typically comes from trials in which companies decide which drugs to test, at what doses, on how many people. Often the experimental statistical “evidence” in such studies establishes a benefit for a drug that is of little practical value; the supposed benefit, while perhaps real in a mathematical sense, is so slight as to be meaningless for real patients. In other cases, perfectly sound evidence regarding a particular disease is rendered irrelevant in patients afflicted with more than one disorder (patients who are commonly seen in medical practice, but typically excluded from the trials that produced the evidence). On top of all that, Greenhalgh and colleagues point out, the sheer volume of medical evidence makes assessing it all intelligently essentially impossible. Even the guidelines summarizing the evidence are too voluminous to be useful to doctors. “The number of [evidence based] clinical guidelines is now both unmanageable and unfathomable,” Greenhalgh and coauthors note. In one 24-hour period in 2005, for instance, a hospital in the United Kingdom admitted 18 patients with 44 diagnoses. The relevant U.K. national guidelines for those patients totaled 3,679 pages. Estimated reading time: 122 hours.'" https://www.sciencenews.org/blog/context/evidence-based-medicine-actually-isnt https://www.sciencenews.org/blog/context/evidence-based-medi...

08-15 8y ago
Medicine and life sciences adopted bad statistics. Every single paper claims something along the lines of "what we observed is so incredibly unlikely (<5%) assuming complete randomness, it must mean our theory is true, our new drug works, the bad pollutant causes cancer, etc." Papers that should claim "what we observed is completely ordinary and there is nothing to see here" never get published. Even if this was done correctly (it isn't, as John Ioannidis keeps pointing out), you'd get lots of false positive results (Ioannides again). The moment you look at a collection of statistics, the only available conclusion is "They all contradict each other!!1" Until they ask physicists for help, medicine, life sciences, and many other fields will not be evidence based. We can call them "argument based", but that's not the same...

mbrodersen 8y ago

Politics is mostly about power and ideology. Not truth/evidence/practicality. Even if people agree on the evidence they will still disagree on what to do about it.