8 ms·

Why Is So Much Reported Science Wrong, and What Can Fix That?

109 points by tokenadult 11y ago

kelukelugames 11y ago
I am more cynical after seeing people I agree with cite tenuous studies as facts. All of social science is sketchy, especially studies with results that make you feel good. For example, my friend and I read an Economist article that said 10% of American doctors are Muslim. We could not verify the source, but people are going to believe it's true because it was in the Economist.
- danharaj 11y ago
  http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1490160/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1490160/ Apparently ~2.7% as measured by this study in 2005. Surprised I could not find department of labor statistics on the question. I would say that it is very possible that the proportion has increased drastically since 2005: Many of the people who have fled Iraq and Syria since then are skilled professionals including doctors.
  - kelukelugames 11y ago
    There are about 1 million doctors in the US and 20k new doctors every year. In order to go from 2.7% to 10% in 10 years then that means ~40% of all new doctors from 2005 to 2015 were Muslim. As much as I hate Trump, I don't think this is possible.
    - danieltillett 11y ago
      I find it hard to believe that there are only 20K new doctors a year if there are 1 million doctors. This would imply that the turn over time for doctors is at least 50 years and in all likelihood greater given the growth in total doctor numbers over the last 50 years. This is just not plausible.
      - kelukelugames 11y ago
        You can easily google for stats. I am happy to see different numbers.
        danieltillett 11y ago
        I am not doubting what you are telling us is what you have found, just that the numbers have to be wrong as it is not plausible that there is only 2% turnover in doctors each year.
      - maxerickson 11y ago
        This says 767,000 and ~29,000: https://www.aamc.org/download/426242/data/ihsreportdownload.pdf https://www.aamc.org/download/426242/data/ihsreportdownload.... So that takes the coarse estimate down to 25 years.
        danieltillett 11y ago
        This is at least a plausible number. While we still have not established if the 10% number has any basis in fact, you would only need 19.3% of new doctors to be muslim to shift the numbers. This still seems high to me, but it is possible.
- pron 11y ago
  > All of social science is sketchy Not any more than any other science that relies on sampling humans (I'm looking at medicine). Both start yielding stronger results when they explore the effect of mechanisms that are already known, and have a bad record of discovering "surprising" effects. The antidote is simple: never rely on a single study, or even two.
- iak8god 11y ago
  > All of social science is sketchy... We could not verify the source, but people are going to believe it's true because it was in the Economist. It seems like your real beef here is with journalism (even the higher-than-average quality journalism that normally appears in the Economist). The fact that some news magazine doesn't include citations for claims in an article[1] has nothing to do with the sketchiness (or non-sketchiness) of "social science." It is too bad though that the Economist doesn't provide citations for claims like this. Did they pull "10%" out of thin air? Are 10% of new medical doctorates currently earned by Muslims? Did they include all kinds of doctorates (PhD, etc) in the calculation? A simple link to the source for this number would clear that all up... [1] http://www.economist.com/news/united-states/21679823-despite-attack-san-bernardino-americas-defences-against-jihadism-are-high http://www.economist.com/news/united-states/21679823-despite...
  - kelukelugames 11y ago
    Both. Paper: "We put three women and three men in a room for 10 minutes and they brainstormed 1.3 extra ideas than a room of 6 men." Article: "Study proves gender balance leads to better meetings!"
    - iak8god 11y ago
      Amusing, but I don't quite agree with this characterization of "all of social science." But here's something interesting: I've tracked down a source for the 10% claim, and it's absolutely bonkers: * An editorial[1] in the Detroit Free Press makes the following claim: | "an analysis of statistics provided by the American Medical Association indicates that 10% of all American physicians are Muslims" That certainly makes it sound like the AMA thinks that 10% of American physicians are Muslims. * The source provided for this claim, "Muslim Doctors Abundant, But Muslim Hospitals Non-Existent"[2] is a 2008 post on a site called The Muslim Link. Here's how that post arrived at the 10% claim: * 113,585 Physicians in the US (2006) * 7,000 current and retired physicians are members of the Association of Physicians of Pakistani Descent of North America * 5.9% of African Americans are Muslim; 3.5% of physicians are African American; therefore there are 2000 African American Muslim physicians * 7000 + 2000 = 9000; 9000 is just about 10% of 113,585 I shit you not, that's the most straightforward reading of the analysis at [2]. Now it's in the Economist. I seriously hope they got that number somewhere else. [1] http://www.freep.com/article/20140128/OPINION05/301280121/dave-agema-racism-gay-remarks-muslim-republican-gop-michigan http://www.freep.com/article/20140128/OPINION05/301280121/da... [2] http://www.muslimlinkpaper.com/myjumla/index.php?option=com_content&view=article&id=1440 http://www.muslimlinkpaper.com/myjumla/index.php?option=com_...
bloaf 11y ago
It seems to me that the problems with science as it is done today are open secrets, and can essentially be summarized as: 1: Publication pressure. There is too much emphasis on creating publications. The end result is that people rush out unfinished work, or end up diluting their findings by unnecessarily spreading them across several papers. People are also discouraged from undertaking projects that may take a long time to "pay off" with a publication. 2: Scientific networking. In scientific fields, people get to know each other, and those relationships still heavily influence the process. Fame and reputation matter when getting published, not to mention friendships with publishers or reviewers. 3: Bias towards excitement/positive results. This, I believe, is not entirely the fault of the scientific community. If left to their own devices, I think there would be plenty of scientists willing to double check old findings, or publish negative results. However, it's hard to convince people to pay you or employ you if your work is perceived as "boring," "negative," or as some sort of failure. There are a handful of smaller issues (e.g. excessive emphasis on "impact factor") but I suspect that they are mostly symptoms of the above issues. I've always believed that the whole "publication model of science" is due for review. It seems to be a product of the 18th century rather than something we'd decide was a good idea today. If we do it right, I feel like the positive results bias would immediately go away. I don't think we can "fix" issue #2 in any really meaningful way. Even if you were to take people's names off of publications, they would still chat with each other at meetings or through collaborations. I think the best thing to do is not try to prevent people from influencing each other, but simply to make the whole process more transparent.
- danieltillett 11y ago
  The problem is not the publication model, it is the incentive structure that encourages scientists to rush out publications without checking their data in enough detail, or in putting them under so much pressure to generate “exciting” results that they falsify the results. I will share an anecdote about a paper I was involved in. The work was done in collaboration with another lab within my school. One of the results suggested that one of the instrument was out of alignment and possibly giving false results. I raised this with the head of the other lab and after speaking with his post doc he said that everything was fine. I asked to see the calibration data (there was none), but I was assured that everything was fine and they were going to submit it as it was. At that point I went thermonuclear and said if this paper was submitted without the dubious values being checked I would write to the journal asking for it to be withdrawn. This caused a huge ruckus within the school including visits from the head of school and the dean telling me I was being “unreasonable”. I stood my ground and the post doc eventually ran the calibration experiments which showed the instrument was out of alignment and the results were wrong. Of course this burned a lot of bridges for me and it would have been better for me to just shut up and let the paper go out wrong.
peter303 11y ago
Most journalists post email addresses. I drop them a note when I clearly see a need to improve. But most of the time they dont apply the suggestion. Therefore in any publication I sometime prejudge an article by the history of the author.
danieltillett 11y ago
As with everything to do with smart people it is all about incentives. If you create a system where accuracy of results don’t matter and where you must publish or die then don’t be surprised if what you get are lots of crap publications with dubious or false results. Hoping for a different outcome without changing the incentives is fantasy thinking. The positive is there is lots we could do to change the incentives, but there are some powerful forces benefiting from the current structure. Edit. I should actual say what we could do. We need to focus on the incentives rather than the problem. Fix the incentives and the problem goes away on its own, focus on the problem and you just end up shifting it to somewhere else - the "squeezing on the balloon" effect. The best solution in my opinion is to move from our crude "most publications = funding" model to a hurdle + lottery model. With this you have to publish enough to prove that you are capable of doing good research (the hurdle) and once you have done this you go into a lottery from which we pull out grant winners until we have used up all the funding available. The reason this idea is not popular is it would not work in favour of the current grant winners.
- deleted 11y ago
  [deleted]
- OopsCriticality 11y ago
  Are you aware of any implementations of that lottery model in the wild?
  - danieltillett 11y ago
    No I am not. It gets brought up every so often as the most rational approach, but those who have climbed the greasy pole of the current system, and hence control it, are none to keen to change. The usual argument against a lottery is it does not reward the best scientists and will give grants to third rate hacks. This is true, but it can be solved by having different lotteries with different thresholds. Publish one paper in the Journal of Useless & Pointless Results and you go into Pool A where you might have a 0.1% of getting a grant. Publish 20 articles in Nature and Science and you go into Pool H where you have a 75% chance. We would just need to be careful that we are not recreating the same perverse incentives that the current system encourages, but this is not insurmountable.
EvanPlaice 11y ago
"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." - Goodhart's Law Awarding scientists for publishing works -- whether or not they've passed rigor -- is the mistake. The monetary system for awarding grants is completely broken. Nobody wants to admit they have their hand in the cookie jar for fear of having the lid slammed down on their hand. Universities take a cut of the grant money to subsidize their staff payroll. Professors gain notoriety and a significant increase in income when their published work leads to being awarded a grant. Industry/NGOs have their own financial incentives to 'influence' scientific reporting that favors their own bias'. Nobody calls attention to the corruption because... Woohoo! Free money! I find it very difficult to trust any of the scientific reporting related to current affairs and/or politics. The incentives for 'bad actors' who commit intellectual dishonesty are too high.
deleted 11y ago
[deleted]
- danieltillett 11y ago
  You just need to use the 3-fold rule. If the study wasn’t double blinded and the effect is not at least 3-fold then it is most likely to not hold up over time.
davidgerard 11y ago
Speaking to scientists I know, quite a lot of the problem is university press offices. The bullshit articles start as bullshit press releases, bearing little resemblance to any actual facts in the original paper (or arXiv preprint). Not sure of a way around this one.
- danieltillett 11y ago
  The solution is requiring the scientist to approve the press release before it goes out and having the power to force errors to be corrected.
  - davidgerard 11y ago
    Well yes, but this ignores all power relations and incentives involved. The press offices aren't sending out BS press releases for the sake of it, they're sending them out to get the university's name positive press; and the university has all the power over the scientists.
- lvs 11y ago
  This is accurate. The corporatization of academics is a significant part of the problem.
- capnrefsmmat 11y ago
  There is research backing this up: http://www.bmj.com/content/349/bmj.g7015 http://www.bmj.com/content/349/bmj.g7015 They conclude that "Exaggeration in news is strongly associated with exaggeration in press releases. Improving the accuracy of academic press releases could represent a key opportunity for reducing misleading health related news."
dools 11y ago
Beautiful irony: the only part of the site that doesn't work well on my old android browser is the popup telling me my browser is out of date.
melted 11y ago
Easy to fix for a lot of comp sci papers. Require that to be accepted into a conference the papers must have a working, open source implementation that confirms the results, preferably in the form of a Docker container(s) or a VM image(s). Publish papers with reproducible results alongside.
badwolf93 11y ago
if science is wrong then science moves is not that the goal of science?
n0us 11y ago
> Science and journalism seem to be uniquely incompatible: Where journalism favors neat story arcs, science progresses jerkily, with false starts and misdirections in a long, uneven path to the truth—or at least to scientific consensus. The types of stories that reporters choose to pursue can also be a problem, says Peter Aldhous, a teacher of investigative reporting at UC Santa Cruz’s Science Communication Program, lecturer at Berkeley’s Graduate School of Journalism, and science reporter for BuzzFeed. Seeing that someone from Buzz Feed has this kind of sober non-delusional perspective on journalism restores my faith in the idea that the internet might not have ruined reporting. I used to avoid Buzz Feed like a disease, a cess pool of listicles and ADD style click bait but it seems they have a real goal of producing quality content. Respect.
- danieltillett 11y ago
  Very subtle.
- fbbbbb 11y ago
  One intelligent mind in the combine that is BuzzFeed, doesn't account for the horde of mindless drones who write a huge majority of click-bait material. I don't see where do you see the support for the idea that their goal is quality articles.
jackcosgrove 11y ago
Scientists are seen by lots of people as secular priests or shamans. In this respect science is a victim of its own success. It now commands the attention of a lot of people who expect it to perform some of the social functions that religion used to play. "Science" is more likely to be hijacked by charlatans because of this totemic value.
- pron 11y ago
  That's very interesting. I think people need answers from the "answer authority", and once that authority is established, it is treated reverently and pressed to provide answers. As the "priests" are humans, too, there's no reason to believe they themselves are immune to this effect, even when they know how the sausage is made.
jhoechtl 11y ago
1. To many scientists treating correlation for causality 2. Pressure to publish (requirement of funded projects or because of organisations target objectives) 3. Too many scientist
wavefunction 11y ago
"Why Is So Much Reported Science Incorrect," There's my contribution. This is from an illustrious institution like Berkeley?
J_Darnley 11y ago
Because journalists and audiences are incompetent. Journalists only get a press release from Big Corporation which is worded to give the best impression possible. The journalists then need to dumb it down for the masses. Then when Adam tells Bob he only repeats what he remembers and understands. In a massive game of Chinese whispers it is no wonder that "So Much Reported Science Wrong".
jwatte 11y ago
Bias against negative results are a problem. Frequentist "p values" are another, especially when samples are small and biased ("20 undergraduate students") and 0.05 is considered "significant." Now, the system is still designed to correct itself for errors over time, and reinforcing this design is the most important meta-thing we can do. So it's not all doom and gloom. It's just that "over time" is a longer and messier horizon than we would like!

109 points by tokenadult 11y ago

I am more cynical after seeing people I agree with cite tenuous studies as facts. All of social science is sketchy, especially studies with results that make you feel good. For example, my friend and I read an Economist article that said 10% of American doctors are Muslim. We could not verify the source, but people are going to believe it's true because it was in the Economist.

danharaj 11y ago
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1490160/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1490160/ Apparently ~2.7% as measured by this study in 2005. Surprised I could not find department of labor statistics on the question. I would say that it is very possible that the proportion has increased drastically since 2005: Many of the people who have fled Iraq and Syria since then are skilled professionals including doctors.
- kelukelugames 11y ago
  There are about 1 million doctors in the US and 20k new doctors every year. In order to go from 2.7% to 10% in 10 years then that means ~40% of all new doctors from 2005 to 2015 were Muslim. As much as I hate Trump, I don't think this is possible.
  - danieltillett 11y ago
    I find it hard to believe that there are only 20K new doctors a year if there are 1 million doctors. This would imply that the turn over time for doctors is at least 50 years and in all likelihood greater given the growth in total doctor numbers over the last 50 years. This is just not plausible.
    - kelukelugames 11y ago
      You can easily google for stats. I am happy to see different numbers.
      - danieltillett 11y ago
        I am not doubting what you are telling us is what you have found, just that the numbers have to be wrong as it is not plausible that there is only 2% turnover in doctors each year.
    - maxerickson 11y ago
      This says 767,000 and ~29,000: https://www.aamc.org/download/426242/data/ihsreportdownload.pdf https://www.aamc.org/download/426242/data/ihsreportdownload.... So that takes the coarse estimate down to 25 years.
      - danieltillett 11y ago
        This is at least a plausible number. While we still have not established if the 10% number has any basis in fact, you would only need 19.3% of new doctors to be muslim to shift the numbers. This still seems high to me, but it is possible.
pron 11y ago
> All of social science is sketchy Not any more than any other science that relies on sampling humans (I'm looking at medicine). Both start yielding stronger results when they explore the effect of mechanisms that are already known, and have a bad record of discovering "surprising" effects. The antidote is simple: never rely on a single study, or even two.
iak8god 11y ago
> All of social science is sketchy... We could not verify the source, but people are going to believe it's true because it was in the Economist. It seems like your real beef here is with journalism (even the higher-than-average quality journalism that normally appears in the Economist). The fact that some news magazine doesn't include citations for claims in an article[1] has nothing to do with the sketchiness (or non-sketchiness) of "social science." It is too bad though that the Economist doesn't provide citations for claims like this. Did they pull "10%" out of thin air? Are 10% of new medical doctorates currently earned by Muslims? Did they include all kinds of doctorates (PhD, etc) in the calculation? A simple link to the source for this number would clear that all up... [1] http://www.economist.com/news/united-states/21679823-despite-attack-san-bernardino-americas-defences-against-jihadism-are-high http://www.economist.com/news/united-states/21679823-despite...
- kelukelugames 11y ago
  Both. Paper: "We put three women and three men in a room for 10 minutes and they brainstormed 1.3 extra ideas than a room of 6 men." Article: "Study proves gender balance leads to better meetings!"
  - iak8god 11y ago
    Amusing, but I don't quite agree with this characterization of "all of social science." But here's something interesting: I've tracked down a source for the 10% claim, and it's absolutely bonkers: * An editorial[1] in the Detroit Free Press makes the following claim: | "an analysis of statistics provided by the American Medical Association indicates that 10% of all American physicians are Muslims" That certainly makes it sound like the AMA thinks that 10% of American physicians are Muslims. * The source provided for this claim, "Muslim Doctors Abundant, But Muslim Hospitals Non-Existent"[2] is a 2008 post on a site called The Muslim Link. Here's how that post arrived at the 10% claim: * 113,585 Physicians in the US (2006) * 7,000 current and retired physicians are members of the Association of Physicians of Pakistani Descent of North America * 5.9% of African Americans are Muslim; 3.5% of physicians are African American; therefore there are 2000 African American Muslim physicians * 7000 + 2000 = 9000; 9000 is just about 10% of 113,585 I shit you not, that's the most straightforward reading of the analysis at [2]. Now it's in the Economist. I seriously hope they got that number somewhere else. [1] http://www.freep.com/article/20140128/OPINION05/301280121/dave-agema-racism-gay-remarks-muslim-republican-gop-michigan http://www.freep.com/article/20140128/OPINION05/301280121/da... [2] http://www.muslimlinkpaper.com/myjumla/index.php?option=com_content&view=article&id=1440 http://www.muslimlinkpaper.com/myjumla/index.php?option=com_...

bloaf 11y ago

It seems to me that the problems with science as it is done today are open secrets, and can essentially be summarized as: 1: Publication pressure. There is too much emphasis on creating publications. The end result is that people rush out unfinished work, or end up diluting their findings by unnecessarily spreading them across several papers. People are also discouraged from undertaking projects that may take a long time to "pay off" with a publication. 2: Scientific networking. In scientific fields, people get to know each other, and those relationships still heavily influence the process. Fame and reputation matter when getting published, not to mention friendships with publishers or reviewers. 3: Bias towards excitement/positive results. This, I believe, is not entirely the fault of the scientific community. If left to their own devices, I think there would be plenty of scientists willing to double check old findings, or publish negative results. However, it's hard to convince people to pay you or employ you if your work is perceived as "boring," "negative," or as some sort of failure. There are a handful of smaller issues (e.g. excessive emphasis on "impact factor") but I suspect that they are mostly symptoms of the above issues. I've always believed that the whole "publication model of science" is due for review. It seems to be a product of the 18th century rather than something we'd decide was a good idea today. If we do it right, I feel like the positive results bias would immediately go away. I don't think we can "fix" issue #2 in any really meaningful way. Even if you were to take people's names off of publications, they would still chat with each other at meetings or through collaborations. I think the best thing to do is not try to prevent people from influencing each other, but simply to make the whole process more transparent.

danieltillett 11y ago
The problem is not the publication model, it is the incentive structure that encourages scientists to rush out publications without checking their data in enough detail, or in putting them under so much pressure to generate “exciting” results that they falsify the results. I will share an anecdote about a paper I was involved in. The work was done in collaboration with another lab within my school. One of the results suggested that one of the instrument was out of alignment and possibly giving false results. I raised this with the head of the other lab and after speaking with his post doc he said that everything was fine. I asked to see the calibration data (there was none), but I was assured that everything was fine and they were going to submit it as it was. At that point I went thermonuclear and said if this paper was submitted without the dubious values being checked I would write to the journal asking for it to be withdrawn. This caused a huge ruckus within the school including visits from the head of school and the dean telling me I was being “unreasonable”. I stood my ground and the post doc eventually ran the calibration experiments which showed the instrument was out of alignment and the results were wrong. Of course this burned a lot of bridges for me and it would have been better for me to just shut up and let the paper go out wrong.

peter303 11y ago

Most journalists post email addresses. I drop them a note when I clearly see a need to improve. But most of the time they dont apply the suggestion. Therefore in any publication I sometime prejudge an article by the history of the author.

danieltillett 11y ago

As with everything to do with smart people it is all about incentives. If you create a system where accuracy of results don’t matter and where you must publish or die then don’t be surprised if what you get are lots of crap publications with dubious or false results. Hoping for a different outcome without changing the incentives is fantasy thinking. The positive is there is lots we could do to change the incentives, but there are some powerful forces benefiting from the current structure. Edit. I should actual say what we could do. We need to focus on the incentives rather than the problem. Fix the incentives and the problem goes away on its own, focus on the problem and you just end up shifting it to somewhere else - the "squeezing on the balloon" effect. The best solution in my opinion is to move from our crude "most publications = funding" model to a hurdle + lottery model. With this you have to publish enough to prove that you are capable of doing good research (the hurdle) and once you have done this you go into a lottery from which we pull out grant winners until we have used up all the funding available. The reason this idea is not popular is it would not work in favour of the current grant winners.

deleted 11y ago
[deleted]
OopsCriticality 11y ago
Are you aware of any implementations of that lottery model in the wild?
- danieltillett 11y ago
  No I am not. It gets brought up every so often as the most rational approach, but those who have climbed the greasy pole of the current system, and hence control it, are none to keen to change. The usual argument against a lottery is it does not reward the best scientists and will give grants to third rate hacks. This is true, but it can be solved by having different lotteries with different thresholds. Publish one paper in the Journal of Useless & Pointless Results and you go into Pool A where you might have a 0.1% of getting a grant. Publish 20 articles in Nature and Science and you go into Pool H where you have a 75% chance. We would just need to be careful that we are not recreating the same perverse incentives that the current system encourages, but this is not insurmountable.

EvanPlaice 11y ago

"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." - Goodhart's Law Awarding scientists for publishing works -- whether or not they've passed rigor -- is the mistake. The monetary system for awarding grants is completely broken. Nobody wants to admit they have their hand in the cookie jar for fear of having the lid slammed down on their hand. Universities take a cut of the grant money to subsidize their staff payroll. Professors gain notoriety and a significant increase in income when their published work leads to being awarded a grant. Industry/NGOs have their own financial incentives to 'influence' scientific reporting that favors their own bias'. Nobody calls attention to the corruption because... Woohoo! Free money! I find it very difficult to trust any of the scientific reporting related to current affairs and/or politics. The incentives for 'bad actors' who commit intellectual dishonesty are too high.

deleted 11y ago

[deleted]

danieltillett 11y ago
You just need to use the 3-fold rule. If the study wasn’t double blinded and the effect is not at least 3-fold then it is most likely to not hold up over time.

davidgerard 11y ago

Speaking to scientists I know, quite a lot of the problem is university press offices. The bullshit articles start as bullshit press releases, bearing little resemblance to any actual facts in the original paper (or arXiv preprint). Not sure of a way around this one.

danieltillett 11y ago
The solution is requiring the scientist to approve the press release before it goes out and having the power to force errors to be corrected.
- davidgerard 11y ago
  Well yes, but this ignores all power relations and incentives involved. The press offices aren't sending out BS press releases for the sake of it, they're sending them out to get the university's name positive press; and the university has all the power over the scientists.
lvs 11y ago
This is accurate. The corporatization of academics is a significant part of the problem.
capnrefsmmat 11y ago
There is research backing this up: http://www.bmj.com/content/349/bmj.g7015 http://www.bmj.com/content/349/bmj.g7015 They conclude that "Exaggeration in news is strongly associated with exaggeration in press releases. Improving the accuracy of academic press releases could represent a key opportunity for reducing misleading health related news."

dools 11y ago

Beautiful irony: the only part of the site that doesn't work well on my old android browser is the popup telling me my browser is out of date.

melted 11y ago

Easy to fix for a lot of comp sci papers. Require that to be accepted into a conference the papers must have a working, open source implementation that confirms the results, preferably in the form of a Docker container(s) or a VM image(s). Publish papers with reproducible results alongside.

badwolf93 11y ago

if science is wrong then science moves is not that the goal of science?

n0us 11y ago

> Science and journalism seem to be uniquely incompatible: Where journalism favors neat story arcs, science progresses jerkily, with false starts and misdirections in a long, uneven path to the truth—or at least to scientific consensus. The types of stories that reporters choose to pursue can also be a problem, says Peter Aldhous, a teacher of investigative reporting at UC Santa Cruz’s Science Communication Program, lecturer at Berkeley’s Graduate School of Journalism, and science reporter for BuzzFeed. Seeing that someone from Buzz Feed has this kind of sober non-delusional perspective on journalism restores my faith in the idea that the internet might not have ruined reporting. I used to avoid Buzz Feed like a disease, a cess pool of listicles and ADD style click bait but it seems they have a real goal of producing quality content. Respect.

danieltillett 11y ago
Very subtle.
fbbbbb 11y ago
One intelligent mind in the combine that is BuzzFeed, doesn't account for the horde of mindless drones who write a huge majority of click-bait material. I don't see where do you see the support for the idea that their goal is quality articles.

jackcosgrove 11y ago

Scientists are seen by lots of people as secular priests or shamans. In this respect science is a victim of its own success. It now commands the attention of a lot of people who expect it to perform some of the social functions that religion used to play. "Science" is more likely to be hijacked by charlatans because of this totemic value.

pron 11y ago
That's very interesting. I think people need answers from the "answer authority", and once that authority is established, it is treated reverently and pressed to provide answers. As the "priests" are humans, too, there's no reason to believe they themselves are immune to this effect, even when they know how the sausage is made.

jhoechtl 11y ago

1. To many scientists treating correlation for causality 2. Pressure to publish (requirement of funded projects or because of organisations target objectives) 3. Too many scientist

wavefunction 11y ago

"Why Is So Much Reported Science Incorrect," There's my contribution. This is from an illustrious institution like Berkeley?

J_Darnley 11y ago

Because journalists and audiences are incompetent. Journalists only get a press release from Big Corporation which is worded to give the best impression possible. The journalists then need to dumb it down for the masses. Then when Adam tells Bob he only repeats what he remembers and understands. In a massive game of Chinese whispers it is no wonder that "So Much Reported Science Wrong".

jwatte 11y ago

Bias against negative results are a problem. Frequentist "p values" are another, especially when samples are small and biased ("20 undergraduate students") and 0.05 is considered "significant." Now, the system is still designed to correct itself for errors over time, and reinforcing this design is the most important meta-thing we can do. So it's not all doom and gloom. It's just that "over time" is a longer and messier horizon than we would like!