5 ms·

Not only do some papers not support the claim at all (I found this to often be the case when I used to read the news regularly), but there is often a critical d

Not only do some papers not support the claim at all (I found this to often be the case when I used to read the news regularly), but there is often a critical detail left out of the abstract that is key to how the study should be interpreted. The vast majority of people citing papers, by my measure, are relying solely on the abstract even when the entire paper is available for free. This is often the case outside of Wikipedia as well. Truthfully, I'm a layman because I don't have an advanced degree that suggests I'm qualified to interpret papers, but things are bad enough IMO that part of me wishes people would stop trying to communicate science to the public until this broken system heals at least somewhat. On a related note, it doesn't help that folks like Andrew Huberman are normalizing this idea that one or two studies are good enough to form conclusions about how average people can "optimize" (micromanage) their lives in ways that are clinically relevant. This isn't to take away from the good that Huberman does, but I think it send a the wrong message to people who don't actually have any experience reading papers. One thing that might help is forbidding abstracts and conclusion sections in papers published in journals, which might cut down on some of the misunderstandings and make it harder to pass off a paper as support for a claim. > Trying to correct these articles is some times like going to war with editors who refuse to admit they were wrong. There is virtually no point in trying to fix Wikipedia pages until Wikipedia fixes themselves. They have an extremely hard job, but their prolific editors are too bully-like and contradict themselves all the time.

wayfinder 3y ago
You need an abstract if you are looking up a topic and you’re scanning through articles though, and I think even if articles didn’t have abstracts, people would still link to papers to spin a narrative. I think this all really the cost of admission for the ease of self-publishing. Now any jerk off can post about their opinions on keeping abstracts while using the bathroom on company time.
Karrot_Kream 3y ago
Even when I was in grad school (sadly a while ago), it was understood that to really understand the validity of a paper, you needed to look at its conclusion and methods sections. Generally speaking the abstract was just used as a filter. It helped you answer the question of whether the paper was even worth taking a look at or not. Then you mined the conclusion and methods to see if they supported what you were looking for. Only then did you read the paper in detail. As you say there's a pretty big gap with the way researchers read through papers and the way the science media communicate about research with laypeople. It's a hard problem that needs more attention than it gets. (Probably one of the most useful classes I took in grad school was a class in the beginning that forced us to read 10 papers a week and write summaries. The class was hell as obviously there was no way to properly understand each paper on top of your research and other classes. But it made us great at being able to quickly winnow papers for the meat of each paper and also made us great at finding papers with... dubious methods.)
- sockaddr 3y ago
  Yup. I often will read something like "A connection between X and Y" Then skim down to the conclusion and it will say "In conclusion, we were only able to see some weak connection and we aren't sure if it was one at all, but this warrants further research..." I know negative results are still valuable science, but I feel like a good chunk of the papers I'm reading are basically worthless CV padding.
  - deleted 3y ago
    [deleted]
  - Terr_ 3y ago
    While it focuses a bit more on pop-culture, a relevant comic on "the science news-cycle": https://phdcomics.com/comics/archive.php?comicid=1174 https://phdcomics.com/comics/archive.php?comicid=1174
- mistrial9 3y ago
  hah - I was with you until the "ten papers a week" part. You say it yourself, there are parts to the paper, and also levels of concentration and re-reads. Drilling is useful for hyperactive monkey minds who are doing something else instead of studying
  - Karrot_Kream 3y ago
    It was an "intro to grad school" type class. The point was simply to build endurance. They made very clear in the class that there was no way we were going to absorb the depth of the papers and told us to bound the amount of time we spent on it clearly so that we don't kill ourselves over it. I like to think of it as marathon practice or breathing practice for folks who played woodwinds; it was just helping us get into the muscle of winnowing papers effectively.
wnevets 3y ago
> Not only do some papers not support the claim at all (I found this to often be the case when I used to read the news regularly), but there is often a critical detail left out of the abstract that is key to how the study should be interpreted. The vast majority of people citing papers, by my measure, are relying solely on the abstract even when the entire paper is available for free. This happens in the HN comments all the time. People just assume the other person isn't actually going to stop read it.
RobotToaster 3y ago
> This is often the case outside of Wikipedia as well. Often they just read the wikipedia article that cited the paper.
adastra22 3y ago
You do not need an advanced degree to evaluate a paper. That’s absolute nonsense.
- stev-0 3y ago
  There are many papers I can’t evaluate even with a degree in musical composition…
whatsakandr 3y ago
>One thing that might help is forbidding abstracts and conclusion sections in papers published in journals, which might cut down on some of the misunderstandings and make it harder to pass off a paper as support for a claim. I wouldn't go this far, bstracts are needed to determine if the paper is of interest to what you're researching. Perhaps a more modest refinement that could work would be to forbid conclusions in abstracts.

wayfinder 3y ago

You need an abstract if you are looking up a topic and you’re scanning through articles though, and I think even if articles didn’t have abstracts, people would still link to papers to spin a narrative. I think this all really the cost of admission for the ease of self-publishing. Now any jerk off can post about their opinions on keeping abstracts while using the bathroom on company time.

Karrot_Kream 3y ago

Even when I was in grad school (sadly a while ago), it was understood that to really understand the validity of a paper, you needed to look at its conclusion and methods sections. Generally speaking the abstract was just used as a filter. It helped you answer the question of whether the paper was even worth taking a look at or not. Then you mined the conclusion and methods to see if they supported what you were looking for. Only then did you read the paper in detail. As you say there's a pretty big gap with the way researchers read through papers and the way the science media communicate about research with laypeople. It's a hard problem that needs more attention than it gets. (Probably one of the most useful classes I took in grad school was a class in the beginning that forced us to read 10 papers a week and write summaries. The class was hell as obviously there was no way to properly understand each paper on top of your research and other classes. But it made us great at being able to quickly winnow papers for the meat of each paper and also made us great at finding papers with... dubious methods.)

sockaddr 3y ago
Yup. I often will read something like "A connection between X and Y" Then skim down to the conclusion and it will say "In conclusion, we were only able to see some weak connection and we aren't sure if it was one at all, but this warrants further research..." I know negative results are still valuable science, but I feel like a good chunk of the papers I'm reading are basically worthless CV padding.
- deleted 3y ago
  [deleted]
- Terr_ 3y ago
  While it focuses a bit more on pop-culture, a relevant comic on "the science news-cycle": https://phdcomics.com/comics/archive.php?comicid=1174 https://phdcomics.com/comics/archive.php?comicid=1174
mistrial9 3y ago
hah - I was with you until the "ten papers a week" part. You say it yourself, there are parts to the paper, and also levels of concentration and re-reads. Drilling is useful for hyperactive monkey minds who are doing something else instead of studying
- Karrot_Kream 3y ago
  It was an "intro to grad school" type class. The point was simply to build endurance. They made very clear in the class that there was no way we were going to absorb the depth of the papers and told us to bound the amount of time we spent on it clearly so that we don't kill ourselves over it. I like to think of it as marathon practice or breathing practice for folks who played woodwinds; it was just helping us get into the muscle of winnowing papers effectively.

wnevets 3y ago

> Not only do some papers not support the claim at all (I found this to often be the case when I used to read the news regularly), but there is often a critical detail left out of the abstract that is key to how the study should be interpreted. The vast majority of people citing papers, by my measure, are relying solely on the abstract even when the entire paper is available for free. This happens in the HN comments all the time. People just assume the other person isn't actually going to stop read it.

RobotToaster 3y ago

> This is often the case outside of Wikipedia as well. Often they just read the wikipedia article that cited the paper.

adastra22 3y ago

You do not need an advanced degree to evaluate a paper. That’s absolute nonsense.

stev-0 3y ago
There are many papers I can’t evaluate even with a degree in musical composition…

whatsakandr 3y ago

>One thing that might help is forbidding abstracts and conclusion sections in papers published in journals, which might cut down on some of the misunderstandings and make it harder to pass off a paper as support for a claim. I wouldn't go this far, bstracts are needed to determine if the paper is of interest to what you're researching. Perhaps a more modest refinement that could work would be to forbid conclusions in abstracts.