3 ms·

This is actually very scary to me. This means that many papers citation list is not a result of a thorough literature search but rather the authors recalling pa

This is actually very scary to me. This means that many papers citation list is not a result of a thorough literature search but rather the authors recalling papers they might have read or seen on social media. This is dangerous in that if there were papers from before The Tweet era that actually found that part of your research protocol was bad, they would not be found. An example of this is the rat maze study that Feynman cites. http://calteches.library.caltech.edu/51/2/CargoCult.htm http://calteches.library.caltech.edu/51/2/CargoCult.htm That last thing we need is clickbait science.

Iv 6y ago
Kill Elsevier and paywall then. They are the alternative but no one is going to pay 30$ per paper they need to read. If references were hyper links to the full pdf, they would get more views
- haihaibye 6y ago
  I'm all for open access, but this experiment didn't change the accessibility of the papers.
viraptor 6y ago
Some alternative explanations (likely a mix of them): papers that you see / think about influence the direction of your research; lit review is done early, but relevant papers you see in the process will be added; lit review is never going to be perfect, so more advertising = more chances of spotting something relevant.
LetThereBeNick 6y ago
Unlike in the past, it’s impossible to read every paper in one’s field. Since tools like PubMed, ISI, and Scopus all use a proprietary algorithm to rank search results, I appreciate any alternative channels for bringing relevant papers to my attention. It’s also perfectly justified to search for papers solely based on how influential they’ve been in your field. Papers which trend on Twitter (perhaps long-awaited results) are already having an impact on labs, probably being discussed in journal club and shaping the trajectory of the field.
- xxpor 6y ago
  PubMed is an interesting case, I wonder if the algorithm it uses for ranking would be FOIAable
anjc 6y ago
> This means that many papers citation list is not a result of a thorough literature search Related work sections aren't typically thorough, that's what lit surveys/reviews are for. It's also not a new phenomenon that highly cited work gets cited because it's highly cited, i.e. work is cited simply because it's more readily noticed. It's furthermore common for the same research to arise independently in different fields, with different terminology, i.e. bodies of research are somehow not found. So, no need to be scared, it's nothing new.
webmaven 6y ago
I think you're conflating several different purposes of citations. A thorough literature search is mostly needed to avoid claiming as novel innovations/discoveries things that others have previously published (even if you more-or-less ended up discovering it independently). Secondarily, it also helps you avoid ratholes that others have stumbled into (although the reduced incentives for publishing negative results makes this less effective). However, citations of papers encountered on — and recalled from — social media are more likely to be for the inspiration they provided for the line of inquiry or methodology in your paper (even if that inspiration was negative, ie. you're refuting someone's work or conclusions).
0d9eooo 6y ago
Unfortunately, I think we're already at a clickbait science level to some extent in some fields, depending on what you mean by "clickbait". This effect would have to be replicated in some form to understand its boundaries better but if it were to generalize I think it has much broader implications than just Twitter and thoracic surgery. There have been lots of bibliometric studies of citation impact etc. but this is one of the first times I've seen an actual experimental study in academics of this sort of thing, and it confirms what a lot of people have been experiencing, which is that actual scientific quality is only part of the reason why some work gets lots of attention. Replace twitter with other forms of social networking and the implications become clear. There's ups and downs to this: maybe some underappreciated work would get more attention if it's marketed more, for example. But it speaks to the role of things outside of the domain of study per se (not sure what the term for this would be -- something like nondiagetic but for scientific content).

Iv 6y ago

Kill Elsevier and paywall then. They are the alternative but no one is going to pay 30$ per paper they need to read. If references were hyper links to the full pdf, they would get more views

haihaibye 6y ago
I'm all for open access, but this experiment didn't change the accessibility of the papers.

viraptor 6y ago

Some alternative explanations (likely a mix of them): papers that you see / think about influence the direction of your research; lit review is done early, but relevant papers you see in the process will be added; lit review is never going to be perfect, so more advertising = more chances of spotting something relevant.

LetThereBeNick 6y ago

Unlike in the past, it’s impossible to read every paper in one’s field. Since tools like PubMed, ISI, and Scopus all use a proprietary algorithm to rank search results, I appreciate any alternative channels for bringing relevant papers to my attention. It’s also perfectly justified to search for papers solely based on how influential they’ve been in your field. Papers which trend on Twitter (perhaps long-awaited results) are already having an impact on labs, probably being discussed in journal club and shaping the trajectory of the field.

xxpor 6y ago
PubMed is an interesting case, I wonder if the algorithm it uses for ranking would be FOIAable

anjc 6y ago

> This means that many papers citation list is not a result of a thorough literature search Related work sections aren't typically thorough, that's what lit surveys/reviews are for. It's also not a new phenomenon that highly cited work gets cited because it's highly cited, i.e. work is cited simply because it's more readily noticed. It's furthermore common for the same research to arise independently in different fields, with different terminology, i.e. bodies of research are somehow not found. So, no need to be scared, it's nothing new.

webmaven 6y ago

I think you're conflating several different purposes of citations. A thorough literature search is mostly needed to avoid claiming as novel innovations/discoveries things that others have previously published (even if you more-or-less ended up discovering it independently). Secondarily, it also helps you avoid ratholes that others have stumbled into (although the reduced incentives for publishing negative results makes this less effective). However, citations of papers encountered on — and recalled from — social media are more likely to be for the inspiration they provided for the line of inquiry or methodology in your paper (even if that inspiration was negative, ie. you're refuting someone's work or conclusions).

0d9eooo 6y ago

Unfortunately, I think we're already at a clickbait science level to some extent in some fields, depending on what you mean by "clickbait". This effect would have to be replicated in some form to understand its boundaries better but if it were to generalize I think it has much broader implications than just Twitter and thoracic surgery. There have been lots of bibliometric studies of citation impact etc. but this is one of the first times I've seen an actual experimental study in academics of this sort of thing, and it confirms what a lot of people have been experiencing, which is that actual scientific quality is only part of the reason why some work gets lots of attention. Replace twitter with other forms of social networking and the implications become clear. There's ups and downs to this: maybe some underappreciated work would get more attention if it's marketed more, for example. But it speaks to the role of things outside of the domain of study per se (not sure what the term for this would be -- something like nondiagetic but for scientific content).