4 ms·

Not that simple, and that was something missed in the original article - not all citations are equal. You have to know the context of each. Like, it could've be

by lmc 5y ago

Not that simple, and that was something missed in the original article - not all citations are equal. You have to know the context of each. Like, it could've been 1 positive citation, and 349 calling it bullshit... or vice versa.

motohagiography 5y ago
Indeed, the beauty of using a graph is you can see the direct citations, and then secondary and teritiery ones and then pull apart the clusters to see if there is any opportunity to pull on mathematical threads. A query like, given paper X, show papers that cite it, papers that cite those, and then ones that cite both. Cycles in the graph might show citation rings as well (as I think has happened in the past), but this would be a challenge for young people to apply their high school calculus skills to verifying claims in social science research. A breadth first search across papers using citations selects which paths you want to do your depth first validation on. We do the same thing in vulnerability research with software dependencies, and picking a bad-math vuln in a discipline and tracing its impact though, or picking clusters of tightly dependent research and finding vulns in them seems like an opportunity for an enterprising outsider who never wants a job from any of these people.
- tunesmith 5y ago
  How can a citation ring happen if papers are written one after another?
  - smaddox 5y ago
    Preprints. It happens quite often in deep learning papers, these days.
native_samples 5y ago
Scientists almost never negatively cite papers, and actually never do for incompetence or bad methodologies, so this is not a problem in practice [1]. In fact they don't even notice when citing retracted papers [2] so good luck getting them to notice deeper problems. The reality is, a citation is always either neutral or a positive reference to other work. [1] https://www.pnas.org/content/112/45/13823 https://www.pnas.org/content/112/45/13823 - only ~2.5% of citations are negative. Of those virtually all are about the findings, not the methodology. [2] https://pubmed.ncbi.nlm.nih.gov/18974415/ https://pubmed.ncbi.nlm.nih.gov/18974415/
- deleted 5y ago
  [deleted]
- lmc 5y ago
  Good points, and good to see there's some data behind that as well. That's how it's seemed to me in my areas. I'm not sure to what degree it applies for things like psychology - prob never as extreme as my original example, but it would be interesting to see a similar study.

by lmc 5y ago

motohagiography 5y ago

Indeed, the beauty of using a graph is you can see the direct citations, and then secondary and teritiery ones and then pull apart the clusters to see if there is any opportunity to pull on mathematical threads. A query like, given paper X, show papers that cite it, papers that cite those, and then ones that cite both. Cycles in the graph might show citation rings as well (as I think has happened in the past), but this would be a challenge for young people to apply their high school calculus skills to verifying claims in social science research. A breadth first search across papers using citations selects which paths you want to do your depth first validation on. We do the same thing in vulnerability research with software dependencies, and picking a bad-math vuln in a discipline and tracing its impact though, or picking clusters of tightly dependent research and finding vulns in them seems like an opportunity for an enterprising outsider who never wants a job from any of these people.

tunesmith 5y ago
How can a citation ring happen if papers are written one after another?
- smaddox 5y ago
  Preprints. It happens quite often in deep learning papers, these days.

native_samples 5y ago

Scientists almost never negatively cite papers, and actually never do for incompetence or bad methodologies, so this is not a problem in practice [1]. In fact they don't even notice when citing retracted papers [2] so good luck getting them to notice deeper problems. The reality is, a citation is always either neutral or a positive reference to other work. [1] https://www.pnas.org/content/112/45/13823 https://www.pnas.org/content/112/45/13823 - only ~2.5% of citations are negative. Of those virtually all are about the findings, not the methodology. [2] https://pubmed.ncbi.nlm.nih.gov/18974415/ https://pubmed.ncbi.nlm.nih.gov/18974415/

deleted 5y ago
[deleted]
lmc 5y ago
Good points, and good to see there's some data behind that as well. That's how it's seemed to me in my areas. I'm not sure to what degree it applies for things like psychology - prob never as extreme as my original example, but it would be interesting to see a similar study.