4 ms·

Scite: Smart Citations for Better Research

57 points by polm23 5y ago

uniqueuid 5y ago
On the tech side, this looks nice. But one thing people don't seem to take seriously enough is: If you're doing science and you don't know the literature, then you're doing a bad job. Understanding a paper won't work by reading AI summaries. Finding surprising new papers in your field should be very rare even without tech. [edit] Let me re-phrase the very negative statement above. There are a couple of important tasks in working with literature. One is finding cumulative truth, i.e. figuring out the overall state of research. This should be done in a true meta-analysis of the data, since references are sometimes citing articles incorrectly (i.e. claiming something that's not in the paper) and the linguistic complexity is enormous. Meta-analyes are a lot of manual work, even though you can extract some data from standardized tables. Another one is figuring out what new publications there are on a specific subject. Search engines like google scholar are immensely useful here, because you can follow the reference tree in both directions (citing and cited). This tool would help, but perhaps incrementally so (not a panacea). A third task would be to identify novel interesting papers that you didn't know about, perhaps from a different field. This tool might help, but only if an unknown interesting paper cites something you're looking at (i.e. if the networks are not disjoint). Apart from that, offering structural analyses and visualizations of science is something I believe should be done with open source and open data, because science is about trust and transparency - now more than ever.
- sleepingsoul 5y ago
  Hi, Thanks for the thoughtful response. Disclaimer: I work at scite. I want to clarify one point in your comment because I know a lot of people read HN comments before clicking through: we do NOT offer AI summaries! scite is fundamentally built on the idea that not all citations are equal. We offer disambiguation into citations by extracting the in-text contexts when citations happen between two papers. With that, you can qualitatively see how a paper has been cited by others by seeing the citation contexts from each citing paper. As someone who used to be in research before at the National Cancer Institute, I certainly agree that scientists who are advanced in their field should know the literature inside out. But the utility of scite is I think quite a bit more than that: - If you're an expert scientist, you can configure alerts to know when new citations happen to papers that you care about (e.g. ones you published, or that are important in your field, etc). Except unlike Google Scholar alerts and such, you can now immediately see the citation contexts (as opposed to opening up each paper, and trying to locate the reference(s) and the in-text citations). - If you're a student or an experienced researcher trying to ramp up in a new field (which is increasingly important given how interdisciplinary science is these days), this is a useful tool in your arsenal to digest and discover new papers. - That last comment also applies if you are in the industry! e.g. someone in pharma doing drug discovery or evaluation and you're working on different drugs every <X> months. We do have a preprint we published if you'd like to read more about how scite works: https://www.biorxiv.org/content/10.1101/2021.03.15.435418v1 https://www.biorxiv.org/content/10.1101/2021.03.15.435418v1 I hope that was clear. I'm happy to answer any other questions. I'll try to come back and check periodically. - Ashish
  - rmbyrro 5y ago
    @sleepingsoul , how good is your database? Just tried a random paper I've read recently (DOI 10.1109/32.748920). It shows "no citations" in Scite, but Google Scholar shows "Cited by 1180": https://scholar.google.com/scholar?cites=9052935302971671123&as_sdt=2005&sciodt=0,5&hl=en https://scholar.google.com/scholar?cites=9052935302971671123...
    - sleepingsoul 5y ago
      Hi there, Well, that's a tricky question to answer! :) At the moment we use DOIs to link papers / citations, and get our information on DOIs from crossref (https://www.crossref.org/ https://www.crossref.org/). So every DOI there is also in our system (and any citation coming from a source without a DOI currently won't be picked up). From crossref we also have metadata on which publications a given paper references, so we can always show information on how many publications cite a given paper ("traditional citations"). As for our database of Smart Citations (the extracted in-text citation statements) -- the answer is that it depends on our ability to access full-text articles to actually do the citation statement extraction. We're constantly growing in this area through new indexing agreements with publishers and so on. To your specific question about that DOI, here is the corresponding scite report for that paper: https://scite.ai/reports/a-unified-framework-for-coupling-dZAL2j https://scite.ai/reports/a-unified-framework-for-coupling-dZ... I see that we identified 371 distinct citing publications (and extracted 374 in-text citation statements from those publications). Can you let me know where you saw it say "no citations"? That sounds like a bug... I'll take a look. You can email me here: ashish@scite.ai - Ashish
      - rmbyrro 5y ago
        Thanks! Weird, I searched for that DOI moments before and it showed this result: "This publication either has no citations yet, or we are still processing them" Let me look into others
      - rmbyrro 5y ago
        I think you have an issue in your UI, or I don't understand how it's supposed to be used When I have the results from an article in my screen and use the top search to find another, it only updates the upper part of the page with the paper title, not the stats and summary below it I had searched for an article before that had no citations. That's why I continued to see "no citations" when I searched for the DOI I shared...
- rsa4046 5y ago
  This. As a recently retired academic research scientist (i.e., still ‘working’, but no longer in the lab, and no longer paid LOL), this can't be emphasized enough. Most scientists are highly specialized: experts in their local area of interest, the command of which must be fairly deep. They are aware of which papers are about to appear in press, either because they were either a referee at some point in the review process, or received a 'heads up' preprint. Your statement on ‘finding cumulative truth, i.e. figuring out the overall state of research’ is a key point.To become newly acquainted with a research area (on one's own) is a daunting task, requires plowing through dozens and dozens of past research articles, review summaries, abstracts, texts, even conversing with authors of these works. But rapid progress is oft realized when people in two or more separate areas serendipitously interact in new projects, thus bringing expertise — heretofore missing or weak — to bear on a common problem. This is the intrinsically collaborative aspect of science that survives despite intense intramural competition.
rerx 5y ago
That name has just been burnt into my mind to stand for the Scintilla Text Editor: https://www.scintilla.org/SciTE.html https://www.scintilla.org/SciTE.html Amazing how this is still getting releases after 20+ years.
- hutzlibu 5y ago
  Yeah, I assumed by the title, SciTE added a feature to help with citations.
glup 5y ago
I find it interesting how this is premised on the "one big claim per paper" model, which for better or worse seems increasingly common. In my experience a lot of 60's-90's research is of the form "the results of our first study were consistent with [some specific paper], and our second experiment was not." This nuance is pretty hard for people to keep in mind, and often seems to get lost over time (as a postdoc in cognitive science at MIT, I've noticed a big difference between what papers actually say, vs. what people say those papers say 30 years on). I'm excited to see how well Scite actually works as I test it over the next week!
- darosati 5y ago
  Disclaimer: I work at scite. We, at scite, actually designed scite to help folks see beyond the "one big claim per paper" model. Specifically, we designed scite to present show how specific results were either supported or contrasted by other papers, rather than showing that the publication itself was disputed or reproduced as a whole. If there was any feedback you could give on how we could present things so that folks don't get that impression we'd love it - feel free to DM me or let us know at hi@scite.ai as you test it out. In an ideal world we would present the claims, results, and argumentation as structured information to the user but we are not quite at that stage yet!
elektor 5y ago
Highly recommend Scite, I use it on a nearly daily basis when doing lit reviews for my job.
- darosati 5y ago
  Thanks! I am glad you are finding it useful (I work at scite)

57 points by polm23 5y ago

On the tech side, this looks nice. But one thing people don't seem to take seriously enough is: If you're doing science and you don't know the literature, then you're doing a bad job. Understanding a paper won't work by reading AI summaries. Finding surprising new papers in your field should be very rare even without tech. [edit] Let me re-phrase the very negative statement above. There are a couple of important tasks in working with literature. One is finding cumulative truth, i.e. figuring out the overall state of research. This should be done in a true meta-analysis of the data, since references are sometimes citing articles incorrectly (i.e. claiming something that's not in the paper) and the linguistic complexity is enormous. Meta-analyes are a lot of manual work, even though you can extract some data from standardized tables. Another one is figuring out what new publications there are on a specific subject. Search engines like google scholar are immensely useful here, because you can follow the reference tree in both directions (citing and cited). This tool would help, but perhaps incrementally so (not a panacea). A third task would be to identify novel interesting papers that you didn't know about, perhaps from a different field. This tool might help, but only if an unknown interesting paper cites something you're looking at (i.e. if the networks are not disjoint). Apart from that, offering structural analyses and visualizations of science is something I believe should be done with open source and open data, because science is about trust and transparency - now more than ever.

sleepingsoul 5y ago
Hi, Thanks for the thoughtful response. Disclaimer: I work at scite. I want to clarify one point in your comment because I know a lot of people read HN comments before clicking through: we do NOT offer AI summaries! scite is fundamentally built on the idea that not all citations are equal. We offer disambiguation into citations by extracting the in-text contexts when citations happen between two papers. With that, you can qualitatively see how a paper has been cited by others by seeing the citation contexts from each citing paper. As someone who used to be in research before at the National Cancer Institute, I certainly agree that scientists who are advanced in their field should know the literature inside out. But the utility of scite is I think quite a bit more than that: - If you're an expert scientist, you can configure alerts to know when new citations happen to papers that you care about (e.g. ones you published, or that are important in your field, etc). Except unlike Google Scholar alerts and such, you can now immediately see the citation contexts (as opposed to opening up each paper, and trying to locate the reference(s) and the in-text citations). - If you're a student or an experienced researcher trying to ramp up in a new field (which is increasingly important given how interdisciplinary science is these days), this is a useful tool in your arsenal to digest and discover new papers. - That last comment also applies if you are in the industry! e.g. someone in pharma doing drug discovery or evaluation and you're working on different drugs every <X> months. We do have a preprint we published if you'd like to read more about how scite works: https://www.biorxiv.org/content/10.1101/2021.03.15.435418v1 https://www.biorxiv.org/content/10.1101/2021.03.15.435418v1 I hope that was clear. I'm happy to answer any other questions. I'll try to come back and check periodically. - Ashish
- rmbyrro 5y ago
  @sleepingsoul , how good is your database? Just tried a random paper I've read recently (DOI 10.1109/32.748920). It shows "no citations" in Scite, but Google Scholar shows "Cited by 1180": https://scholar.google.com/scholar?cites=9052935302971671123&as_sdt=2005&sciodt=0,5&hl=en https://scholar.google.com/scholar?cites=9052935302971671123...
  - sleepingsoul 5y ago
    Hi there, Well, that's a tricky question to answer! :) At the moment we use DOIs to link papers / citations, and get our information on DOIs from crossref (https://www.crossref.org/ https://www.crossref.org/). So every DOI there is also in our system (and any citation coming from a source without a DOI currently won't be picked up). From crossref we also have metadata on which publications a given paper references, so we can always show information on how many publications cite a given paper ("traditional citations"). As for our database of Smart Citations (the extracted in-text citation statements) -- the answer is that it depends on our ability to access full-text articles to actually do the citation statement extraction. We're constantly growing in this area through new indexing agreements with publishers and so on. To your specific question about that DOI, here is the corresponding scite report for that paper: https://scite.ai/reports/a-unified-framework-for-coupling-dZAL2j https://scite.ai/reports/a-unified-framework-for-coupling-dZ... I see that we identified 371 distinct citing publications (and extracted 374 in-text citation statements from those publications). Can you let me know where you saw it say "no citations"? That sounds like a bug... I'll take a look. You can email me here: ashish@scite.ai - Ashish
    - rmbyrro 5y ago
      Thanks! Weird, I searched for that DOI moments before and it showed this result: "This publication either has no citations yet, or we are still processing them" Let me look into others
    - rmbyrro 5y ago
      I think you have an issue in your UI, or I don't understand how it's supposed to be used When I have the results from an article in my screen and use the top search to find another, it only updates the upper part of the page with the paper title, not the stats and summary below it I had searched for an article before that had no citations. That's why I continued to see "no citations" when I searched for the DOI I shared...
rsa4046 5y ago
This. As a recently retired academic research scientist (i.e., still ‘working’, but no longer in the lab, and no longer paid LOL), this can't be emphasized enough. Most scientists are highly specialized: experts in their local area of interest, the command of which must be fairly deep. They are aware of which papers are about to appear in press, either because they were either a referee at some point in the review process, or received a 'heads up' preprint. Your statement on ‘finding cumulative truth, i.e. figuring out the overall state of research’ is a key point.To become newly acquainted with a research area (on one's own) is a daunting task, requires plowing through dozens and dozens of past research articles, review summaries, abstracts, texts, even conversing with authors of these works. But rapid progress is oft realized when people in two or more separate areas serendipitously interact in new projects, thus bringing expertise — heretofore missing or weak — to bear on a common problem. This is the intrinsically collaborative aspect of science that survives despite intense intramural competition.

rerx 5y ago

That name has just been burnt into my mind to stand for the Scintilla Text Editor: https://www.scintilla.org/SciTE.html https://www.scintilla.org/SciTE.html Amazing how this is still getting releases after 20+ years.

hutzlibu 5y ago
Yeah, I assumed by the title, SciTE added a feature to help with citations.

glup 5y ago

I find it interesting how this is premised on the "one big claim per paper" model, which for better or worse seems increasingly common. In my experience a lot of 60's-90's research is of the form "the results of our first study were consistent with [some specific paper], and our second experiment was not." This nuance is pretty hard for people to keep in mind, and often seems to get lost over time (as a postdoc in cognitive science at MIT, I've noticed a big difference between what papers actually say, vs. what people say those papers say 30 years on). I'm excited to see how well Scite actually works as I test it over the next week!

darosati 5y ago
Disclaimer: I work at scite. We, at scite, actually designed scite to help folks see beyond the "one big claim per paper" model. Specifically, we designed scite to present show how specific results were either supported or contrasted by other papers, rather than showing that the publication itself was disputed or reproduced as a whole. If there was any feedback you could give on how we could present things so that folks don't get that impression we'd love it - feel free to DM me or let us know at hi@scite.ai as you test it out. In an ideal world we would present the claims, results, and argumentation as structured information to the user but we are not quite at that stage yet!

elektor 5y ago

Highly recommend Scite, I use it on a nearly daily basis when doing lit reviews for my job.

darosati 5y ago
Thanks! I am glad you are finding it useful (I work at scite)