4 ms·

There already is a project trying to use NLP to detect the sentiment of scientific citations: https://scite.ai/ https://scite.ai/

amelius 4y ago
Yes, using NLP (DL) techniques makes more sense than to require every author to annotate their work.
- unixhero 4y ago
  I agree, Natural Language Processing is the right tool and field for this. What is the purpose of language, if you as an author must add an extra layer of metadata to it to convey meaning. Or maybe academic authors should just bute the billet and start using smileys to convey sentiment. These would be easy to work with.
- andrepd 4y ago
  Running some superficial "sentiment analysis" neural net over scholarly work and basing any sort of analysis on that is a terrible idea for obvious reasons. It is already a terrible idea for content moderation / sorting reviews / etc, and the stakes are lower.
  - Topolomancer 4y ago
    Don't understand why this keeps being downvoted. NLP is still in its infancy when it comes to detecting nuances of human communication. These are valid objections.
    - notahacker 4y ago
      Moreover, scientific citations are amongst the worst candidates for sentiment analysis since authors often express no sentiment about cited papers at all, and often prefer to use dry, neutral tones when describing a paper's shortcomings. You might need both general intelligence and a deep knowledge of the field to conclude an author noting that X discovered Y on the assumption of Z is not a neutral description but a stark emphasis of their differences, or possibly even straw-manning the original paper.
  - make3 4y ago
    the goal is ease of use though, not serious analysis
Topolomancer 4y ago
This seems to be very hard to pull off, though, given the way people regularly cite work in some scientific domains. For instance, unless specifically comparing to an approach, researchers in ML rarely endorse or disavow a specific approach (just my observation in graph learning).

amelius 4y ago

Yes, using NLP (DL) techniques makes more sense than to require every author to annotate their work.

unixhero 4y ago
I agree, Natural Language Processing is the right tool and field for this. What is the purpose of language, if you as an author must add an extra layer of metadata to it to convey meaning. Or maybe academic authors should just bute the billet and start using smileys to convey sentiment. These would be easy to work with.
andrepd 4y ago
Running some superficial "sentiment analysis" neural net over scholarly work and basing any sort of analysis on that is a terrible idea for obvious reasons. It is already a terrible idea for content moderation / sorting reviews / etc, and the stakes are lower.
- Topolomancer 4y ago
  Don't understand why this keeps being downvoted. NLP is still in its infancy when it comes to detecting nuances of human communication. These are valid objections.
  - notahacker 4y ago
    Moreover, scientific citations are amongst the worst candidates for sentiment analysis since authors often express no sentiment about cited papers at all, and often prefer to use dry, neutral tones when describing a paper's shortcomings. You might need both general intelligence and a deep knowledge of the field to conclude an author noting that X discovered Y on the assumption of Z is not a neutral description but a stark emphasis of their differences, or possibly even straw-manning the original paper.
- make3 4y ago
  the goal is ease of use though, not serious analysis

Topolomancer 4y ago

This seems to be very hard to pull off, though, given the way people regularly cite work in some scientific domains. For instance, unless specifically comparing to an approach, researchers in ML rarely endorse or disavow a specific approach (just my observation in graph learning).