5 ms·

For manual flagging, you can use Unfold Research to leave a review on the paper itself for anyone to see it, and you can add review tags and a description about

For manual flagging, you can use Unfold Research to leave a review on the paper itself for anyone to see it, and you can add review tags and a description about what you think is wrong with it: https://twitter.com/UnfoldResearch/status/1560995366492848129 https://twitter.com/UnfoldResearch/status/156099536649284812... Tools like scite (https://scite.ai/ https://scite.ai/) do automatic processing (NLP) of other papers and determine whether a reference relationships supports or rejects the paper. Also, http://octopus.ac/ http://octopus.ac/ did NLP processing on papers to make a breakdown of them and extract pieces to populate the content on their website, but they seem to have removed the content on their Github since then.

karencarits 4y ago
You have also got https://pubpeer.com/ https://pubpeer.com/
gchamonlive 4y ago
Could manual flagging be abused by a country or agent against another country or agent in order to maliciously discredit papers from that target?
- abstractalgo 4y ago
  Most definitely. An advantage of having open peer review system is also that this kind of attempts could be detected by tools doing automatic processing of scientific knowledge graphs generated by peer review activity. This, coupled with verification of users, should be able to minimize the risk quite significantly.
  - gchamonlive 4y ago
    Seems to me that this wouldn't be enough in case of state actors or large organizations. They would just have to employ real people to carry the fake review bombs.
- bittercynic 4y ago
  Instead of a flagging system ranking the credibility of papers once for everyone, can we have a system where each user decides which other users they trust to evaluate papers, and ignore any signal from untrusted users? For example, I could L1 trust people to evaluate papers and the trustworthiness of other users, and L2 trust others only to evaluate the trustworthiness of papers. If I notice a spurious signal getting in, I can find out who it came from and prune my list of trusted users. I imagine I'm not the first to think of this. Anyone know of reading material on the topic?
  - abstractalgo 4y ago
    The one thing I'm aware of is Brave's Goggles project: https://search.brave.com/help/goggles https://search.brave.com/help/goggles - it basically allows you to set up your own ranking algorithm with a specific syntax (it links to Github repo and, there, to a white paper). I haven't seen more attempts at similar things, which doesn't mean they don't exist though. (There were mentions of a similar feature being added when Elon Musk was about to buy Twitter, but just gossips)
  - jpeloquin 4y ago
    Sounds like journal clubs but organized through a specialized tech platform rather than normal communication channels. (Journal clubs are groups of people, usually working in the same field, who pick & present a paper to the rest of the group on an rotating basis.)

karencarits 4y ago

You have also got https://pubpeer.com/ https://pubpeer.com/

gchamonlive 4y ago

Could manual flagging be abused by a country or agent against another country or agent in order to maliciously discredit papers from that target?

abstractalgo 4y ago
Most definitely. An advantage of having open peer review system is also that this kind of attempts could be detected by tools doing automatic processing of scientific knowledge graphs generated by peer review activity. This, coupled with verification of users, should be able to minimize the risk quite significantly.
- gchamonlive 4y ago
  Seems to me that this wouldn't be enough in case of state actors or large organizations. They would just have to employ real people to carry the fake review bombs.
bittercynic 4y ago
Instead of a flagging system ranking the credibility of papers once for everyone, can we have a system where each user decides which other users they trust to evaluate papers, and ignore any signal from untrusted users? For example, I could L1 trust people to evaluate papers and the trustworthiness of other users, and L2 trust others only to evaluate the trustworthiness of papers. If I notice a spurious signal getting in, I can find out who it came from and prune my list of trusted users. I imagine I'm not the first to think of this. Anyone know of reading material on the topic?
- abstractalgo 4y ago
  The one thing I'm aware of is Brave's Goggles project: https://search.brave.com/help/goggles https://search.brave.com/help/goggles - it basically allows you to set up your own ranking algorithm with a specific syntax (it links to Github repo and, there, to a white paper). I haven't seen more attempts at similar things, which doesn't mean they don't exist though. (There were mentions of a similar feature being added when Elon Musk was about to buy Twitter, but just gossips)
- jpeloquin 4y ago
  Sounds like journal clubs but organized through a specialized tech platform rather than normal communication channels. (Journal clubs are groups of people, usually working in the same field, who pick & present a paper to the rest of the group on an rotating basis.)