6 ms·

I've been drafting a P2P app to (legitimately) share publications without intermediaries. Authenticity and traceability of all data - including metadata such as

by eecc 10y ago

I've been drafting a P2P app to (legitimately) share publications without intermediaries. Authenticity and traceability of all data - including metadata such as tags to represent peer approval - are supported using PGP cryptography. If anyone is interested to at least provide a critique to the code (it's my pet project so it's also rife with "let me try this new framework" syndrome) please give me a ping. If you want to contribute yay!

Mediterraneo10 10y ago
Peer approval at respected journals is anonymous: the writer doesn't know who the reviewers are. Sometimes the reviewers don't even know the identity of the author. How does your system allow for that? PGP signatures do not permit anonymity.
- lmm 10y ago
  PGP keys can be used with any authentication mechanism or none. A PGP signature is always associated with a particular key, sure, but a key ID is just a random number.
  - Mediterraneo10 10y ago
    How would a PGP signature from a certain key lend any prestige to the paper it signed, unless that key were linked to some respected identity?
    - lmm 10y ago
      Whatever your model is, you can implement on top of PGP. E.g. if you want pseudonymous reviewers selected by the editors of the Foo journal (the current model), there could be a key that's publicly linked to the Foo journal and you could use that key to sign short-lived pseudonymous "review keys" for "approved reviewer of paper xyz" which could sign their reviews.
- rocqua 10y ago
  There was an article on HN a while ago from someone whom decided to start signing his reviews. To me, that seemed like a decent initiative. The only real downside I see is that it allows the authors to contact and potentially bias the reviewer to affect a potential review of the written material.
  - michaelt 10y ago
    Well, the other downside is that reviewers may be unduly lax in signed reviews. After all, people are often emotionally invested in their work / publications, and rejection sometimes offends.
    - ferdterguson 10y ago
      SciPy did signed reviews for the 2017 conference the opposite reason. Obviously SciPy is not Science or Nature, but still. They cite the idea that if your review is tied for your name, you probably aren't going to do a crappy review because it will hurt your reputation. There's probably a balance.
    - rocqua 10y ago
      I'd expect that people might soften the wording of their reviews, but to the detriment of their own time (i.e. spend more time by offering the same critique more constructively). I don't worry that people might soften their reviews by simply being less critical.
- pdexter 10y ago
  They're anonymous but after the reviews the authors are then usually (always?) revealed to the reviewers. Let's not kid ourselves though, academia, and especially in smaller subfields, is made up of small communities where you can usually guess whose paper you're reviewing unless it's a brand new contribution in a field by someone who doesn't usually publish there. You can often guess who the review was performed by too, since the program committee is always public.
  - geofft 10y ago
    You could implement something like Certificate Transparency's "precertificates" where you store a hash of the digital signature in a public Merkle tree, so during review the reviewer's identity isn't known. But once the paper is published, the actual signature can also be published and checked against the review hashes in the paper. If they're missing or inconsistent, treat the paper as unreviewed.
    - pdexter 10y ago
      Sounds like a good approach
  - Mediterraneo10 10y ago
    > You can often guess who the review was performed by too, since the program committee is always public. This must vary across disciplines. While in your field the pool of reviewers might be public, in my own field most journals keep confidential their set of preferred people to turn to for reviews.
  - kem 10y ago
    Eh ... in my experience as an editor, author, and reviewer, this varies quite a bit. Sometimes you can take a pretty good guess who it is based on the topic and the references (e.g., citations to in-press papers on obscure topics is usually a give away). Other times, though, it's difficult to tell who the authors are. People often also think they know who the author and/or reviewers are when they are really totally wrong. I've reviewed papers and seen other reviewers cite my papers in a way that I thought "damn, that looks like something I'd write" when it was not. Similarly, I've seen reviewers think they know who the author is when they're completely off. I personally think anonymous reviews are critical, because it keeps some modicum of shielding of reviewers against retribution for negative comments. It's not 100% anonymous of course, but it's anonymous enough to create enough doubt about who the reviewers and authors actually are.
eoinmurray92 10y ago
So the publication gets shared, and if you think its a good paper you set the a tag? This is a pretty good idea - but do you make the reviewer identify known?
Blahah 10y ago
hey - I'm also working on something along these lines [1]. v1 will be a search + reading interface for scientific articles with p2p datasources (provided as hyperdrive feeds [2]). v2 will have arbitrary features that can be used to customise the reading experience, such as marking retractions/update/expressions of concern, connecting post-publication peer reviews to articles, etc. we're at #dat and #sciencefair on freenode, would love to talk 1: https://github.com/codeforscience/sciencefair https://github.com/codeforscience/sciencefair 2: https://github.com/mafintosh/hyperdrive https://github.com/mafintosh/hyperdrive

by eecc 10y ago

Mediterraneo10 10y ago

Peer approval at respected journals is anonymous: the writer doesn't know who the reviewers are. Sometimes the reviewers don't even know the identity of the author. How does your system allow for that? PGP signatures do not permit anonymity.

lmm 10y ago
PGP keys can be used with any authentication mechanism or none. A PGP signature is always associated with a particular key, sure, but a key ID is just a random number.
- Mediterraneo10 10y ago
  How would a PGP signature from a certain key lend any prestige to the paper it signed, unless that key were linked to some respected identity?
  - lmm 10y ago
    Whatever your model is, you can implement on top of PGP. E.g. if you want pseudonymous reviewers selected by the editors of the Foo journal (the current model), there could be a key that's publicly linked to the Foo journal and you could use that key to sign short-lived pseudonymous "review keys" for "approved reviewer of paper xyz" which could sign their reviews.
rocqua 10y ago
There was an article on HN a while ago from someone whom decided to start signing his reviews. To me, that seemed like a decent initiative. The only real downside I see is that it allows the authors to contact and potentially bias the reviewer to affect a potential review of the written material.
- michaelt 10y ago
  Well, the other downside is that reviewers may be unduly lax in signed reviews. After all, people are often emotionally invested in their work / publications, and rejection sometimes offends.
  - ferdterguson 10y ago
    SciPy did signed reviews for the 2017 conference the opposite reason. Obviously SciPy is not Science or Nature, but still. They cite the idea that if your review is tied for your name, you probably aren't going to do a crappy review because it will hurt your reputation. There's probably a balance.
  - rocqua 10y ago
    I'd expect that people might soften the wording of their reviews, but to the detriment of their own time (i.e. spend more time by offering the same critique more constructively). I don't worry that people might soften their reviews by simply being less critical.
pdexter 10y ago
They're anonymous but after the reviews the authors are then usually (always?) revealed to the reviewers. Let's not kid ourselves though, academia, and especially in smaller subfields, is made up of small communities where you can usually guess whose paper you're reviewing unless it's a brand new contribution in a field by someone who doesn't usually publish there. You can often guess who the review was performed by too, since the program committee is always public.
- geofft 10y ago
  You could implement something like Certificate Transparency's "precertificates" where you store a hash of the digital signature in a public Merkle tree, so during review the reviewer's identity isn't known. But once the paper is published, the actual signature can also be published and checked against the review hashes in the paper. If they're missing or inconsistent, treat the paper as unreviewed.
  - pdexter 10y ago
    Sounds like a good approach
- Mediterraneo10 10y ago
  > You can often guess who the review was performed by too, since the program committee is always public. This must vary across disciplines. While in your field the pool of reviewers might be public, in my own field most journals keep confidential their set of preferred people to turn to for reviews.
- kem 10y ago
  Eh ... in my experience as an editor, author, and reviewer, this varies quite a bit. Sometimes you can take a pretty good guess who it is based on the topic and the references (e.g., citations to in-press papers on obscure topics is usually a give away). Other times, though, it's difficult to tell who the authors are. People often also think they know who the author and/or reviewers are when they are really totally wrong. I've reviewed papers and seen other reviewers cite my papers in a way that I thought "damn, that looks like something I'd write" when it was not. Similarly, I've seen reviewers think they know who the author is when they're completely off. I personally think anonymous reviews are critical, because it keeps some modicum of shielding of reviewers against retribution for negative comments. It's not 100% anonymous of course, but it's anonymous enough to create enough doubt about who the reviewers and authors actually are.

eoinmurray92 10y ago

So the publication gets shared, and if you think its a good paper you set the a tag? This is a pretty good idea - but do you make the reviewer identify known?

Blahah 10y ago

hey - I'm also working on something along these lines [1]. v1 will be a search + reading interface for scientific articles with p2p datasources (provided as hyperdrive feeds [2]). v2 will have arbitrary features that can be used to customise the reading experience, such as marking retractions/update/expressions of concern, connecting post-publication peer reviews to articles, etc. we're at #dat and #sciencefair on freenode, would love to talk 1: https://github.com/codeforscience/sciencefair https://github.com/codeforscience/sciencefair 2: https://github.com/mafintosh/hyperdrive https://github.com/mafintosh/hyperdrive