4 ms·

> Your comment essentially says - this is not a high quality submission because readers might not actually read it I'd argue that on average, most readers won'

> Your comment essentially says - this is not a high quality submission because readers might not actually read it I'd argue that on average, most readers won't have a good enough understanding, or read the paper far enough, to understand that the reality is closer to "it's not a breakthrough" rather than "Transformers with Unlimited Length Input". So, I wholeheartedly welcome this type of hype-breaking leading comment.

jjoonathan 3y ago
Agreed 100%. Not only do I appreciate "well actually" comments, I think they are the single most useful aspect of forum discussions. The headline will always be "BATTERY BREAKTHROUGH PROMISES TO ROCKET ELON MUSK TESLA TO THE MOON!!!" and while it's easy to know that some amount of cold water is necessary you need to spend a nontrivial amount of attention and have a nontrivial amount of knowledge to figure out just how much cold water. It's a useful thing to outsource. Did a research group see outperformance in an experiment with 1% probability of translating into production? Or is CATL scaling up a production process? The "well actually" comment will contextualize for you. If there's a "well actually" reply to the "well actually" comment, that tells you something too. Upvotes/downvotes dial in the distributed consensus. It's far from perfect, but I'd challenge detractors to point to a more effective method for large-scale democratic truth seeking.
- zamnos 3y ago
  well actually (sorry, couldn't resist) the refrain of "correlation is not causation" despite not having read beyond the headline, or "...in mice" when that's mentioned in the abstract, is pretty frustrating when that's the entire substance of the comment. it seems some technical and social countermeasures could be deployed so there was at least token visiting of a link before commenting was allowed, which court raise discourse in this forum, at least. It's a false dichotomy to only consider the two extremes of Peer Reviewed and HN reviewed. In particular, as mentioned up thread, the incentive to do peer review (or replicate an experiment, for that matter) isn't as high as working on your own research, attempting to do some novel work for a shot at a Nobel prize. as every coder who's been the reviewer on a code review knows, it's difficult, and one that often is not well prioritized among other priorities a senior IC might have, leading to poor quality reviewing or other work slipping (only so many hours in a week). Thus, one could imagine a system that gives direct compensation for review work to grad and smart undergrad students who work in the area being discussed vetting and/or contextualizing claims like "this work is/is not novel", rather than hoping someone who does work in that area is procrastinating on HN at just the right time to make the claim and the rebuttal and the rebuttal-rebuttal. If the rubuttal is posted after the thread falls off the front page, is anyone not in that thread even going to know that what they read was wrong?
swores 3y ago
It's possible to approve of the "hype-breaking" (aka TLDRing / ELI5ing so that HN comment readers can understand the degree to which it's interesting for those of us not close enough to the field to understand that for ourselves) without agreeing that that same comment should also complain that preprints shouldn't be submitted to / upvoted on HN. That's how I feel, anyway. I'd rather have seen a comment that has the same explanations in it but just generally less grumpy! Saying stuff like "It's not really that novel." doesn't really contribute much, when it could either be explained why it isn't novel by explaining how similar it is to something earlier that can be referenced, or thinking about what if anything is novel in this research - assuming it isn't being accused of just replicating something already done.

jjoonathan 3y ago

Agreed 100%. Not only do I appreciate "well actually" comments, I think they are the single most useful aspect of forum discussions. The headline will always be "BATTERY BREAKTHROUGH PROMISES TO ROCKET ELON MUSK TESLA TO THE MOON!!!" and while it's easy to know that some amount of cold water is necessary you need to spend a nontrivial amount of attention and have a nontrivial amount of knowledge to figure out just how much cold water. It's a useful thing to outsource. Did a research group see outperformance in an experiment with 1% probability of translating into production? Or is CATL scaling up a production process? The "well actually" comment will contextualize for you. If there's a "well actually" reply to the "well actually" comment, that tells you something too. Upvotes/downvotes dial in the distributed consensus. It's far from perfect, but I'd challenge detractors to point to a more effective method for large-scale democratic truth seeking.

zamnos 3y ago
well actually (sorry, couldn't resist) the refrain of "correlation is not causation" despite not having read beyond the headline, or "...in mice" when that's mentioned in the abstract, is pretty frustrating when that's the entire substance of the comment. it seems some technical and social countermeasures could be deployed so there was at least token visiting of a link before commenting was allowed, which court raise discourse in this forum, at least. It's a false dichotomy to only consider the two extremes of Peer Reviewed and HN reviewed. In particular, as mentioned up thread, the incentive to do peer review (or replicate an experiment, for that matter) isn't as high as working on your own research, attempting to do some novel work for a shot at a Nobel prize. as every coder who's been the reviewer on a code review knows, it's difficult, and one that often is not well prioritized among other priorities a senior IC might have, leading to poor quality reviewing or other work slipping (only so many hours in a week). Thus, one could imagine a system that gives direct compensation for review work to grad and smart undergrad students who work in the area being discussed vetting and/or contextualizing claims like "this work is/is not novel", rather than hoping someone who does work in that area is procrastinating on HN at just the right time to make the claim and the rebuttal and the rebuttal-rebuttal. If the rubuttal is posted after the thread falls off the front page, is anyone not in that thread even going to know that what they read was wrong?

swores 3y ago

It's possible to approve of the "hype-breaking" (aka TLDRing / ELI5ing so that HN comment readers can understand the degree to which it's interesting for those of us not close enough to the field to understand that for ourselves) without agreeing that that same comment should also complain that preprints shouldn't be submitted to / upvoted on HN. That's how I feel, anyway. I'd rather have seen a comment that has the same explanations in it but just generally less grumpy! Saying stuff like "It's not really that novel." doesn't really contribute much, when it could either be explained why it isn't novel by explaining how similar it is to something earlier that can be referenced, or thinking about what if anything is novel in this research - assuming it isn't being accused of just replicating something already done.