3 ms·

I don't think you're wrong, I just don't think those signals are very strong. I do think they used to be stronger, but paper discovery isn't really dominated th

I don't think you're wrong, I just don't think those signals are very strong. I do think they used to be stronger, but paper discovery isn't really dominated that way anymore. Word of mouth was always common, but now that's expanded to social media as well as things like Google scholar or semantic. Hell, most journals and conferences are hard to search through. Maybe I'll land on their page from a Google search of the paper or when looking for those topics, but that's not any different from ArXiv. And truth is that I still have to do the same thing when landing in either location. I still have to read the abstract, give it a sniff test, and read it to varying depth depending on what's needed. There's a signal, I just haven't found it to be that meaningful

fc417fc802 1y ago
I never claimed it was useful for targeted searches by modern standards. It's primarily a time saving measure from my perspective. If I come across a paper and it is in a publication that has a track record of publishing shoddy results I am likely to prioritize other works (assuming that I have a large potential pool of them to investigate). If a nature paper claims a seemingly impossible result I'll most likely spend the time to at least glance through it and see if maybe my preconceptions were mistaken. If something on arxiv looks like nonsense I'm not going to waste any time on it (unless someone else told me it's worth it).
- godelski 1y ago
  > It's primarily a time saving measure from my perspective. Your perspective as what? Help me understand the point you are arguing from. A researcher? A general member of the public? Grad student? Something else? I'm specifically asking this because there are different perspectives from the inside than the outside. There's a lot of internal problems that the general public isn't going to see. I also do not think papers should be aimed at the public. They should be aimed at other researchers. There are spaces for communicating with the public and it uses very different language. But these papers are supposed to be our communication to one another. We do it in the open. Doing this more openly means no gatekeeping, which putting them behind paywalls is also gatekeeping. But we need a space to talk to one another. The best way to do that is out in the open, mostly because we don't know who we're talking to. But just because it is visible by all doesn't mean everyone is our audience. > If I come across a paper and it is in a publication that has a track record of publishing shoddy results I am likely to prioritize other works Taking a stab in the dark here, but I'm guessing the answer isn't "researcher" to the above question? The reason I say this is because this seems like a weird scenario. There's so many venues that except those in your specific domain you're probably not going to know what is a reputable venue from one that isn't. It's also not exactly an easy thing to verify. If it is your field, well, you can tell 99% of the time from just the abstract. > If a nature paper You may wish to have a look at this conversation[0]. Or look at the controversy section of the wiki which provides more examples of my earlier claim about routinely rejecting good papers.[1] A good place to always look at is Retraction Watch[2,3] [0] https://news.ycombinator.com/item?id=5906389 https://news.ycombinator.com/item?id=5906389 [1] https://en.wikipedia.org/wiki/Nature_(journal)#Controversies https://en.wikipedia.org/wiki/Nature_(journal)#Controversies [2] https://retractionwatch.com/2024/01/26/springer-nature-journal-pulls-nearly-three-dozen-papers-from-special-issues/ https://retractionwatch.com/2024/01/26/springer-nature-journ... [3] https://retractionwatch.com/2024/11/07/another-springer-nature-journal-has-retracted-over-300-papers-since-july/ https://retractionwatch.com/2024/11/07/another-springer-natu...
  - fc417fc802 1y ago
    Rejecting good papers is different from accepting bad ones. False negatives (the former) don't waste my time as a reader whereas false positives might. From my perspective the goal (which is a means to an end) is to improve the signal to noise ratio. Not to sound uncaring but I'm not losing sleep if someone got his feelings hurt because his groundbreaking paper was rejected by whatever high profile journal. That used to be potentially damaging but these days we have arxiv so you're still the original researcher to make the discovery. Suck it up and resubmit elsewhere. Rejection is an integral part of the job. To answer your earlier question I am sometimes a researcher and sometimes a layman. There are many reasons to consult the academic literature after all. > There's so many venues that except those in your specific domain you're probably not going to know what is a reputable venue from one that isn't. It's also not exactly an easy thing to verify. You are massively overthinking this. If of the last N nonsense results 5% of them were in a given journal you bet I'm going to remember the name and associate it with bullshit. I don't need to "verify" anything it's nothing more than a personal bias regarding a given publication venue. Nonetheless I find it (and quite a few other metrics) useful in rapidly prioritizing a list of hundreds of potentially relevant results when deciding which threads to start pulling on for my current question. That's a more important question than you might think (at least in my experience) since you generally don't have time to pull on all the threads and ask all the new questions that come up in doing so. It's a multi headed search that generally prioritizes depth so the quality of the commentary and references in the papers that you spend significant time on is of the utmost importance to the end result.

fc417fc802 1y ago

I never claimed it was useful for targeted searches by modern standards. It's primarily a time saving measure from my perspective. If I come across a paper and it is in a publication that has a track record of publishing shoddy results I am likely to prioritize other works (assuming that I have a large potential pool of them to investigate). If a nature paper claims a seemingly impossible result I'll most likely spend the time to at least glance through it and see if maybe my preconceptions were mistaken. If something on arxiv looks like nonsense I'm not going to waste any time on it (unless someone else told me it's worth it).

godelski 1y ago
> It's primarily a time saving measure from my perspective. Your perspective as what? Help me understand the point you are arguing from. A researcher? A general member of the public? Grad student? Something else? I'm specifically asking this because there are different perspectives from the inside than the outside. There's a lot of internal problems that the general public isn't going to see. I also do not think papers should be aimed at the public. They should be aimed at other researchers. There are spaces for communicating with the public and it uses very different language. But these papers are supposed to be our communication to one another. We do it in the open. Doing this more openly means no gatekeeping, which putting them behind paywalls is also gatekeeping. But we need a space to talk to one another. The best way to do that is out in the open, mostly because we don't know who we're talking to. But just because it is visible by all doesn't mean everyone is our audience. > If I come across a paper and it is in a publication that has a track record of publishing shoddy results I am likely to prioritize other works Taking a stab in the dark here, but I'm guessing the answer isn't "researcher" to the above question? The reason I say this is because this seems like a weird scenario. There's so many venues that except those in your specific domain you're probably not going to know what is a reputable venue from one that isn't. It's also not exactly an easy thing to verify. If it is your field, well, you can tell 99% of the time from just the abstract. > If a nature paper You may wish to have a look at this conversation[0]. Or look at the controversy section of the wiki which provides more examples of my earlier claim about routinely rejecting good papers.[1] A good place to always look at is Retraction Watch[2,3] [0] https://news.ycombinator.com/item?id=5906389 https://news.ycombinator.com/item?id=5906389 [1] https://en.wikipedia.org/wiki/Nature_(journal)#Controversies https://en.wikipedia.org/wiki/Nature_(journal)#Controversies [2] https://retractionwatch.com/2024/01/26/springer-nature-journal-pulls-nearly-three-dozen-papers-from-special-issues/ https://retractionwatch.com/2024/01/26/springer-nature-journ... [3] https://retractionwatch.com/2024/11/07/another-springer-nature-journal-has-retracted-over-300-papers-since-july/ https://retractionwatch.com/2024/11/07/another-springer-natu...
- fc417fc802 1y ago
  Rejecting good papers is different from accepting bad ones. False negatives (the former) don't waste my time as a reader whereas false positives might. From my perspective the goal (which is a means to an end) is to improve the signal to noise ratio. Not to sound uncaring but I'm not losing sleep if someone got his feelings hurt because his groundbreaking paper was rejected by whatever high profile journal. That used to be potentially damaging but these days we have arxiv so you're still the original researcher to make the discovery. Suck it up and resubmit elsewhere. Rejection is an integral part of the job. To answer your earlier question I am sometimes a researcher and sometimes a layman. There are many reasons to consult the academic literature after all. > There's so many venues that except those in your specific domain you're probably not going to know what is a reputable venue from one that isn't. It's also not exactly an easy thing to verify. You are massively overthinking this. If of the last N nonsense results 5% of them were in a given journal you bet I'm going to remember the name and associate it with bullshit. I don't need to "verify" anything it's nothing more than a personal bias regarding a given publication venue. Nonetheless I find it (and quite a few other metrics) useful in rapidly prioritizing a list of hundreds of potentially relevant results when deciding which threads to start pulling on for my current question. That's a more important question than you might think (at least in my experience) since you generally don't have time to pull on all the threads and ask all the new questions that come up in doing so. It's a multi headed search that generally prioritizes depth so the quality of the commentary and references in the papers that you spend significant time on is of the utmost importance to the end result.