5 ms·

> Science got along fine for thousands of years without Journals and Conferences deciding what's "correct" and not. And was available only to rich bored people

> Science got along fine for thousands of years without Journals and Conferences deciding what's "correct" and not. And was available only to rich bored people who could self teach themselves

_aavaa_ 3y ago
Right, because a 40$-per-article paywall has done wonders for accessibility.
- EvgeniyZh 3y ago
  I guess you're just trolling, but in case you're serious, yes, being able to group together as scientists (university) and pay a single (or small amount) of entities to get all the scientific results, peer reviewed, has done wonders. Being able as university/funding agency to evaluate someone's scientific output without actually reading their papers (because other experts did it) allowed scaling. And to be honest, I do not know a single person how pays $40 for a paper, there are so many ways to get it for free (including just asking the authors). There is space for improvements, but "get rid of journals and conferences and hope scientists will somehow figure it out" is not a solution
  - godelski 3y ago
    And pre-print servers have even accelerated science. Because frankly this is a MCMC system because the nature of the game is everyone stumbling around in the dark. I am a scientist and I'm all for "get rid of journals and conferences BECAUSE we already do figure it out." It isn't "somehow" we're already fucking doing it. We aren't going to journals and searching them, we go to arxiv. We read what our peers share with us. We read what comes out of other groups that we know that are doing similar work to us. Journals and conferences aren't helping us do science, they are helping us advance our careers because we live in a publish or perish ecosystem that quantifies our work based on the prestige of a conference who is prestigious because they reject 80% of works. Nowhere in here is a measure for the actual quality of a work. That is often a difficult task and can only be done by peers, which is already happening. Identifying bad works is easy but identifying good works is hard. I'm sorry, but we already live in the system that you're suggesting is silly and it's been this way for centuries. Journals are just people capitalizing on our work and labor. Conferences at least put us in the same room, but they have no business claiming that they can accurately ascertain the quality of a work.
    - EvgeniyZh 3y ago
      > We aren't going to journals I do. Some of my peers do and some don't. It differs between fields and you're probably from CS (peer reviewed conferences) where progress is often faster than publication cycle, but it is not like that everywhere. > they are helping us advance our careers And it is important role which cannot be disregarded just because it doesn't match your idea of best possible promotion scheme. > but we already live in the system that you're suggesting is silly and it's been this way for centuries. Centuries? Can you elaborate? Peer review in journals as we know it is less than 100 years old.
      - godelski 3y ago
        > And it is important role which cannot be disregarded I disagree. I think academia has fallen for Goodhart's law. While I would say that CS is probably on the worse side of things (especially ML), I don't think we're special in having critical problems. The publish or perish paradigm combined with the noisy signal of jorunaling is a big reason we are in the replication crisis. Because novelty is held in high regard while replication isn't of any concern. The irony being that novelty is quite rare and that the faster you publish the less novel works will be (in general. There are exceptions of course). You just can't do novel work fast. > Centuries? Can you elaborate? Peer review in journals as we know it is less than 100 years old. What I was referring to is that we've been just sending papers to one another for centuries. Uploading to arxiv is just an easier form of that (and was the explicit reason for its existence). That we scientists have been communicating with one another for centuries "figuring it out" without journals just fine. As far as I'm concerned, it is a failed experiment. It was okay in the beginning, had some good ideas, but then the rot grew and took over. It only works when everyone acts in good faith, and at scale only requires a few bad actors to spoil the barrel. Maybe I'm biased because of the problems of my field, but my friends in other areas still frequently hit problematic walls that just slow things down rather than improve them.
        EvgeniyZh 3y ago
        > I disagree. I think academia has fallen for Goodhart's law. It may or may not, but my claim was that we need some mechanism to decide who gets a position and who doesn't, and it should be scalable for the current scale of science and beyond, i.e., at least hundreds/thousands of applications. Saying that journals are used for career decisions and this is not good (enough?) and thus doesn't need replacement mechanism is weird. > a big reason we are in the replication crisis. The stated reasons are the same for different branches of science, while replication crisis is far from that. Replication of 100-year-old experiments is also not always easy/successful. Publish or perish motivates people to cut corners, but I think it is far from being the main reason. > What I was referring to is that we've been just sending papers to one another for centuries. Uploading to arxiv is just an easier form of that (and was the explicit reason for its existence). (Almost) nobody reads arxiv (edit: I mean (relevant part of) arxiv as a whole, of course most people read specific papers on arxiv). In fact the snr of arxiv is so low, even compared to journals that just by filtering garbage you can get quarter a mil followers [1]. So instead of random experts of varying quality and motivation I'll get a single rando on twitter (no offense to A K, he does good job)? No thanks. > It only works when everyone acts in good faith, and at scale only requires a few bad actors to spoil the barrel. Citation needed? You could claim the same for science as whole, but self-correction turns to be extremely robust, even if slower than you may want. [1] https://twitter.com/_akhaliq https://twitter.com/_akhaliq
  - _aavaa_ 3y ago
    I was very serious. The journals take works others have done (work often paid for with public money) and get unpaid volunteers to peer-review it. Next they require the authors to sign away their copyright to it. And then they turn right back around and sell it to very same universities where it came from for thousands of dollars a year. And for what? So they can have have profit margins rivaling big tech? (Their major expenses are lawyers and server costs for pdf documents…) And why do researchers publish there? Because they need to in order to keep their jobs, or to get funding. You need those high-impact-factor and prestigious journals. And so you sign away your research. Most 1st world researchers don’t care or don’t know about the costs. But most of the world is not so lucky. Yes you could go around begging the authors for a free copy (but definitely don’t go to scihub or library genesis and download them for free, that would be illegal…). The “space for improvements” starts with burning down the journals and scattering their ashes among the fields to fertilize new science.
    - EvgeniyZh 3y ago
      > burning down the journals and scattering their ashes among the fields to fertilize new science. Oh yeah, destroy working system and then maybe figure it out, works like a charm every time.
      - _aavaa_ 3y ago
        Again, we had scientific progress (and problems with it) before the current system. The only currency that matters is if your results can be reproduced, as as far as I'm aware the overwhelming majority of journals make no attempt at reproducing the results before accepting it. Hell, it doesn't seem like they even make an attempt at make string there's enough information provided to even attempt to reproduce it. When I come across a paper that does any sort of modelling, I simply assume I won't be able to reproduce it if the code isn't included. They all seem to leave out important information.
        EvgeniyZh 3y ago
        > Again, we had scientific progress (and problems with it) before the current system. Again, the scale and accessibility of science is different now and then. We had commute before steam engine, it doesn't mean we can get rid of all engine-based transportation now without consequences. > The only currency that matters is if your results can be reproduced This is extreme reductionism. LHC results cannot be replicated unless you have multi billion particle accelerator in your backyard. JWST cannot be replicated. Many other large scale experiments are nearly impossible to replicate. In fact, these are often considered "important" ones. And this not necessarily due to price, but rather combination of price, unique expertise and willing to spend effort. You know who will be able to tell you if the experiment description makes sense and has chances to be replicated if you ask them? Peers. Peer review at will never worked (and there were attempts) and never will because nobody likes to do peer review. I'm yet to see any proposal for peer review without editorial invitations which has at least theoretical chances to succeed. > I won't be able to reproduce it if the code isn't included. So you do mean "reproduction", not "replication". This is very low threshold and definitely not "only currency that matters". If I send you the code that alters the results somewhere deep inside, you'll be able to reproduce them.

> Science got along fine for thousands of years without Journals and Conferences deciding what's "correct" and not. And was available only to rich bored people who could self teach themselves

_aavaa_ 3y ago

Right, because a 40$-per-article paywall has done wonders for accessibility.

EvgeniyZh 3y ago
I guess you're just trolling, but in case you're serious, yes, being able to group together as scientists (university) and pay a single (or small amount) of entities to get all the scientific results, peer reviewed, has done wonders. Being able as university/funding agency to evaluate someone's scientific output without actually reading their papers (because other experts did it) allowed scaling. And to be honest, I do not know a single person how pays $40 for a paper, there are so many ways to get it for free (including just asking the authors). There is space for improvements, but "get rid of journals and conferences and hope scientists will somehow figure it out" is not a solution
- godelski 3y ago
  And pre-print servers have even accelerated science. Because frankly this is a MCMC system because the nature of the game is everyone stumbling around in the dark. I am a scientist and I'm all for "get rid of journals and conferences BECAUSE we already do figure it out." It isn't "somehow" we're already fucking doing it. We aren't going to journals and searching them, we go to arxiv. We read what our peers share with us. We read what comes out of other groups that we know that are doing similar work to us. Journals and conferences aren't helping us do science, they are helping us advance our careers because we live in a publish or perish ecosystem that quantifies our work based on the prestige of a conference who is prestigious because they reject 80% of works. Nowhere in here is a measure for the actual quality of a work. That is often a difficult task and can only be done by peers, which is already happening. Identifying bad works is easy but identifying good works is hard. I'm sorry, but we already live in the system that you're suggesting is silly and it's been this way for centuries. Journals are just people capitalizing on our work and labor. Conferences at least put us in the same room, but they have no business claiming that they can accurately ascertain the quality of a work.
  - EvgeniyZh 3y ago
    > We aren't going to journals I do. Some of my peers do and some don't. It differs between fields and you're probably from CS (peer reviewed conferences) where progress is often faster than publication cycle, but it is not like that everywhere. > they are helping us advance our careers And it is important role which cannot be disregarded just because it doesn't match your idea of best possible promotion scheme. > but we already live in the system that you're suggesting is silly and it's been this way for centuries. Centuries? Can you elaborate? Peer review in journals as we know it is less than 100 years old.
    - godelski 3y ago
      > And it is important role which cannot be disregarded I disagree. I think academia has fallen for Goodhart's law. While I would say that CS is probably on the worse side of things (especially ML), I don't think we're special in having critical problems. The publish or perish paradigm combined with the noisy signal of jorunaling is a big reason we are in the replication crisis. Because novelty is held in high regard while replication isn't of any concern. The irony being that novelty is quite rare and that the faster you publish the less novel works will be (in general. There are exceptions of course). You just can't do novel work fast. > Centuries? Can you elaborate? Peer review in journals as we know it is less than 100 years old. What I was referring to is that we've been just sending papers to one another for centuries. Uploading to arxiv is just an easier form of that (and was the explicit reason for its existence). That we scientists have been communicating with one another for centuries "figuring it out" without journals just fine. As far as I'm concerned, it is a failed experiment. It was okay in the beginning, had some good ideas, but then the rot grew and took over. It only works when everyone acts in good faith, and at scale only requires a few bad actors to spoil the barrel. Maybe I'm biased because of the problems of my field, but my friends in other areas still frequently hit problematic walls that just slow things down rather than improve them.
      - EvgeniyZh 3y ago
        > I disagree. I think academia has fallen for Goodhart's law. It may or may not, but my claim was that we need some mechanism to decide who gets a position and who doesn't, and it should be scalable for the current scale of science and beyond, i.e., at least hundreds/thousands of applications. Saying that journals are used for career decisions and this is not good (enough?) and thus doesn't need replacement mechanism is weird. > a big reason we are in the replication crisis. The stated reasons are the same for different branches of science, while replication crisis is far from that. Replication of 100-year-old experiments is also not always easy/successful. Publish or perish motivates people to cut corners, but I think it is far from being the main reason. > What I was referring to is that we've been just sending papers to one another for centuries. Uploading to arxiv is just an easier form of that (and was the explicit reason for its existence). (Almost) nobody reads arxiv (edit: I mean (relevant part of) arxiv as a whole, of course most people read specific papers on arxiv). In fact the snr of arxiv is so low, even compared to journals that just by filtering garbage you can get quarter a mil followers [1]. So instead of random experts of varying quality and motivation I'll get a single rando on twitter (no offense to A K, he does good job)? No thanks. > It only works when everyone acts in good faith, and at scale only requires a few bad actors to spoil the barrel. Citation needed? You could claim the same for science as whole, but self-correction turns to be extremely robust, even if slower than you may want. [1] https://twitter.com/_akhaliq https://twitter.com/_akhaliq
- _aavaa_ 3y ago
  I was very serious. The journals take works others have done (work often paid for with public money) and get unpaid volunteers to peer-review it. Next they require the authors to sign away their copyright to it. And then they turn right back around and sell it to very same universities where it came from for thousands of dollars a year. And for what? So they can have have profit margins rivaling big tech? (Their major expenses are lawyers and server costs for pdf documents…) And why do researchers publish there? Because they need to in order to keep their jobs, or to get funding. You need those high-impact-factor and prestigious journals. And so you sign away your research. Most 1st world researchers don’t care or don’t know about the costs. But most of the world is not so lucky. Yes you could go around begging the authors for a free copy (but definitely don’t go to scihub or library genesis and download them for free, that would be illegal…). The “space for improvements” starts with burning down the journals and scattering their ashes among the fields to fertilize new science.
  - EvgeniyZh 3y ago
    > burning down the journals and scattering their ashes among the fields to fertilize new science. Oh yeah, destroy working system and then maybe figure it out, works like a charm every time.
    - _aavaa_ 3y ago
      Again, we had scientific progress (and problems with it) before the current system. The only currency that matters is if your results can be reproduced, as as far as I'm aware the overwhelming majority of journals make no attempt at reproducing the results before accepting it. Hell, it doesn't seem like they even make an attempt at make string there's enough information provided to even attempt to reproduce it. When I come across a paper that does any sort of modelling, I simply assume I won't be able to reproduce it if the code isn't included. They all seem to leave out important information.
      - EvgeniyZh 3y ago
        > Again, we had scientific progress (and problems with it) before the current system. Again, the scale and accessibility of science is different now and then. We had commute before steam engine, it doesn't mean we can get rid of all engine-based transportation now without consequences. > The only currency that matters is if your results can be reproduced This is extreme reductionism. LHC results cannot be replicated unless you have multi billion particle accelerator in your backyard. JWST cannot be replicated. Many other large scale experiments are nearly impossible to replicate. In fact, these are often considered "important" ones. And this not necessarily due to price, but rather combination of price, unique expertise and willing to spend effort. You know who will be able to tell you if the experiment description makes sense and has chances to be replicated if you ask them? Peers. Peer review at will never worked (and there were attempts) and never will because nobody likes to do peer review. I'm yet to see any proposal for peer review without editorial invitations which has at least theoretical chances to succeed. > I won't be able to reproduce it if the code isn't included. So you do mean "reproduction", not "replication". This is very low threshold and definitely not "only currency that matters". If I send you the code that alters the results somewhere deep inside, you'll be able to reproduce them.