4 ms·

> The pressure will be for quality, not quantity. Well, popularity, anyway. While popularity is sometimes a useful proxy for quality, that's not always the cas

> The pressure will be for quality, not quantity. Well, popularity, anyway. While popularity is sometimes a useful proxy for quality, that's not always the case.

FredPret 4y ago
Probably the best proxy for quality is popularity that endures for centuries since we can’t measure quality directly. Either way, if there’s a max of one paper per year, scientists will be trying for the most impactful work.
- Retric 4y ago
  People are already boosting their impact scores by requiring papers they review to add citations for their past work. Performance Metrics always result in people gaming the system, and impact scores have quickly become unreliable.
  - ModernMech 4y ago
    At the same time, reviewers are chosen based on their expertise in a field, and one of the things reviewers look for is that new papers cite relevant prior research. Are you suggesting that reviewers should just let papers go through without citing relevant prior research if it is their own research? Does it actually matter whose research it is as long as it is relevant prior research?
    - Retric 4y ago
      Of course it matters, adding citations to already written research is deceptive and thus actively harmful. The point of citations isn’t simply an index of prior research it’s there to inform readers of the writers context.
      - ModernMech 4y ago
        When a paper is rejected because it doesn't contain references to relevant prior research, the citations shouldn't simply be added; the relevant prior research that wasn't considered should thus be considered, and the paper should be reworked with that new context in mind. Research should be put into the proper context in the field, not based on what the writer personally looked at. If a researcher's context is myopic enough not to include relevant prior work known or done by reviewers, then the research itself is of questionable value. If someone submits work that purports to be novel, yet they haven't done a proper literature review to include relevant studies, I'd say that is itself deceptive and more actively harmful to the field as a whole.
        Retric 4y ago
        Suppose an experiment takes 5 years, nothing written after the experiment begin should have any barring on what was done and the resulting data. The same is true to a lesser extent for any research. Now if the field has discovered some flaw in what was done then things go well beyond the simple need for a citation.
        ModernMech 4y ago
        That's not how competent research is done. You don't go into a hole for 5 years and then come back into the world with a fully-formed paper. While you're doing that research, the world moves on, and if your citations are from 5 years ago, any publication will ask the extremely appropriate question "How does this matter w.r.t. the research published since you started this work?", and send the paper back for rewrite. Or, it may ask the even more appropriate question "Why does your research matter given the results of papers x,y,z which are directly related to your research question?" Any competent researcher should preempt in their literature review, and if they can't then it's not research but instead PR. And this doesn't even touch upon the fact that research that's going to take 5 years is going to be well-funded, and you'll have to write interim reports over that period. When you get 5 years of funding, you don't get that all at once. During that time, you'll have to answer for advances in the field and justify why the research is still worth doing.

> The pressure will be for quality, not quantity. Well, popularity, anyway. While popularity is sometimes a useful proxy for quality, that's not always the case.

FredPret 4y ago

Probably the best proxy for quality is popularity that endures for centuries since we can’t measure quality directly. Either way, if there’s a max of one paper per year, scientists will be trying for the most impactful work.

Retric 4y ago
People are already boosting their impact scores by requiring papers they review to add citations for their past work. Performance Metrics always result in people gaming the system, and impact scores have quickly become unreliable.
- ModernMech 4y ago
  At the same time, reviewers are chosen based on their expertise in a field, and one of the things reviewers look for is that new papers cite relevant prior research. Are you suggesting that reviewers should just let papers go through without citing relevant prior research if it is their own research? Does it actually matter whose research it is as long as it is relevant prior research?
  - Retric 4y ago
    Of course it matters, adding citations to already written research is deceptive and thus actively harmful. The point of citations isn’t simply an index of prior research it’s there to inform readers of the writers context.
    - ModernMech 4y ago
      When a paper is rejected because it doesn't contain references to relevant prior research, the citations shouldn't simply be added; the relevant prior research that wasn't considered should thus be considered, and the paper should be reworked with that new context in mind. Research should be put into the proper context in the field, not based on what the writer personally looked at. If a researcher's context is myopic enough not to include relevant prior work known or done by reviewers, then the research itself is of questionable value. If someone submits work that purports to be novel, yet they haven't done a proper literature review to include relevant studies, I'd say that is itself deceptive and more actively harmful to the field as a whole.
      - Retric 4y ago
        Suppose an experiment takes 5 years, nothing written after the experiment begin should have any barring on what was done and the resulting data. The same is true to a lesser extent for any research. Now if the field has discovered some flaw in what was done then things go well beyond the simple need for a citation.
        ModernMech 4y ago
        That's not how competent research is done. You don't go into a hole for 5 years and then come back into the world with a fully-formed paper. While you're doing that research, the world moves on, and if your citations are from 5 years ago, any publication will ask the extremely appropriate question "How does this matter w.r.t. the research published since you started this work?", and send the paper back for rewrite. Or, it may ask the even more appropriate question "Why does your research matter given the results of papers x,y,z which are directly related to your research question?" Any competent researcher should preempt in their literature review, and if they can't then it's not research but instead PR. And this doesn't even touch upon the fact that research that's going to take 5 years is going to be well-funded, and you'll have to write interim reports over that period. When you get 5 years of funding, you don't get that all at once. During that time, you'll have to answer for advances in the field and justify why the research is still worth doing.