3 ms·

I like your point but let's dive another layer down the rabbithole. Why do researchers need to buy prestige in the first place? Academic research should be don

by gtsop 3y ago

I like your point but let's dive another layer down the rabbithole. Why do researchers need to buy prestige in the first place? Academic research should be done for the sake of advancement of knowledge and of course the people doing it should make a good living out of it. How does buying prestige come into play here? (just asking)

foven 3y ago
Consider you are another researcher (not the one who has just published). You may want to keep up to date with your field. This is good to do and it helps foster new ideas for now and for future proposals. There will be a selection of journals that are relevant to you and they will publish work that, generally, will be tiered by either the scope or impact of the journal. Journal A publishes a lot of papers you find very insightful or useful, journal B publishes some work that is good to know but not profound. If you have maybe 20 minutes, maybe you will read one paper that catches your eye from Journal A. The truest metric of how good the work you do is, at a surface level, is how well cited it is. How much do people read it, and then do they care once they put it down. It is not necessarily the case that putting your paper in a high impact journal gets a lot of citations, but its a lot of exposure and it maximises those chances.
- gtsop 3y ago
  Maybe I am wrong here, but shouldn't the quality metric be whether it acts as a basis for practical applications? As far as I understand, research is supposed to give theoretical answers and frameworks. X works like this. Y doesn't work like that. This knowledge should then be applied to the real world and will be prooved wrong or right. Maybe a silly thought, but I suppose pre 1800 science didn't have to pay such fees to proove the quality of their work, they just applied it. (I understand not all papers can be immediatelly applied, but is that the majotity of the work? And if so, is that a good thing?)
  - deleted 3y ago
    [deleted]
  - 6nf 3y ago
    Unpublished science for practical applications of course exists but that is called R&D and is not generally made public cause it’s corporate IP. If that’s what you mean.
  - sseagull 3y ago
    As a society, we want to have a "portfolio" of research. We want short-term research with tangible results, moderate-term research that builds on the current state of the art and advances it, and long-term research that might pay off decades from now. You also want to mix in "moonshot" research that likely won't pay off, but if it did would be revolutionary. The proportion you invest in each is up for debate. Generally, as the time horizon increases, risk increases, and it becomes less and less likely any particular research direction will pay off, but you still need to do it or you won't have anything after all the current directions have dried up, matured, or otherwise run their course. I think the error people make is to assume academia is supposed to cover all three. Largely, academia covers long-term and maybe some moderate-term and moonshot. Short term is better handled by industry where there is a more direct measure of applicability ($$). But business is generally not interested in the longer term stuff because it is too risky. So academia as a whole is about forming the bricks that industry uses to build useful things. While any particular paper is not going to be useful to society anytime soon (if at all), the aggregate knowledge will likely be useful at some point in the future.
  - foldr 3y ago
    Most scientific research has no practical applications (or at least, not within the lifetime of the people doing it). Consider e.g. cosmology. The quality of research in cosmology varies, as in any other discipline, but even the very best research in cosmology has few if any immediate practical applications.
suddenclarity 3y ago
You need some kind of ranking. Anyone who have tried to write a scientific paper know how much junk there is being published through the "pay to publish" channels and it will just get worse with the technical development of chatGPT. Journals with high prestige tend to work as a barrier for junk.
- matwood 3y ago
  More generally, curation has value (see web search or even hardware like an iPhone). And to your point, curation value is increasing because the volume of stuff needing to be curated is growing at what feels like an exponential rate.
PeterisP 3y ago
The people who need to evaluate academics (both individuals and institutions) for purposes of funding, promotions, tracking goals, etc, can't really do it well on their own (you need to have domain-specific knowledge to be able to pass independent judgement, you can't have domain-specific knowledge in all the disciplines you need to manage, and you can't/don't trust self-evaluations) so they need some independents oracle that gives any metrics that are at least roughly correlated with output and quality even if they inevitably get gamed; and it needs to be something that can be independently evaluated cheaply and simply, so that it can be written into some formal criteria and there won't be a legal dispute whether a research outcome is "sufficiently good" or not, or whether applicant A has "more points" than applicant B on some objective scale that doesn't put the burden of personally proving that on some official who realistically can't do that anyway. Publishers, prestige of publication venue (including their ranking in citation metrics), indexes like Web of Science/Scopus/etc all provide this function to the administrators of scientific funding, who want (and need) to evaluate academic prestige. The other need is for a filter to allow a researcher to not access most of what is published. There is generally a problem of too much "research-like work" that is impossible to read all of what gets written, so researchers want a filter system where someone else throws out at least some of the crap (or forces the crap to be re-written to be better) so that there's a better signal-to-noise ratio. In the current system where there's motivation to publish in "better places", if something is published in a "worse place" that's some evidence that the paper is weak (because if the authors could have published it in a better place, why didn't they?) and thus perhaps can be discarded without reading, or deprioritized to read only after you've read all of the better-ranked stuff.
bradleyjg 3y ago
It’s proxies all the way down. Evaluating candidates is really hard so employers rely on the prestige of their colleges. Evaluating professors is really hard so colleges rely on the prestige of their publications. Evaluating papers from post docs is really hard, so referees rely on the prestige of their advisors/institutions. Graduate admissions is really hard so admissions officers rely on the prestige of authors of letters of recommendation. Undergrad admissions is really hard so admissions officers rely on SAT and diversity factors.

by gtsop 3y ago

foven 3y ago

Consider you are another researcher (not the one who has just published). You may want to keep up to date with your field. This is good to do and it helps foster new ideas for now and for future proposals. There will be a selection of journals that are relevant to you and they will publish work that, generally, will be tiered by either the scope or impact of the journal. Journal A publishes a lot of papers you find very insightful or useful, journal B publishes some work that is good to know but not profound. If you have maybe 20 minutes, maybe you will read one paper that catches your eye from Journal A. The truest metric of how good the work you do is, at a surface level, is how well cited it is. How much do people read it, and then do they care once they put it down. It is not necessarily the case that putting your paper in a high impact journal gets a lot of citations, but its a lot of exposure and it maximises those chances.

gtsop 3y ago
Maybe I am wrong here, but shouldn't the quality metric be whether it acts as a basis for practical applications? As far as I understand, research is supposed to give theoretical answers and frameworks. X works like this. Y doesn't work like that. This knowledge should then be applied to the real world and will be prooved wrong or right. Maybe a silly thought, but I suppose pre 1800 science didn't have to pay such fees to proove the quality of their work, they just applied it. (I understand not all papers can be immediatelly applied, but is that the majotity of the work? And if so, is that a good thing?)
- deleted 3y ago
  [deleted]
- 6nf 3y ago
  Unpublished science for practical applications of course exists but that is called R&D and is not generally made public cause it’s corporate IP. If that’s what you mean.
- sseagull 3y ago
  As a society, we want to have a "portfolio" of research. We want short-term research with tangible results, moderate-term research that builds on the current state of the art and advances it, and long-term research that might pay off decades from now. You also want to mix in "moonshot" research that likely won't pay off, but if it did would be revolutionary. The proportion you invest in each is up for debate. Generally, as the time horizon increases, risk increases, and it becomes less and less likely any particular research direction will pay off, but you still need to do it or you won't have anything after all the current directions have dried up, matured, or otherwise run their course. I think the error people make is to assume academia is supposed to cover all three. Largely, academia covers long-term and maybe some moderate-term and moonshot. Short term is better handled by industry where there is a more direct measure of applicability ($$). But business is generally not interested in the longer term stuff because it is too risky. So academia as a whole is about forming the bricks that industry uses to build useful things. While any particular paper is not going to be useful to society anytime soon (if at all), the aggregate knowledge will likely be useful at some point in the future.
- foldr 3y ago
  Most scientific research has no practical applications (or at least, not within the lifetime of the people doing it). Consider e.g. cosmology. The quality of research in cosmology varies, as in any other discipline, but even the very best research in cosmology has few if any immediate practical applications.

suddenclarity 3y ago

You need some kind of ranking. Anyone who have tried to write a scientific paper know how much junk there is being published through the "pay to publish" channels and it will just get worse with the technical development of chatGPT. Journals with high prestige tend to work as a barrier for junk.

matwood 3y ago
More generally, curation has value (see web search or even hardware like an iPhone). And to your point, curation value is increasing because the volume of stuff needing to be curated is growing at what feels like an exponential rate.

PeterisP 3y ago

The people who need to evaluate academics (both individuals and institutions) for purposes of funding, promotions, tracking goals, etc, can't really do it well on their own (you need to have domain-specific knowledge to be able to pass independent judgement, you can't have domain-specific knowledge in all the disciplines you need to manage, and you can't/don't trust self-evaluations) so they need some independents oracle that gives any metrics that are at least roughly correlated with output and quality even if they inevitably get gamed; and it needs to be something that can be independently evaluated cheaply and simply, so that it can be written into some formal criteria and there won't be a legal dispute whether a research outcome is "sufficiently good" or not, or whether applicant A has "more points" than applicant B on some objective scale that doesn't put the burden of personally proving that on some official who realistically can't do that anyway. Publishers, prestige of publication venue (including their ranking in citation metrics), indexes like Web of Science/Scopus/etc all provide this function to the administrators of scientific funding, who want (and need) to evaluate academic prestige. The other need is for a filter to allow a researcher to not access most of what is published. There is generally a problem of too much "research-like work" that is impossible to read all of what gets written, so researchers want a filter system where someone else throws out at least some of the crap (or forces the crap to be re-written to be better) so that there's a better signal-to-noise ratio. In the current system where there's motivation to publish in "better places", if something is published in a "worse place" that's some evidence that the paper is weak (because if the authors could have published it in a better place, why didn't they?) and thus perhaps can be discarded without reading, or deprioritized to read only after you've read all of the better-ranked stuff.

bradleyjg 3y ago

It’s proxies all the way down. Evaluating candidates is really hard so employers rely on the prestige of their colleges. Evaluating professors is really hard so colleges rely on the prestige of their publications. Evaluating papers from post docs is really hard, so referees rely on the prestige of their advisors/institutions. Graduate admissions is really hard so admissions officers rely on the prestige of authors of letters of recommendation. Undergrad admissions is really hard so admissions officers rely on SAT and diversity factors.