7 ms·

Academic publishing as ruinous competition: Is there a way out?

71 points by randomwalker 13y ago

ekm2 13y ago
We need to have HN for academic research papers:Karma counts as the citation index,or something similar.
- Bjoern 13y ago
  Interestingly, Mendeley [1] tried to be (one of many) social platform with the specific aim to find more easily relevant papers for you. Unfortuantely, (or fortuantely for the founders?) it was bought recently by Elsevier, so I'm curious to see what will happen to it. Finding good relevant papers is still a difficult task imo. [1] http://www.mendeley.com/ http://www.mendeley.com/
  - tensor 13y ago
    A big problem with this is that academics tend to avoid social networking platforms like the plague.
- rfnslyr 13y ago
  I want to work on a website for all sorts of publishers, invite / interview process to post as to have no shitposting. Users will be able to register and set a monthly reading budget, which will be divided equally amongst all the authors you read in X time period. I want this service to have an API so you can easily push content to other platforms like Tumblr, facebook, or your own personal website, housing all your content on my site. I would use this. I believe in paying people for their hard work and I want the money to go directly into their pockets. No overhead.
- deleted 13y ago
  [deleted]
- czr80 13y ago
  Good != Popular
- betterunix 13y ago
  I see a bigger problem: the fact that things like citation counts are being used as a metric. We should be judging researchers by whether or not they are solving problems (or if they are helping other researchers solve problems -- like developing new research tools and techniques, or publishing negative results that tell other researchers what to avoid). It makes no difference whether a paper is cited one time or a hundred times. It makes no difference if a research publishes one big paper or ten smaller papers. What matters is whether or not progress is being made.
  - czr80 13y ago
    But this avoids the central issue - how can we know if a paper helps people make progress on solving problems? Citation counts are just a proxy for exactly that - to move away from them you have to come up with a better proxy or evaluation method.
    - betterunix 13y ago
      Conferences are a good starting point: researchers meet each other, share work, judge others' work, and discuss research. I am not saying it is a complete solution, just that citation counts are a terrible solution that encourage the wrong things (and that we need something better).
    - testbro 13y ago
      In the UK universities' research output are assessed by HEFCE using the "Research Excellence Framework" (which replaced the Research Assessment Exercise) [1]. Sadly, it suffers the same problem of being another metric [2]. It's a time sink for academics and it fosters a cutthroat atmosphere in the run-up. I heard some unconfirmed rumours that academics not submitting case studies to the REF will have their contracts re-assessed. I think this exposes the bigger problem of metrics in general. It's hard to determine how much impact research has, particularly in the short term. I'd be interested to see what the outcome of the REF is, but I remain sceptical and apathetic towards it, as does everyone else I've spoken to. [1] : http://www.ref.ac.uk/ http://www.ref.ac.uk/ [2] : I get the impression it does anyway - I'm not involved in it this round.
- ryguytilidie 13y ago
  There would be an insanely high correlation between school prestige and karma though :\
  - acadien 13y ago
    I've never noticed a correlation between school prestige and paper citation in my field, of course that doesn't mean there isn't one. I'm curious if you have anything to back up this statement or if its just anecdotal?
    - ssivark 13y ago
      I've come across quite a bit of anecdotal evidence as blog posts (Damn it, can't find the links now) and scientists will often confess to being biased about prestige while reading articles. It's also probably the reason why mathematical papers do not print author affiliations at the front of the paper (they are listed at the back). It's simply a matter of choosing where to invest your scarce time. A paper from prestigious authors/affiliation presumably has a better chance of being worth your time. So such papers get more attention and hence more citations, even if they might not have been the most novel or the most comprehensive. The same also applies to papers in high impact factor journals where citations have some kind of a positive feedback loop.
    - ryguytilidie 13y ago
      It's just anecdotal, but it is a lot of anecdotes. Both of my parents are professors, one at a much more prestigious school than the other. I've worked for a regular publisher, an academic publisher and I went to graduate school myself, so I've had a chat about this exact situation many times, though I admit I have no hard data.
- rmc 13y ago
  You need to find a way to prevent people from hacking it. Google page rank is similar, and people were motivated to hack it
- SapphireSun 13y ago
  This happened a few years ago (~2008?). HNA (HN Academic) was launched and didn't get a lot of traffic. Reading papers just takes a lot of time. There might be other models that work (other people mention Medley). The attention span for an HN clone is too short though.
dvt 13y ago
I would argue that this "ruinous competition" is a direct implication of primary, undergraduate, and graduate institutions. I've said this before on HN (there was another thread about science about a week ago), but I believe that schools have failed in doing what they are supposed to be doing: teaching. Instead, they are slowly turning into Olympiads - into fierce competitions that pit students against other students solely for the purpose of producing some kind of taxonomy (he has a 3.2, she has a 3.8 - she's clearly better/smarter). It's only natural that the same kind of competitive spirit would make its way into professional academia. I'm 27 and finishing my undergraduate degree at a somewhat elite institution (top-20 worldwide according to THE), and even though after a brief stint of working "in the real world" as a software engineer and doing some start-upping, I was considering either law school or academia (once I finished my degree), right now my only thought is: fuck no! It's not that I don't have the grades or that I may not be smart enough, it's just that I can't stand the intellectual dishonesty and political machinations. I'd rather eat Ramen for a whole year while coddling my newest start-up rather than partake in the mentally-masturbatory hallowed halls of most academic institutions. I'd rather build (and sell) new and interesting things that might actually have an effect on people instead of working harder trying to publish my papers than writing them.
infinity0 13y ago
I think academia has a lot to learn from internet-based free open source projects. I find much easier to learn from RFCs, git-hosted documentation projects, and wikis. (Going from more professional/less open to less professional/more open - not that these are necessarily exclusive, but it seems costly to have a system that satisfies both.) For documentation, I really love sphinx[1] and rst[2]. I know some people love markdown and it's good for light manuals/READMEs but IMO not adequate for full documentation. [1] http://sphinx-doc.org/ http://sphinx-doc.org/ [2] http://docutils.sourceforge.net/docs/user/rst/quickref.html http://docutils.sourceforge.net/docs/user/rst/quickref.html
- javert 13y ago
  I really disagree. Writing actual papers and having them peer reviewed is a useful model for actually doing science, and you can learn a ton from reading good academic papers. I don't know why you bring up documentation, the current tools used to write academic papers seem fine. As the OP states, the problem is the incentive system, not the particular set of tools and practices. (However, I don't think his suggestions are fundamental enough to change things, and I am skeptical about his claimed expertise on the stock market and copyright.)
  - jbooth 13y ago
    Writing down your thoughts and having them read by people is indeed pretty critical for science. What does the current publishing model do for us that a blog wouldn't? What does academia-speak accomplish? From the few CS papers I've read, generally speaking the harder the paper is to understand, the less profound the insights contained. What does the formalism get us, besides formalism itself?
    - glesica 13y ago
      Peer review doesn't demand arcane terminology or difficult-to-understand writing. Ideally, peer review is just someone writing down some thoughts and other people who have some expertise in the field providing comments on those thoughts before they are published. The problem with a blog is that there is no way to screen for quality or obvious mistakes. Of course the current method of doing this might be imperfect, or even completely broken depending on the field, but the solution isn't to just tell academics to start blogs and call it a day. It's a more complex problem than that.
frossie 13y ago
There's work being done in this area; whether it is going to be ultimately successful I don't know. In terms of generating metrics better than journal publications and citations, there's projects like this: http://impactstory.org/ http://impactstory.org/ (no connection, I just heard of them in this context). These kind of metrics would have addressed the OP's "blogger" problem. Another strand, which at least my small scientific field is pursuing, is to facilitate citations of datasets, or dataset+software combinations (think of it as making a directory structure citeable). This (in theory - it's a recent development) rewards people who take their time to clean up and give open access to their datasets, instead of doing the minimum to extract a couple of papers out of it and move on.
abecedarius 13y ago
I would recommend the book mentioned at the bottom as well -- http://www.amazon.com/Reinventing-Discovery-The-Networked-Science/dp/0691148902 http://www.amazon.com/Reinventing-Discovery-The-Networked-Sc...

71 points by randomwalker 13y ago

ekm2 13y ago

We need to have HN for academic research papers:Karma counts as the citation index,or something similar.

Bjoern 13y ago
Interestingly, Mendeley [1] tried to be (one of many) social platform with the specific aim to find more easily relevant papers for you. Unfortuantely, (or fortuantely for the founders?) it was bought recently by Elsevier, so I'm curious to see what will happen to it. Finding good relevant papers is still a difficult task imo. [1] http://www.mendeley.com/ http://www.mendeley.com/
- tensor 13y ago
  A big problem with this is that academics tend to avoid social networking platforms like the plague.
rfnslyr 13y ago
I want to work on a website for all sorts of publishers, invite / interview process to post as to have no shitposting. Users will be able to register and set a monthly reading budget, which will be divided equally amongst all the authors you read in X time period. I want this service to have an API so you can easily push content to other platforms like Tumblr, facebook, or your own personal website, housing all your content on my site. I would use this. I believe in paying people for their hard work and I want the money to go directly into their pockets. No overhead.
deleted 13y ago
[deleted]
czr80 13y ago
Good != Popular
betterunix 13y ago
I see a bigger problem: the fact that things like citation counts are being used as a metric. We should be judging researchers by whether or not they are solving problems (or if they are helping other researchers solve problems -- like developing new research tools and techniques, or publishing negative results that tell other researchers what to avoid). It makes no difference whether a paper is cited one time or a hundred times. It makes no difference if a research publishes one big paper or ten smaller papers. What matters is whether or not progress is being made.
- czr80 13y ago
  But this avoids the central issue - how can we know if a paper helps people make progress on solving problems? Citation counts are just a proxy for exactly that - to move away from them you have to come up with a better proxy or evaluation method.
  - betterunix 13y ago
    Conferences are a good starting point: researchers meet each other, share work, judge others' work, and discuss research. I am not saying it is a complete solution, just that citation counts are a terrible solution that encourage the wrong things (and that we need something better).
  - testbro 13y ago
    In the UK universities' research output are assessed by HEFCE using the "Research Excellence Framework" (which replaced the Research Assessment Exercise) [1]. Sadly, it suffers the same problem of being another metric [2]. It's a time sink for academics and it fosters a cutthroat atmosphere in the run-up. I heard some unconfirmed rumours that academics not submitting case studies to the REF will have their contracts re-assessed. I think this exposes the bigger problem of metrics in general. It's hard to determine how much impact research has, particularly in the short term. I'd be interested to see what the outcome of the REF is, but I remain sceptical and apathetic towards it, as does everyone else I've spoken to. [1] : http://www.ref.ac.uk/ http://www.ref.ac.uk/ [2] : I get the impression it does anyway - I'm not involved in it this round.
ryguytilidie 13y ago
There would be an insanely high correlation between school prestige and karma though :\
- acadien 13y ago
  I've never noticed a correlation between school prestige and paper citation in my field, of course that doesn't mean there isn't one. I'm curious if you have anything to back up this statement or if its just anecdotal?
  - ssivark 13y ago
    I've come across quite a bit of anecdotal evidence as blog posts (Damn it, can't find the links now) and scientists will often confess to being biased about prestige while reading articles. It's also probably the reason why mathematical papers do not print author affiliations at the front of the paper (they are listed at the back). It's simply a matter of choosing where to invest your scarce time. A paper from prestigious authors/affiliation presumably has a better chance of being worth your time. So such papers get more attention and hence more citations, even if they might not have been the most novel or the most comprehensive. The same also applies to papers in high impact factor journals where citations have some kind of a positive feedback loop.
  - ryguytilidie 13y ago
    It's just anecdotal, but it is a lot of anecdotes. Both of my parents are professors, one at a much more prestigious school than the other. I've worked for a regular publisher, an academic publisher and I went to graduate school myself, so I've had a chat about this exact situation many times, though I admit I have no hard data.
rmc 13y ago
You need to find a way to prevent people from hacking it. Google page rank is similar, and people were motivated to hack it
SapphireSun 13y ago
This happened a few years ago (~2008?). HNA (HN Academic) was launched and didn't get a lot of traffic. Reading papers just takes a lot of time. There might be other models that work (other people mention Medley). The attention span for an HN clone is too short though.

dvt 13y ago

I would argue that this "ruinous competition" is a direct implication of primary, undergraduate, and graduate institutions. I've said this before on HN (there was another thread about science about a week ago), but I believe that schools have failed in doing what they are supposed to be doing: teaching. Instead, they are slowly turning into Olympiads - into fierce competitions that pit students against other students solely for the purpose of producing some kind of taxonomy (he has a 3.2, she has a 3.8 - she's clearly better/smarter). It's only natural that the same kind of competitive spirit would make its way into professional academia. I'm 27 and finishing my undergraduate degree at a somewhat elite institution (top-20 worldwide according to THE), and even though after a brief stint of working "in the real world" as a software engineer and doing some start-upping, I was considering either law school or academia (once I finished my degree), right now my only thought is: fuck no! It's not that I don't have the grades or that I may not be smart enough, it's just that I can't stand the intellectual dishonesty and political machinations. I'd rather eat Ramen for a whole year while coddling my newest start-up rather than partake in the mentally-masturbatory hallowed halls of most academic institutions. I'd rather build (and sell) new and interesting things that might actually have an effect on people instead of working harder trying to publish my papers than writing them.

infinity0 13y ago

I think academia has a lot to learn from internet-based free open source projects. I find much easier to learn from RFCs, git-hosted documentation projects, and wikis. (Going from more professional/less open to less professional/more open - not that these are necessarily exclusive, but it seems costly to have a system that satisfies both.) For documentation, I really love sphinx[1] and rst[2]. I know some people love markdown and it's good for light manuals/READMEs but IMO not adequate for full documentation. [1] http://sphinx-doc.org/ http://sphinx-doc.org/ [2] http://docutils.sourceforge.net/docs/user/rst/quickref.html http://docutils.sourceforge.net/docs/user/rst/quickref.html

javert 13y ago
I really disagree. Writing actual papers and having them peer reviewed is a useful model for actually doing science, and you can learn a ton from reading good academic papers. I don't know why you bring up documentation, the current tools used to write academic papers seem fine. As the OP states, the problem is the incentive system, not the particular set of tools and practices. (However, I don't think his suggestions are fundamental enough to change things, and I am skeptical about his claimed expertise on the stock market and copyright.)
- jbooth 13y ago
  Writing down your thoughts and having them read by people is indeed pretty critical for science. What does the current publishing model do for us that a blog wouldn't? What does academia-speak accomplish? From the few CS papers I've read, generally speaking the harder the paper is to understand, the less profound the insights contained. What does the formalism get us, besides formalism itself?
  - glesica 13y ago
    Peer review doesn't demand arcane terminology or difficult-to-understand writing. Ideally, peer review is just someone writing down some thoughts and other people who have some expertise in the field providing comments on those thoughts before they are published. The problem with a blog is that there is no way to screen for quality or obvious mistakes. Of course the current method of doing this might be imperfect, or even completely broken depending on the field, but the solution isn't to just tell academics to start blogs and call it a day. It's a more complex problem than that.

frossie 13y ago

There's work being done in this area; whether it is going to be ultimately successful I don't know. In terms of generating metrics better than journal publications and citations, there's projects like this: http://impactstory.org/ http://impactstory.org/ (no connection, I just heard of them in this context). These kind of metrics would have addressed the OP's "blogger" problem. Another strand, which at least my small scientific field is pursuing, is to facilitate citations of datasets, or dataset+software combinations (think of it as making a directory structure citeable). This (in theory - it's a recent development) rewards people who take their time to clean up and give open access to their datasets, instead of doing the minimum to extract a couple of papers out of it and move on.

abecedarius 13y ago

I would recommend the book mentioned at the bottom as well -- http://www.amazon.com/Reinventing-Discovery-The-Networked-Science/dp/0691148902 http://www.amazon.com/Reinventing-Discovery-The-Networked-Sc...