4 ms·

I truly don't understand why academics put up with journals. If I were in the biz, I'd be organizing a torches-and-pitchforks mob to burn them down and replace

I truly don't understand why academics put up with journals. If I were in the biz, I'd be organizing a torches-and-pitchforks mob to burn them down and replace them with arXiv, personal websites, and peer review panels who link to good papers.

bbgm 17y ago
Because that's not how you get funded. Your funding/tenure, etc are related to where you publish, how often you publish, etc. It's a systemic issue, a system designed for a different era which just does not translate to today's science
- azanar 17y ago
  The system needs to be redesigned, yet there is very little catalyst for it to be redesigned. People are still more motivated to turn their interests inward, to things like funding, publication lists, tenure, etc. The system is designed to reward those who selfishly work to maintain the status quo. This isn't me passing judgement on these people, just a realization that they are only doing what provides them with the greatest incentives. In a similar situation, I might do the same. But it is also a realization that this isn't just a shallow local minimum. The lifestyle and security of many researcher are tightly coupled to getting funding out of this system, and thus is tightly coupled to following the rules of the system. Either some external force is going to have to shove this system aside, or the members of the community are going to have to collectively accept the pain of a governing body seeing the writing on the wall and attempting to regain control. With the system as entrenched as this, it would have to be a coup of epic proportions to unseat it. The only other option I can see is this system remaining in perpetuity. I noted above that being in a similar position, I'd probably work this system the same way most everyone else does. I might provide the usual excuses as well: family, food, survival, lifestyle, etc. I don't know, because normally it takes a person being put under considerable pressure before they decide what is truly valuable to them, and to what degree they are willing to sacrifice one goal for another. I took a third option while going through my undergraduate years; I decided to not even get involved. I am not saying this as a sanctimonious thumbing-of-my-nose at those who did; these people have my respect for being able to work around the flaws and not be driven mad by them. The allure of being able to work on interesting problems with smart people tempted me toward working to get involved in research, but the glimpses I caught of the politics, mechanisms and incentives chased me away. My significant other at the time was involved in a research effort in computer vision, and the amount of inanity she tolerated, and even in some cases manufactured, killed the desire for me. Since then, I've been told by a number of people I have the mind and personality for research work. Even if that were true, I know I don't have the personality for all the bullshit that comes with it. My philosophy on this is the same that jwz suggests for people thinking of working at a public company: "sometimes the only way to win is not to play."
- JulianMorrison 17y ago
  So in my proposed system "publish" maps exactly to "get linked by this or that prestigious review panel". Everyone who replied me seems to have missed that. Basically, review panels become journals turned inside out. They don't host the content, but they do certify it.
  - bentoner 17y ago
    But is there really a difference then? Hosting the content is a tiny proportion of the work involved in running a journal. If you're going to go through all the effort required to peer-review and certify work, why wouldn't you also throw up a website and collect the papers you certify?
    - JulianMorrison 17y ago
      A larger part is deciding what not to host, given the crazy old-fashioned notion that journals ought also to have dead-tree versions, and therefore each paper costs them quite a lot for a small print run. Why would actual science come in neat magazine-sized packets? It's obvious they must be normalizing the signal, either by floating the cutoff or just picking until the slots are full and trashing the rest. A review panel that doesn't have to pay some company to stamp ink onto wood pulp can afford to link as many, or as few good papers as arrive. And, they can afford to do this asynchronously, as soon as they arrive.
  - bbgm 17y ago
    That is kind of what the Faculty of 1000 does today. Essentially picks papers regardless of source that they feel is significant. Perhaps in the future that will be the role of Science, Nature, etc with papers getting published all over the place. Of course, the whole peer review process needs an upgrade too, although I don't know if peer review as a concept is going away (nor should it)
bentoner 17y ago
All the things you mention are fine for dissemination but publishing in journals in not really about that any more; it's about certification. Experts in your area---those who can actually understand your paper---mostly won't care whether your paper's been peer-reviewed or is just on the arxiv. You get it published mainly for folks who can't (or don't want to have to) understand it. You put up with all the crap this entails because the latter category usually includes people responsible for hiring you, promoting you or awarding you grants.
- azanar 17y ago
  This has the same failings as a system of information distribution where an aristocracy is given the authority to decide what is and is not important and correct. This introduces all sorts of random factors having nothing to do with any sort of objective measurement of the publication. Those random factors don't even have to be nefarious; an expert in an area may not have sufficient sub-domain knowledge to fully understand the work of another author. The hope is that the publication is clear enough that they can, but every publication has to make a trade-off about how much knowledge they assume the reader has. Each of these random factors chisel away at the intended meaning of the certification. If enough of these accrete, you end up with something that tells as much about a potential publication as a GPA does about a high school student. But then, people depend on things like that for hiring and promotions as well. It would be a damn shame if this is really sufficiently optimal that everyone stops working for better.
- jvdh 17y ago
  I've just finished my PhD, so I'm an expert in a very small field. Yes, it is true that I've read a good paper that I do not care whether it is peer-reviewed, on arxiv, or just on a homepage. As long as it is of good quality and there is a way to cite it, I'll probably cite it in one of my own articles. However, and I'm not saying this out of defense for the current system, you must understand that the system is also in place to filter out some of the complete and total crap some people dare to call research. I've reviewed articles that made my toes curl, because of bad grammer, bad presentation, or just duplicating another article with 10% of the article changed. Already at the moment there are so many journals and magazines that I have to keep track of that I simply don't have the time to search the web or dig up things on arxiv. I simply have to trust some system to filter out the good articles.
jvdh 17y ago
The problem is that your articles get measured by the "quality" of the journals that they appear in. The problem is that the whole system is fundamentelly broken and it is going to take a complete mindshift of everyone involved to change that. The OpenAccess movement is something that may have a chance now that funders are also seeing the positive effects of it. But this is going to take years to change.

bbgm 17y ago

Because that's not how you get funded. Your funding/tenure, etc are related to where you publish, how often you publish, etc. It's a systemic issue, a system designed for a different era which just does not translate to today's science

azanar 17y ago
The system needs to be redesigned, yet there is very little catalyst for it to be redesigned. People are still more motivated to turn their interests inward, to things like funding, publication lists, tenure, etc. The system is designed to reward those who selfishly work to maintain the status quo. This isn't me passing judgement on these people, just a realization that they are only doing what provides them with the greatest incentives. In a similar situation, I might do the same. But it is also a realization that this isn't just a shallow local minimum. The lifestyle and security of many researcher are tightly coupled to getting funding out of this system, and thus is tightly coupled to following the rules of the system. Either some external force is going to have to shove this system aside, or the members of the community are going to have to collectively accept the pain of a governing body seeing the writing on the wall and attempting to regain control. With the system as entrenched as this, it would have to be a coup of epic proportions to unseat it. The only other option I can see is this system remaining in perpetuity. I noted above that being in a similar position, I'd probably work this system the same way most everyone else does. I might provide the usual excuses as well: family, food, survival, lifestyle, etc. I don't know, because normally it takes a person being put under considerable pressure before they decide what is truly valuable to them, and to what degree they are willing to sacrifice one goal for another. I took a third option while going through my undergraduate years; I decided to not even get involved. I am not saying this as a sanctimonious thumbing-of-my-nose at those who did; these people have my respect for being able to work around the flaws and not be driven mad by them. The allure of being able to work on interesting problems with smart people tempted me toward working to get involved in research, but the glimpses I caught of the politics, mechanisms and incentives chased me away. My significant other at the time was involved in a research effort in computer vision, and the amount of inanity she tolerated, and even in some cases manufactured, killed the desire for me. Since then, I've been told by a number of people I have the mind and personality for research work. Even if that were true, I know I don't have the personality for all the bullshit that comes with it. My philosophy on this is the same that jwz suggests for people thinking of working at a public company: "sometimes the only way to win is not to play."
JulianMorrison 17y ago
So in my proposed system "publish" maps exactly to "get linked by this or that prestigious review panel". Everyone who replied me seems to have missed that. Basically, review panels become journals turned inside out. They don't host the content, but they do certify it.
- bentoner 17y ago
  But is there really a difference then? Hosting the content is a tiny proportion of the work involved in running a journal. If you're going to go through all the effort required to peer-review and certify work, why wouldn't you also throw up a website and collect the papers you certify?
  - JulianMorrison 17y ago
    A larger part is deciding what not to host, given the crazy old-fashioned notion that journals ought also to have dead-tree versions, and therefore each paper costs them quite a lot for a small print run. Why would actual science come in neat magazine-sized packets? It's obvious they must be normalizing the signal, either by floating the cutoff or just picking until the slots are full and trashing the rest. A review panel that doesn't have to pay some company to stamp ink onto wood pulp can afford to link as many, or as few good papers as arrive. And, they can afford to do this asynchronously, as soon as they arrive.
- bbgm 17y ago
  That is kind of what the Faculty of 1000 does today. Essentially picks papers regardless of source that they feel is significant. Perhaps in the future that will be the role of Science, Nature, etc with papers getting published all over the place. Of course, the whole peer review process needs an upgrade too, although I don't know if peer review as a concept is going away (nor should it)

bentoner 17y ago

All the things you mention are fine for dissemination but publishing in journals in not really about that any more; it's about certification. Experts in your area---those who can actually understand your paper---mostly won't care whether your paper's been peer-reviewed or is just on the arxiv. You get it published mainly for folks who can't (or don't want to have to) understand it. You put up with all the crap this entails because the latter category usually includes people responsible for hiring you, promoting you or awarding you grants.

azanar 17y ago
This has the same failings as a system of information distribution where an aristocracy is given the authority to decide what is and is not important and correct. This introduces all sorts of random factors having nothing to do with any sort of objective measurement of the publication. Those random factors don't even have to be nefarious; an expert in an area may not have sufficient sub-domain knowledge to fully understand the work of another author. The hope is that the publication is clear enough that they can, but every publication has to make a trade-off about how much knowledge they assume the reader has. Each of these random factors chisel away at the intended meaning of the certification. If enough of these accrete, you end up with something that tells as much about a potential publication as a GPA does about a high school student. But then, people depend on things like that for hiring and promotions as well. It would be a damn shame if this is really sufficiently optimal that everyone stops working for better.
jvdh 17y ago
I've just finished my PhD, so I'm an expert in a very small field. Yes, it is true that I've read a good paper that I do not care whether it is peer-reviewed, on arxiv, or just on a homepage. As long as it is of good quality and there is a way to cite it, I'll probably cite it in one of my own articles. However, and I'm not saying this out of defense for the current system, you must understand that the system is also in place to filter out some of the complete and total crap some people dare to call research. I've reviewed articles that made my toes curl, because of bad grammer, bad presentation, or just duplicating another article with 10% of the article changed. Already at the moment there are so many journals and magazines that I have to keep track of that I simply don't have the time to search the web or dig up things on arxiv. I simply have to trust some system to filter out the good articles.

jvdh 17y ago

The problem is that your articles get measured by the "quality" of the journals that they appear in. The problem is that the whole system is fundamentelly broken and it is going to take a complete mindshift of everyone involved to change that. The OpenAccess movement is something that may have a chance now that funders are also seeing the positive effects of it. But this is going to take years to change.