4 ms·

As the article states, papers are mostly career advancement tools and scientists are incentivized to put the least amount of useful information into them they c

by hoseja 5y ago

As the article states, papers are mostly career advancement tools and scientists are incentivized to put the least amount of useful information into them they can get away with. Real scientists mostly learn from their instructors who possess all the jealously guarded institutional knowledge. Yes, it is very broken.

jokteur 5y ago
It is always difficult to try to understand and implement the theory explained in papers which seem fine on the surface, but when you look more closely, you find a bunch of mistakes, there are giant holes in the details, and you end up trying to redo the whole paper. There should be journals/websites/blogs dedicated to trying to reexplain / implement papers.
- bluGill 5y ago
  Doing a good review is hard. I've been reviewing a few C++ papers (things proposed to C++23) lately. Many of them are over my head and all I can find are a few spelling errors. The ones I've understood took me 3 readings before I found some giant holes in the details (which I pointed out to the authors, the next revision corrected them). In one case I actually started implementing the feature myself, and only then did I realize there was something ambiguous in the paper (after talking with the author we decided to document an implementors choices as it doesn't matter for the 99% use case, and the rest could go either way depending on hardware and data details so better to allow choice) The vast majority if papers I'm far too laze to go into that level of detail on. I just assume the authors is a smart person and so I trustingly let them go. It may well be if I understood the paper I'd be horrified, ask me in 2043 when we have 20 years of hindsight... I have to believe that peer review is the same - many reviewers are just reading and looking for something obvious but not really understanding details.
  - Mezzie 5y ago
    SO difficult, especially given that just because you're in the 'same' field and technically qualified to do a peer review doesn't mean you actually understand what you're reading. For example, I'm qualified to review papers on educational programs for children. I should never be asked to do that.
  - mellavora 5y ago
    Someone once said that they enjoyed one of my papers, and that even though they thought the writing was very clear they still had to read it 3 times to understand it. I told them that I had to write it 100 times and spend two years before I understood it. So if they could pick it up in 3 readings over 3 days, they were doing pretty good.
civilized 5y ago
It's hard for me to even comprehend how this could be true, but it does sound familiar enough from credible sources that maybe it's right regardless of what makes sense to me.
anonymousDan 5y ago
I mean honestly this is just total bullshit. There is plenty of value in academic papers. It's just that there is very little money to be made in developing tools such as those mentioned by the OP as there is very little money in academia.
- viewfromafar 5y ago
  I understood the criticism directed at the value of papers as instruments of knowledge sharing. The argument is not that papers are completely useless in terms of knowledge sharing but that this pure purpose of dissemination is largely overshadowed by considerations of carreer, prestige, funding or any interest other than knowledge sharing. This is the world we live in. A scientist is a person that needs to make a living and is subject to various constraints. The reason that there is little money to be made is that society hasn't found a way to set up the scientific process in such a way that the constraints would value the increase in public domain knowledge higher than the incentives to hold some knowledge back. Part of this may stem from leaving specialized knowledge to academia while letting only companies reap the monetary rewards of putting the knowledge to use. Society benefits only indirectly (better drugs, machines, etc) but industry players will rather shield knowledge and adapt its representation to their own needs.
- Fomite 5y ago
  Very little money, and more than that, very little unallocated money. I'm fortunate enough to have a decently well funded lab, but almost all of that funding is spoken for. Your Compelling New Product needs to be compelling enough for me to put it into grants anticipating it'll still be around in several years, often enough to get around most grants not being funded.
mellavora 5y ago
Hard disagree. With a caveat -- I do acknowledge that for an important number of professional academics your statement may be true, and I have heard a former post-doc at ETH Zurich describe their papers as career points (so also a grain of truth at elite institutes). But for most of the academics I have known and worked with, publications are taken quite seriously, and institutional knowledge is freely shared. There is an incentive to reduce the content in papers, but it is out of respect for the reader (a paper is not a textbook) and an honest attempt to limit the discussion to the core hypothesis of the work. You have 6 pages to 1) describe the content of 6*100 pages (the 100 other relevant papers on the topic), 2) present your addition to this body of knowledge, 3) discuss the insights your work brings, again referring to the content of 600 pages. and those 600 pages you are summarizing are as information-dense as your work.
- Fomite 5y ago
  Also hard disagree. Everyone I know takes papers quite seriously, there's a major push to get more information into the literature (publish null effect estimates!), etc. This feels a lot like the graduate students on the Academia StackExchange who are convinced the moment they present their idea it will get stolen, while every faculty member is like "I have a list of my own ideas that I don't have time to work on as long as my arm."

by hoseja 5y ago

jokteur 5y ago

It is always difficult to try to understand and implement the theory explained in papers which seem fine on the surface, but when you look more closely, you find a bunch of mistakes, there are giant holes in the details, and you end up trying to redo the whole paper. There should be journals/websites/blogs dedicated to trying to reexplain / implement papers.

bluGill 5y ago
Doing a good review is hard. I've been reviewing a few C++ papers (things proposed to C++23) lately. Many of them are over my head and all I can find are a few spelling errors. The ones I've understood took me 3 readings before I found some giant holes in the details (which I pointed out to the authors, the next revision corrected them). In one case I actually started implementing the feature myself, and only then did I realize there was something ambiguous in the paper (after talking with the author we decided to document an implementors choices as it doesn't matter for the 99% use case, and the rest could go either way depending on hardware and data details so better to allow choice) The vast majority if papers I'm far too laze to go into that level of detail on. I just assume the authors is a smart person and so I trustingly let them go. It may well be if I understood the paper I'd be horrified, ask me in 2043 when we have 20 years of hindsight... I have to believe that peer review is the same - many reviewers are just reading and looking for something obvious but not really understanding details.
- Mezzie 5y ago
  SO difficult, especially given that just because you're in the 'same' field and technically qualified to do a peer review doesn't mean you actually understand what you're reading. For example, I'm qualified to review papers on educational programs for children. I should never be asked to do that.
- mellavora 5y ago
  Someone once said that they enjoyed one of my papers, and that even though they thought the writing was very clear they still had to read it 3 times to understand it. I told them that I had to write it 100 times and spend two years before I understood it. So if they could pick it up in 3 readings over 3 days, they were doing pretty good.

civilized 5y ago

It's hard for me to even comprehend how this could be true, but it does sound familiar enough from credible sources that maybe it's right regardless of what makes sense to me.

anonymousDan 5y ago

I mean honestly this is just total bullshit. There is plenty of value in academic papers. It's just that there is very little money to be made in developing tools such as those mentioned by the OP as there is very little money in academia.

viewfromafar 5y ago
I understood the criticism directed at the value of papers as instruments of knowledge sharing. The argument is not that papers are completely useless in terms of knowledge sharing but that this pure purpose of dissemination is largely overshadowed by considerations of carreer, prestige, funding or any interest other than knowledge sharing. This is the world we live in. A scientist is a person that needs to make a living and is subject to various constraints. The reason that there is little money to be made is that society hasn't found a way to set up the scientific process in such a way that the constraints would value the increase in public domain knowledge higher than the incentives to hold some knowledge back. Part of this may stem from leaving specialized knowledge to academia while letting only companies reap the monetary rewards of putting the knowledge to use. Society benefits only indirectly (better drugs, machines, etc) but industry players will rather shield knowledge and adapt its representation to their own needs.
Fomite 5y ago
Very little money, and more than that, very little unallocated money. I'm fortunate enough to have a decently well funded lab, but almost all of that funding is spoken for. Your Compelling New Product needs to be compelling enough for me to put it into grants anticipating it'll still be around in several years, often enough to get around most grants not being funded.

mellavora 5y ago

Hard disagree. With a caveat -- I do acknowledge that for an important number of professional academics your statement may be true, and I have heard a former post-doc at ETH Zurich describe their papers as career points (so also a grain of truth at elite institutes). But for most of the academics I have known and worked with, publications are taken quite seriously, and institutional knowledge is freely shared. There is an incentive to reduce the content in papers, but it is out of respect for the reader (a paper is not a textbook) and an honest attempt to limit the discussion to the core hypothesis of the work. You have 6 pages to 1) describe the content of 6*100 pages (the 100 other relevant papers on the topic), 2) present your addition to this body of knowledge, 3) discuss the insights your work brings, again referring to the content of 600 pages. and those 600 pages you are summarizing are as information-dense as your work.

Fomite 5y ago
Also hard disagree. Everyone I know takes papers quite seriously, there's a major push to get more information into the literature (publish null effect estimates!), etc. This feels a lot like the graduate students on the Academia StackExchange who are convinced the moment they present their idea it will get stolen, while every faculty member is like "I have a list of my own ideas that I don't have time to work on as long as my arm."