2 ms·
To be clear, I'm not thinking specifically about hyped-up research that doesn't pan out. Low quality research in general is what I'm thinking about. And I see t
by _hxy4 7y ago
To be clear, I'm not thinking specifically about hyped-up research that doesn't pan out. Low quality research in general is what I'm thinking about. And I see the vast majority of academics as well-intentioned and not manipulative. (And that includes my advisor, who, despite the disagreements we have as described below, has good intentions.)
Academics are incentivized to publish papers in recognized journals and bring in external funding. I'll focus specifically on publishing here for simplicity. I could write about funding if desired.
The source of many problems I have with the current system is that I want to do high quality research, but the current system doesn't make that easy. The most important research I've done so far has been to make a large data compilation and use this to test different models and hypotheses in my subfield. But my advisor never liked this, and mostly discouraged me from doing this research. He would have preferred instead that I stick to the more established path of skimming the literature for a problem some review article called open, doing a naive experiment about that, and then publishing the experiment. (Yes, I am being facetious, but unfortunately this characterization is accurate in my experience.) My experience is that review articles are far from comprehensive and frequently contain misinformation. It's important to actually assemble a large amount of data to see where the real gaps are, not just what a review article called a gap. This sort of data compilation is very rare in my experience.
It really was clearly about getting publications as quickly as possible. My advisor repeatedly pointed out to me what others recently published in my field and exclusively what he pointed out to me I'd call naive. His point was "This is good enough to get a publication" and my point is "So what? Just because other people do a bad job doesn't mean that I should." To him, other people are getting publications when he's getting nothing! He didn't seem concerned that I thought most of these papers weren't making a contribution. That's not to say that he doesn't care at all about quality, but in practice he's more concerned about getting publications than quality. (He might not see the two as in conflict.)
In the end, I think I am making several large contributions, even if the "gestation" time was longer than people would like. To get tenure, you need to publish often and regularly, but a lot of valuable research takes longer than people would like to get results. The data compilation took a long time. I started this data compilation in roughly Jan. 2017. I had a conference presentation on it in Nov. 2017. I published a conference paper based in part on it in May 2018, and it wasn't until December 2019 that I submitted two journal papers using the data. (I prefer to take my time to perfect an article rather than rush it.) This sort of delay isn't acceptable in the current system, regardless of the value of the work at the end. In terms of total publications, I think I'm actually coming out ahead of most PhD students in my department, but by now I'm seen as unproductive by my advisor so that doesn't matter.
At this point, I'm waiting for one of my papers to be accepted because my advisor won't allow me to graduate until I have a paper accepted at a peer-reviewed journal. My original plan was to publish a few years after my PhD. I've been told multiple times by professors that the basic purpose of a PhD is to learn how to publish in peer reviewed journals, and they'll let you graduate once you've demonstrated that you can do that. And this does actually seem to be the crux in my case. Doesn't seem to matter that nowhere in my department's rules does it say that peer-reviewed publication is required.
It would have been much easier to just pump out a naive paper every year. I would have been done my PhD years ago if that were the case, and science wouldn't have advanced much if at all.
Also, don't mistake my advisor's position as "the incentives say to publish papers so that's what we should do". No, his position is something along the lines of "publishing papers is part of the point of a PhD". He's not thinking about incentives. He genuinely believes that publishing tons of papers is a good thing. The incentives selected him; he is not responding to the incentives as far as I can tell.
- nl 7y agoI don't have a PhD, but I did run a large R&D program which funded a bunch (30+) of PhD students, and I worked closely with them and their supervisors. I think you should consider listening more to your supervisor. I know nothing about your field, but I've seen a lot of PhDs go off the rails because they chased after something they thought was a great idea only to find it was a problem no one really cared about. Young PhD students are really smart, but I've certainly seen that experience does seem to count in choosing good questions. The incremental "find an open problem mentioned elsewhere and work on that for a while" isn't a bad way to start a PhD career. You learn a bunch about the field doing that. At this point, I'm waiting for one of my papers to be accepted because my advisor won't allow me to graduate until I have a paper accepted at a peer-reviewed journal. My original plan was to publish a few years after my PhD. I've been told multiple times by professors that the basic purpose of a PhD is to learn how to publish in peer reviewed journals, and they'll let you graduate once you've demonstrated that you can do that. And this does actually seem to be the crux in my case. Doesn't seem to matter that nowhere in my department's rules does it say that peer-reviewed publication is required. This seems an odd thing to complain about. Surely a PhD is all about peer review, and should be so. Peer review means people who know about the topic have reviewed it - how else should a PhD be done?
- _hxy4 7y ago> I think you should consider listening more to your supervisor. I know nothing about your field, but I've seen a lot of PhDs go off the rails because they chased after something they thought was a great idea only to find it was a problem no one really cared about. Unfortunately only so much can be communicated in an online comment. I tried going into detail, but I'll leave it at this: I do listen to my advisor, but I disagree with him about how science is best done. > The incremental "find an open problem mentioned elsewhere and work on that for a while" isn't a bad way to start a PhD career. You learn a bunch about the field doing that. Reading your paragraph makes me think my point wasn't communicated clearly. I'm not arguing against working on open problems. I'm saying to make sure that the problem is actually open, and don't start doing expensive things like experiments without having a good idea of what you're doing. Given the existence of a large amount of archival literature, your great idea is likely to have been done already. There is value in reproducing previous experiments, but people should do new research too. > This seems an odd thing to complain about. Surely a PhD is all about peer review, and should be so. Peer review means people who know about the topic have reviewed it - how else should a PhD be done? For what it's worth I don't believe peer review as currently practiced in journals provides much value. The reviews tend to be superficial at best, and are provided far too late in the research process to have a strong influence on it. In my experience the reviews mostly help to make the paper more clear, and not because the reviewers give good suggestions on writing themselves, rather, because the reviewers seem confused about things I thought were written clearly. However, my advisor is definitely a strong believer in peer review as currently practiced. I mis-characterized him before. For him it's less "publishing in journals is the right thing to do" and more "publishing in journals provides a strong check on the work". In my view, the most important thing someone can do to improve their research is to be deeply critical of it. Unfortunately few people are critical of their own work. Getting feedback from others is valuable, but I believe that the best way to do so at present is to actively solicit it yourself, which is what I've been doing. Peer review in a journal should be the last step, not the only step aside from one's supervisor looking over the work. Also, I removed a paragraph about how I strongly dislike the current academic publishers. That's a major factor here as well. My field does not have good open access options at present. I'd really prefer to wait a few years as I think the landscape could change dramatically.