15 ms·
Peer-reviewed papers are getting increasingly boring
- kome 6y agoI just got my first paper accepted, after 8 months and 2 round of r&r (please clap): the reviewers and editors try very very hard to kill any joy, ambition and big idea. papers should be written with a lot of accademic mannerism. The goal should be to make the paper clear, but it's not true. It just a style to decide if you are part of the clan or not.
- burlesona 6y agoCongrats! What is your research about? What drew you to this field, and why do you think it’s important?
- kome 6y agoThank you! your comment and interest are quite heartwarming! My research is about household debt: I try to understand why some countries have more private indebtedness than others. Growing up during the American crisis of 2008, and the subsequent global crisis, made me realize that debt can be dangerous, and I wanted to understand what pushes people to borrow money. For a website, I wrote a short introduction to my research: https://progressive.international/blueprint/3596cc12-0128-4f81-9718-bd13356de766-comelli-the-debt-narrative-is-dead-wrong/en https://progressive.international/blueprint/3596cc12-0128-4f... - soon the full research will be published in a peer reviewed journal. :)
- ArnoVW 6y agoInteresting intro. It's even translated into French I noticed. So northern Europe is borrowing three times more (per capita) than the US? That goes against everything I know. One question: did you separate out consumptive credit from mortgages? The amount is so high that it seems to me that it includes mortgages. Because in that case, is it not the cases that it is simply because norther Europe is wealthy and has more expensive houses and more home ownership. Those mortgages are backed by houses, and are a sign of good financial practices, allowing gradual increase of wealth. Where consumptive credit is backed by nothing and is generally a sign of bad financial practices.
- skybrian 6y agoWell, it's unsecured debt, but a credit card is informally backed by your future income. You might say that all personal debt is backed primarily by your future income. Even mortgages are backed primarily by future income and secondarily by the house, and this is why the bank wants to know how much money you make. Whether or not credit card debt results in future wealth depends on what you buy.
- ArnoVW 6y agoI'm not an expert in finance, so don't hold it against me if I'm wrong, but does "backed by" not rather mean that the bank can have your house if you can't make payments? This in opposition to a credit card debt where they have no come-back. Other than the courts and collections agencies. I was under the impression that this was the reason that consumptive loans generally are capped far far lower than mortgages. And have triple their interest rates. While it is true that I can put my university on my credit card, it is very expensive and very rare. I would wager that the majority of these loans are for cars. Which, I agree can create wealth, but it would have been 'better practise' to buy a cheaper car, and switch when that anticipated wealth has been created. Moreover, those cars explain only part of the loans. The rest is 'bigger tv' or the likes. And those do not even create wealth.
- skybrian 6y agoIt's ambiguous. I'm pointing out some grey areas. When speaking loosely, we can talk about what a loan is "backed by" meaning how people expect to pay it back. Or you could talk about what property secures the loan, legally. Regarding investment, there are startups that got off the ground by running up credit card debt. It's not what I would do, but it's more like investment than consumption. Or you could pay for your car to be repaired using a credit card, and if you need the car to get to work, well...
- kome 6y agoThank you for taking the time to read! Yes, in the full paper I separated consumer credit from mortgages. In the intro I linked, indeed they are aggregated. Long story short: in northern and continental Europe consumer credit is almost non-existent. Very tiny numbers. Notable exception: the UK. But more mortgages don't necessarily mean more home ownership eh. "Those mortgages are backed by houses, and are a sign of good financial practices, allowing gradual increase of wealth. Where consumptive credit is backed by nothing and is generally a sign of bad financial practices." That's almost my conclusion as well, but I put more emphasis on what social policies encourage you to do, more than on good or bad practices. In the US they really encourage going for consumer credit, also their bankruptcy legislation is more lenient and encouraging a fresh start; while in Europe we are much more draconian with debt.
- guicho271828 6y agoCongraturations on paper acceptance, but joy is very optional in research. Joy is emotional and it clobbers logical insight. Peer review is an attempt to guarantee a steady firm progress. It is a matter of taste --- do you want progress or just want to be comfortable? If you want joy, better to be a SF writer... Still, strong results with good execution are allowed to enjoy some freedom.
- adwn 6y ago> [...] but joy is very optional in research. Joy is emotional and it clobbers logical insight. I'm pretty sure OP meant "joy" as in "motivation/enjoyment of one's work", not as in "hedonistic pleasure".
- darkhorse13 6y agoWow, you basically proved the parent comment right by taking the joy out of yours.
- guicho271828 6y agoActually, there is a good opportunity to be full-on joy mode after acceptance --- Presentation at the conference that follows. You can say as many sketchy things as you want (within caveat) and crave attention. So it is just this basic point --- There is no need to do this in a paper.
- tgv 6y agoTBH, big ideas should not be in a paper, unless you can really prove it. I've read too many papers with grand new theories that hinged on an experiment testing the tiniest of tiny facets. All of these have been forgotten now, even by their creators. Once you have enough experience, know the field inside out, and have gotten a bit of a name, you can write a book or edit a special edition of a journal in which you may set out your ideas.
- fancyfredbot 6y agoThe style of academic papers is intended to encourage objectivity I think. All the same I would agree it can indeed obfuscate at times and a more informal style could be both more enjoyable and easier to read.
- rleigh 6y agoAnd this was how papers were written historically. If you go back and read papers from well known researchers from the early 20th century, they can be a delight to read. We have gone down a road of increased formalism and a removal of any character in the name of "objectivity" and "dispassionate impartiality". This is, of course, utter bunk. Researchers are often not objective or impartial about their work, but it's all carefully phrased to be as dry as possible. If you read some older papers, you have the author interjecting with their thoughts and opinions directly in the first person, and it can make for much more compelling reading. So long as the data presented is objective and accurate, I can't say I have much problem with it. Today we use the "discussion" section for this, but it's just not the same. Put it this way, I've fallen asleep reading more papers than I care to admit. But some of those older papers were so enjoyable and interesting to read I'd devour them and go looking for more. Objectivity is of course important, but I think modern publishing has lost something which is also important: the excitement and interest of the authors in their own work.
- wott 6y agoI occasionally read Geography articles from the first half of the XXth century, they are a joy to read, until at least the 50s or 60s. It reads like a book: the vocabulary is very accessible (the few words/concepts which do not belong to the common language are introduced and defined) and while the writing flows pleasantly like a fiction or documentary, it conveys much useful information. The recent ones? They're horrible piles of pretentious jargon, jargon which oscillates between hyper-technical and utter-bullshit. They read like those technical marketing fluff pieces. Philosophico-technocratic verbiage all along; when you're done with it (assuming you didn't give up in despair or anger), you haven't learned anything, basically. Anyway, they usually don't bother really describing concrete things any more, they rather talk about fancy social constructs. The synergy of the Promethean dynamic of the actants of transformative innovation of mountainity. Right... (I made up the example, but from real pieces from my last 2 attempts. I unfortunately can't remember or find the previous one, it was 10 times worse, I wouldn't even have needed to make up this example, I could just have copy-pasted its abstract.) The difference is like between reading a page-turner, and trying to read the latest ISO standard about development methodology in safety-related domains (perfect for falling asleep before the second page).
- awillen 6y agoBut on the other hand, with a severe replication crisis in science (https://www.vox.com/future-perfect/21504366/science-replication-crisis-peer-review-statistics https://www.vox.com/future-perfect/21504366/science-replicat...), isn't it a good thing that we have a bunch of boring papers that are close to stuff that's already happened? It's not people replicating others' findings exactly, but it at least gives points of comparison that can help to weed out some of the problematic work out there.
- lisper 6y agoBoring is often a good thing, not just in science. There's a reason that "May you live in interesting times" is considered a curse. Think about the year we just had. Whatever else you might say about it, it wasn't boring. [Clarification] Individual local experiences may have been boring at times, but the year as a whole, from a historical point of view, was not. And the things that made it non-boring also made it suck big fat honking weenies. IMHO of course, but I'm pretty sure most people on earth would agree with that assessment.
- awillen 6y agoI disagree - boring is critical in science. Good science means detailed record keeping and precise experimentation. It's good if the results are exciting, but the process of doing good science is typically quite boring when it's done right. Edit: This comment makes no sense, because I misread the comment I was responding to as saying "just not in science" not "not just in science"
- bobthepanda 6y agoThis doesn’t sound like disagreement.
- awillen 6y agoI was really confused for a bit there, and I finally realized the comment I was responding to said "not just in science," not "just not in science" as I had originally read it. Whoops.
- burlesona 6y agoThis is a reasonable and well-explained argument, which I’ve heard many times before: the incentives are currently producing a large quantity of mediocre research, rather than groundbreaking science. So let me ask the research community on HN something I see discussed less often. What alternative incentives exist today, or could be created, that would push more scientists to try higher risk / higher reward research?
- austincheney 6y agoPercentage ownership of resultant intellectual property. A good example is the Minneola, a wonderful citrus fruit. The plant patent is owned by three parties: the University of Minnesota and the two inventors.
- varjag 6y agoLargely inapplicable beyond applied sciences.
- whatshisface 6y agoImagine how much the Maxwell Trust would own today... The biggest problem with a viral IP system is that there would be nothing left for applied scientists.
- varjag 6y agoAnd science would likely devolve into patent trolling system.
- logifail 6y ago> Percentage ownership of resultant intellectual property. Pfft. I've been through this, my ex-supervisor attempted to file a patent which included a bunch of figures and illustrations I drew single-handedly while writing up my PhD and which were lifted from my thesis without my consent. My PhD was funded by a government research council. My ex-supervisor was filing a patent on behalf of a private company he set up to run in parallel out of his university laboratory. There are some people who just don't care about rules and figure they can just ask for forgiveness later. I'm still cross about this, over 20 years later.
- bonoboTP 6y agoStepping back a bit, I feel the argument is analogous to how music is becoming more mainstream and generic, how casual gamers "corrupt" the art form of gaming, how clickbait and stupit stuff is overshadowing lovingly curated independent websites, eternal September etc. One answer to all these propositions is that the valuable stuff is still there, you just have an additional flood of mediocre boring predictable stuff. You can still find enough indie music and games and movies to spend all your free time on. Similarly, if you know where to look and whose papers to watch out for, you can read more interesting stuff. The existence of more bullshit makes the filtering somewhat more effort, but it's bearable. Most papers are never deeply read anyway, outside of the reviewers. Even citations don't mean someone really read the paper in depth. So, most medíocre, boring papers get ignored. It's an illusion to expect that every researcher can pump out multiple really interesting non-boring papers every year. So we pretend. We write, we cite, we present, overiflate, overpromise etc. It must look like there is steady hard noble work and toil. It's like a ritual. The scientists are writing all these papers so tax payers can sleep well that their money is paid for hard work.
- timkam 6y agoI largely agree, but we do not write all these papers "so tax payers can sleep". We write these papers because the administration incentivizes us to get the highest profile publications with the lowest effort possible. In relatively creative fields (i.e., much of computer science), this can even mean to make friends in the community and co-author papers with them without being the one who does any of the hard work. (In other sciences, it means being the one who pulls in the money, as far as I understand.) A related problem I see is that we treat all papers the same: position (opinion) papers, reviews & surveys, and "hard" results (relevant proofs, strong empirical results, software artifacts that are deployed at scale in practice etc.). As the blog post somewhat suggests, the administration should primarily ask us to summarize these results when assessing whether we are worthy of funding and faculty positions; sure, it takes more effort, but I think it is possible.
- nextos 6y agoExactly, the big issue is that paper counts have become the dominant metric. And like with all structures dominated by a single metric, it's easy to game the system. Most professors I have worked with in experimental sciences are completely disconnected from actual research. They just optimize for hiring hordes of people to churn out tons of mediocre papers, which they don't care about. I came from theoretical CS & math, so this was really shocking. It's really depressing and it needs to change. Academia is now in a phase similar to that of an innovative startup which has been filled in with many middle managers, and has turned into a corporate monstrosity.
- reilly3000 6y agoThe people who fund research look too hard for ROI. The funds that get invested into research buy hours of time of people doing research and maybe gear. They cannot buy discoveries. Until this mismatch is addressed broadly, we will continue to have both a reproducibility crisis and ‘boring’ papers.
- bsf_ 6y agoI believe this is close to the truth. Moving one step further upstream, the emphasis on high ROI stems from an overall lack of availability of funding. Since there isn't enough money to go around, it becomes necessary to choose some more selective metric to pare down the pool of grant applicants.
- virtuallynathan 6y agoDo we just have too many people doing “science”? Clearly people are getting funding to crank out nonsense, boring papers, studies that can’t be replicated, etc.
- bsf_ 6y agoOnly if you believe that as a society we have solved all of the interesting problems. Since this obviously is not true, I would instead focus on the way we evaluate science to improve it.
- virtuallynathan 6y agoI’m mostly suggesting the money isn’t well distributed, and I’m pretty sure that’s due to the misaligned incentives. If we incentivized outcomes we care about, I suspect more money would come naturally from the results.
- garden_hermit 6y agoThere is a lot of caution in research funding. Some research is considered "safe", whereas others risky. IN a properly balanced portfolio, perhaps 80% of funding should go to safe work and the remaining 20% to more risky, but potentially high-reward studies. However this isn't how funding agencies usually operate. At least in the context of the U.S. government, funding agencies are terrified of being seen as "wasting" funding, because that is a sure way to have funding cut by congress. The "Golden Fleece" awards, a congressman's attack on perceived frivolous spending, have routinely been targeted at specific research funded by the NSF and have helped make the agency afraid of being attacked by congress.
- tracyhenry 6y agoBy database folks: We are Drowning in a Sea of Least Publishable Units (LPUs) https://researchsetup.github.io/files/lpu.pdf https://researchsetup.github.io/files/lpu.pdf
- coliveira 6y agoThe same can be said about software development: almost the totality of software developed is boring, unnecessary, and sometimes wrong. That said, we still know that developing software is an important activity.
- jabl 6y agoOne big issue is that grants are short time (meaning researchers spend a stupid fraction of their time writing grant applications), and incredibly competitive. Trying a slightly more ambitious project and failing means the end of your research career. So researchers do the logical thing which is the kind of boring incremental research the article complains about. Combine this with the focus on bibliometrics in evaluating research output, and we have the mess we have today.
- s0rce 6y agoI read a good study that suggested that so much time and effort (and hence money) is wasted on writing proposals for non-funded grants we should just assign a sizable fraction of the money by lottery instead, without needing lengthy proposals. Everyone knows you use the money for other stuff not specific to the grant regardless. The opportunity cost of writing all the rejected proposals would be saved and could be spent on actual science, even if some of it isn't top notch that otherwise would have been funded (and thats assuming that the grant proposal system actually selects the stuff mostly likely to be ground breaking).
- rpedela 6y agoYeah I agree, I would like to see some percentage be lottery based. I think you should still need to write a proposal, but if the proposal isn't selected then it goes in the lottery pool. There are two reasons to still write the proposal: helps organize the researcher's thoughts and shows the researcher is serious.
- jabl 6y agoYou'd still want some screening for your proposal to be admitted into the lottery pool to weed out the crackpots, but yes.
- NotPavlovsDog 6y agoI am currently pursuing a second master's degree (in management), some of my personal experience in academia: 1) Publish or perish is a real thing. Citation ratings and measuring affects many of the academics I have encountered. This contributes to boring papers. 2) Mentioning the replication crisis in social science gets a lot of the social science academics observably upset and defensive. They have to take care of 1)! Compare this to the exemplary stance in CS, as demonstrated by researchers criticizing Google for not backing magic AI claims. [ https://www.nature.com/articles/s41586-020-2766-y https://www.nature.com/articles/s41586-020-2766-y ] This vigorous scientific stance is almost impossible to imagine in social science outside of Critical Management Studies. 3) I have enjoyed my time spent with CS academics a lot more, luckily have some interaction even now - the ones I have spoken to seem to lack the self-criticism blind spots the social scientists exhibited. The CS group seems to have a lot more fun with their research. It was also interesting to observe that for term papers, many CS professors wanted to see a report with working software and a well-thought-out story of what challenges one encountered and how they were overcome. Not one paper was returned because of improper styling nor citing. They appeared to enjoy the battle stories of trouble-shooting and finding solutions to assignment challenges. They even read the sources I linked to, commenting on what they thought was a good find. From that experience (and others), CS sure feels like more of a meritocracy. In opposition, some social science professors were obsessed with proper APA or Harvard styling and citing "the right thinkers". Circular citation is a real thing. Perhaps since they can't prove anything, establishing credibility via cliques, mutual citation and signaling may be their academic survival strategy. It does feel like management and CS are orthogonal, and this has motivated me to direct my research towards a hard realism approach to facilitating communication between developers and value driven management. I.e. I have observed that even when a manager or leader wants to hire developers, social science damaged managers and HR will actively sabotage these efforts because they are still, fundamentally, driven by Taylorist, control fetish concepts (which they refuse to accept). The SS victims seem to wish to push their personal struggle on developers. "I suffered through school, you must show a diploma as well". Some even demand transcripts! When I consulted a company that was complaining about how hard it was to recruit developers, I looked at their application process. They demanded college transcripts even before the review of applicants began. I hope you are laughing with me at the entitled absurdity of this. Oh you want a personal letter as well? Bitter HR person, do you fail to understand that you are competing in your recruitment against multiple actors actively scraping through code repositories and blogs to find and contact devs? Or does accepting that a developer is incomparably more competitive on the job market than you and has multiple offers of employment at any time just hurt too much? One of my favorite quotes from a researcher (well cited) about managing developers includes a rant on how creative, industry competitive (can get another job) and financially stable developers are "hard to manage". This may be how many incompetent managers feel, deeply. If I can facilitate getting these emotionally insecure individuals out of recruitment and management, assisting developers and managers that want to produce value to find each other, it would be in opposition to many well peer-reviewed publications. Boring publications.
- pelasaco 6y agoI have the impression that yes the papers are getting boring, and peer-review is one of problems, however i have it as side effect. The main problem, IMO, is Research as profession, where the researcher is just interested in finish his/her paper, to get it published, to be able to get a next grant and keep researching about a topic that he/she is not really passionate about. The comparison with Einstein is therefore unfair. For big part of the researchers, science is just a job.
- bjornsing 6y agoYes. And when “it’s just a job” people dominate it sets the culture, and makes Einstein’s life very difficult / improbable.
- colincooke 6y agoI think peer-review is an answer, perhaps not a great one, to how can we trust specialist science. In a world where the renaissance man role is no longer feasible, to move science forward we must narrow our focus on particular sub-fields and problems within that field. One issue of this is that it becomes much harder for the general science community to verify your results, which is where peer review attempts to help, by forcing you to get other experts in your sub-field to review your work before it can be stamped as trusted. Does it work? Kind of. I've personally seen papers in reputable journals that while not fradulant, are pretty misleading. At the same time however I'm yet to see a workable alternative that fixes the trust issue. The question of boringness I think is pretty field dependent. In the ML community I've often seen papers almost be rejected by the peer review system because they're NOT exciting enough, despite them being pretty influential (a great example is AdamW [1]). Honestly my assesment of peer review is that not enough trainees (read: grad students/post docs) are doing reviewing. Trainees are often better acquainted with the details of methods, but also have a more open mind to accept a finding that could go a little against the grain. Additionally, they're often faster and do a better job since they have more time on their hands then senior researchers. There have been some efforts to fix this, but so far mostly isolated to specific fields. [1] https://openreview.net/forum?id=Bkg6RiCqY7 https://openreview.net/forum?id=Bkg6RiCqY7
- jpmattia 6y ago> The peer-reviewed research papers allows you to “measure” productivity. How many papers in top-tier venues did research X produce? And that is why it grew so strong. If I were king, my rule would be to look at the number of citations per paper and years of citation, rather than total publication count. I think that would solve a number of issues, although I'm sure that it would eventually be gamed as well. I should mention though: It turns out that people have been bitching about this for decades. For example: I was at Bell Labs in the 90s, and there was regular lunchtime discussion about the Least Publishable Increment (LPI). Everyone had some example about how the LPI had decreased to near zero, and then a few wags went on to show how the LPI had in fact gone negative in certain subfields, not least because reviewers were overloaded and couldn't keep up. Hopefully, some of this will be self-correcting: Publications are no longer the money-making activity they used to be, so resources will begin to dry up. Eventually, pubs are going to start rejecting valid papers for being too incremental.
- colincooke 6y agoMost people do actually judge by citations rather than raw pubs. It's an open secret that you can shove out papers (acceptance to any venue, regardless of the quality of the work is probabilistic) with enough effort. Getting cites is harder (other than self-cites which is a seperate issue). When I look up a researcher first thing I look at is their number of citations and H-10 index, then I look through their top papers (the ones that have been cited a lot). As far as I know hiring committees also care about these things. Of course it isn't the only thing that matters, or the most important, but its much more useful than raw number of papers published.
- mattkrause 6y agoEven those are garbage measures though. They can be gamed (albeit with a bit more effort), they vary a ton between fields and subfields, and citation rates often reflect the "prestige" of the lab/authors and the trendiness of the topic, rather than the intrinsic "merit" of a paper. I don't think there's any good automatic proxy for research quality.
- paulpauper 6y agoHow are there so many papers if the papers are so long and complicated these days and acceptance rates so low? Econ papers for example have tons of data and stat analysis and are often 50+ pages and have multiple authors because there is so much data to crunch and so much stats involved. The era of the 5-10 page paper written by a single person that make an interesting observation or novel insight, is over.
- zwieback 6y agoThe good papers are still out there but it's hard to slog through the mediocre or downright worthless stuff if you're not an expert in the field. That's a real problem, I think, even venturing into an adjacent field takes a lot of reading and sorting through the chaff unless you have someone to guide you. And number of citations is not a great metric to figure out what's worth reading. By its nature, science and R&D is very incremental though. I have some sympathy for the PhD student wanting to get something published (or the prof wanting to get the PhD student to publish) and 5 or even 10 years is short for truly groundbreaking results.
- dekhn 6y agoNearly every "exciting" paper I've encountered in my career (spanning multiple areas of biology and computer science) has turned out to be much less exciting when subjected to critical analysis by a team of grad students (journal club). I've found that many boring papers stand up better.
- amirkdv 6y agoI can't help but but think a universal basic income, at least in societies that can afford it, would dramatically change the entrenched misalignment that causes this. The signal/noise issue is real but as others have noted not the most concerning. The main structural problem IMO is that all the structures that support scientific research (academic evaluation, university admin, granting agencies, publishing industry, etc.) are all re-enforcing untenable illusions. A few benefit immensely from this, but there are many who have the incentive to change, yet they are cornered by their economic circumstances to play along.
- pasttense01 6y agoOne thing to keep in mind is that some of the papers which reviewers consider very marginal turn out to be breakthroughs. For example Jim Allison, who later won the Nobel Prize: "Allison was hoping to be published by one of the leading peer-reviewed research journals. But nobody at Cell or Nature or any of the A-list, peer-reviewed journals was willing to publish the findings of this junior academic from Smithville, Texas. “Finally, I ended up publishing the results in a new journal called The Journal of Immunology.” It wasn’t Science or the New England Journal of Medicine, but it was in print, and in the world. “At the end of the paper, I said, ‘This might be the cell antigen receptor, and here are the reasons why I think that it is the T-cell antigen receptor,’ and I just listed it out, all the reasons.” It was a bold announcement regarding the biggest topic in immunology. “And nobody noticed it,” Allison says. “Except in one lab.”" https://www.wired.com/story/meet-jim-allison-the-texan-who-just-won-a-nobel-cancer-breakthrough/ https://www.wired.com/story/meet-jim-allison-the-texan-who-j... So you really need to publish the marginal stuff to make sure the breakthrough stuff gets published as well.
- inglor_cz 6y agoKatalin Karikó, the Hungarian scientist that pioneered the mRNA research, was academically demoted at UPenn because the university considered her research as "impractical" and "waste of time" [0]. That was in 1995. In 2021, she is a hot candidate for the Nobel Prize for the very same work, which led to mRNA vaccines. [0] https://www.wired.co.uk/article/mrna-coronavirus-vaccine-pfizer-biontech?fbclid=IwAR0uTyPvEyhlia7tvlH3XFx8kVrCyx0A8IIy8ALVSrUJ8m7jV4b8E7mf-Ec https://www.wired.co.uk/article/mrna-coronavirus-vaccine-pfi...
- mattkrause 6y agoAnd in a similar vein... Douglas Prasher did a lot of the foundational work on GFP (2010 Nobel Prize) was driving a courtesy shuttle at a Toyota dealership in 2008 because he ran out of funding. Barry Marshall and Robin Warren had a really tough time convincing people that H. pylori caused stomachs ulcers (but won the 2005 Nobel Prize). Number theory was famously nothing more than an intellectual curiosity, but makes stuff like this website possible. Microbial opsins were also a weird intellectual backwater before becoming vital to neuroscience. Paul Lauterbur famously quipped that "you could write the entire history of science in the last 50 years in terms of papers rejected by Science or Nature." We are really bad at predicting the future impact of projects and people, and it's important to recognize that as a counterweight to schemes that try to select for "excellence." Things that seem laughable at the time can also be the basis for future breakthroughs, so take stupid things like Rand Pauls' ranting about silly-sounding projects with a mountain of salt.
- Areading314 6y agoIrritating to see this Atlantic "study" being accepted as valid and written about. This happened with the "bullshit jobs" meme where it stirred up enough controversy that it became an accepted fact for many people, even though the original research was highly questionable and didn't actually prove much of anything
- darkhorse13 6y agoI wouldn't call this validation. I think it's being discussed positively because it articulates what many of us feel to be true: peer-review is a broken mess. Same thing about bullshit jobs.
- Areading314 6y agoIf you have a better system, you should suggest it. Peer review (and any process) may be imperfect. But I trust something written in a peer reviewed journal more than I trust pretty much anything else I read these days.
- samch93 6y agoDoug Altman actually wrote a paper about this issue already in 1994 [1]. A great quote: „We need less research, better research, and research done for the right reasons“ [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2539276/pdf/bmj00425-0005.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2539276/pdf/bmj...
- bjornsing 6y agoFeynman made some interesting observations in his 1973 Cargo Cult Science speech...
- ivoras 6y agoa) 90% of everything is junk b) that's what you get when you start optimising for paper acceptance
- frongpik 6y agoI think it's a real thing and has the same nature as the force that keeps a water droplet together: it's the surface tension that prevents others from sticking out and supports them to be like others.
- xakahnx 6y agoI find the same failed incentive scheme in industry. When a company gets big enough (tens of thousands of employees), taking risks to achieve something new and interesting doesn't pay off. It's more reliable to just be present and focus on writing about yourself during performance review time.
- Dig1t 6y agoSturgeon's law: 90% of everything is shit. More everything means more shit.
- 2038AD 6y agoSurprised to see no mention of Thomas Kuhn or The Structure of Scientific Revolutions in the blogpost or in the discussion here. Though briefly looking at the linked papers, he's cited by Thurner et al. and directly discussed by Bhattacharya and Packalen. His ideas on incremental science seem pretty appropriate.
- antpls 6y agoImagine if research was like github, but deduplicated : a giant graph database of unique facts, hypothesis, assumptions, proofs and experiment results, where anyone can contribute, from fixing a typo to publishing experiment dataset. You would reference to previous knowledge without having to think about citing authors, because it would be automatically generated from the graph. Then we would have hundreds of "knowledge verifiers", like we have bots scanning github, and ultimately bots deriving new knowledge and contributing to the graph. The whole world would be building a shared database of knowledge, without any duplicated effort. Most of the individual facts in this database would probably be boring (like the trivial facts in logic or mathematic), but once in a while, an interesting and important fact would be found.
- refulgentis 6y agoSo, Wikipedia, except with some sort of syntactical structure that allowed for formal evaluation and analysis of arguments...we could call it a "compiler" for "compiling" sets of rules, or "libraries"
- bongoman37 6y agoThis is an interesting idea and would work for something like Math or maybe theoretical physics, but once you get to messier fields like Biology it would be mired in hopeless confusion. Every experiment has so many varied parameters and restrictions due to things like ethics committees that standardizing it is nigh impossible. This is not even going down to fields like psychology.