25 ms·

I tried to report scientific misconduct. How did it go?

• speedgoose 6y ago
  I think there is at least one or two platforms to rate and review articles after they are published. But I can't find them. Does anyone remember their names ?
  • timbargo 6y ago
    Are you thinking of PubPeer? https://pubpeer.com/ https://pubpeer.com/ Which I recommend the browser extension for. It provides a banner at the top of the page anytime the site displays a paper with PubPeer comments on it. https://addons.mozilla.org/en-US/firefox/addon/pubpeer/ https://addons.mozilla.org/en-US/firefox/addon/pubpeer/ Plenty of comments when you visit this thread's site. :)
• Bucephalus355 6y ago
  I once read an article that theorized that as a covert means of pre-war, countries would publish bogus disease, human health, and pathology data along with fake stats on "how poisonous is XYZ things". Wish I could find it. Struck me as creepy.
• darig 6y ago
  In a world of more scientific misconduct than meaningful science, reporting scientific misconduct IS scientific misconduct.
• jojobas 6y ago
  The model of science that worked from Newton to Landau seems to be falling apart in today's scale. There are millions of people whose livelihood depends on publishing, so they will publish anything they'll get away with. The amount of noise is beyond any researcher's ability to pick through. True incremental improvements in all areas are drowned in a steady flow of bad research. Top tier institutions seem to survive in some sort of bubbles.
  • thaumasiotes 6y ago
    > There are millions of people whose livelihood depends on publishing, so they will publish anything they'll get away with. This is also the problem with the iOS App Store and the Google Play store. They are modeled off Linux repositories, and those do not cause major problems. The phone app stores are riddled with problems, because you can charge for apps in those stores. If you're willing to pay people to lie to you, they will. You have to make the tradeoff between higher participation with lots of fraud, and lower participation with not that much fraud. > Top tier institutions seem to survive in some sort of bubbles. Not really; compare Brian Wansink at Cornell. ( https://en.wikipedia.org/wiki/Brian_Wansink https://en.wikipedia.org/wiki/Brian_Wansink ) Research output is reliable where it is actively relied on by engineers -- and not elsewhere. At this point, fraud is the norm and research is the exception in academia overall.
    • unishark 6y ago
      > Research output is reliable where it is actively relied on by engineers -- and not elsewhere. I'd word it this way: if the person producing the output is not responsible for it really working, it almost certainly won't. Even innocently this will be the case with anything complex, people get things backwards, miss a scale factor, etc. Finding that last bug can take more work than the rest of the project combined, much easier to publish what appears to work and move on. Much more so when there's direct career benefits to "hacking" the system over competing honestly. Especially considering the internet is awash with people trying to cheat through every other competition (exam questions, interview questions, etc.)
      • thaumasiotes 6y ago
        > if the person producing the output is not responsible for it really working, it almost certainly won't. Even innocently this will be the case with anything complex Indeed, there's a great example in the article itself, in a totally unrelated area: > I felt these journals generally did their best, and the slowness of the process likely comes from the bureaucracy of the process and the inexperience editors have with that process. In other words, these reasonable journals weren't able to use their retraction process even though they wanted to, because the process never gets used and therefore isn't in a usable state.
• temikus 6y ago
  Dr. Elizabeth Bik is making efforts to detect image fraud in scientific papers and reading her findings has made me quite worried about not only the general accuracy of the data from lesser-known universities, but how difficult it is to retract/correct those through journals. Check out her Twitter if you’re interested in the topic: https://mobile.twitter.com/microbiomdigest https://mobile.twitter.com/microbiomdigest
  • samvher 6y ago
    Not sure why you’re particularly worried about lesser known universities? These issues seem to have more to do with the state of science and the review process than the prestige/fame of any particular university. I’m pretty worried about the well known universities as well, in particular the expectations and pressure to perform can affect people’s work and decision making.
    • temikus 6y ago
      Because that may introduce new biases around smaller universities from India, Russia, China who already have issues getting things published. As a former scientist in a smaller Russian uni publishing was already difficult and it saddens me to see a couple of bad apples ruining it for everyone.
• ogogmad 6y ago
  "Social psychology" has had a serious problem with scientific misconduct and statistical incompetence. Is this related in some way to that subject area?
  • ogogmad 6y ago
    What about the rest of academic psychology for that matter? Is this endemic to that field?
    • walleeee 6y ago
      The field is over-crowded, for one. Two, its participant pool is almost exclusively undergraduate students on university campuses in WEIRD (western, educated, industrialized, rich, and democratic) nations. Three, experimental designs may rely on questionably reliable, possibly retrospective self-report to infer patterns of affect and cognition. Four, experiments are often conducted in highly contrived laboratory environments. (In-situ data collection as people go about their lives is becoming more common with the rise of smartphones and wearables, however.) As in most domains of science, researchers are also pressured to publish regardless of the quality of the work, and replicated findings are usually not considered interesting or valuable enough to publish. This is anecdata, so take it with a grain of salt. (I minored in psychology and worked as a research assistant with a social/cognitive psych lab as an undergrad.) Also, there is a ton of extremely good science in psychology. It's a shame the recent profusion of low-quality work obscures it.
      • lldbg 6y ago
        What is an example of extremely good science in psychology? What's a result we can hold up to scrutiny and believe truthfully with conviction?
        • shatnersbassoon 6y ago
          Cognitive psychology topics such as memory and reading have a strong paradigm, and established results. In general, their effects are much easier to replicate because you require far fewer participants. This is because you can validly make multiple observations of the same participant.
• Prcmaker 6y ago
  I've reported similar misconduct before. Was told the claims were very concerning, and that there was a quite clear problem that would be investigated. A few months later, I was informed the problem had been resolved, though on inspection, nothing had changed. I learned their investigation involved asking the person of interest if everything was legitimate, to which they said yes. Investigation closed. I am truly disappointed at what has become of the industry I used to appreciate so much.
  • rdtwo 6y ago
    I’m surprised it didn’t involve retaliation against you. I know of a few cases at UW where the outcome was retaliation against the reporting party
    • Prcmaker 6y ago
      It hasn't been. My research is now being actively sabotaged.
• thaumasiotes 6y ago
  This is pretty interesting stuff, but one note: > The correction explains away the failures of randomization as an error in translation; the authors now claim that they let participants self-select their condition. This is difficult for me to believe. The original article’s stressed multiple times its use of random assignment and described the design as a "true experiment.” > They also had perfectly equal samples per condition ("n = 1,524 students watched a 'violent' cartoon and n = 1,524 students watched a 'nonviolent' cartoon.") which is exceedingly unlikely to happen without random assignment. This actually cannot happen with random assignment either. The only way you're going to get equal numbers in each bin is if your process is intentionally constrained to do that. If assignment were random, the odds of assigning 1,524 to one bin and 1,524 to the other bin would be C(3048, 1524) / 2^3048, or 1.4%.
  • maximilianroos 6y ago
    Here, "random assignment" means randomly assigning half the participants to each of two bins, I think.
    • davetannenbaum 6y ago
      Right --- it's called block randomization.
    • dxdm 6y ago
      How do participants get assigned to the respective halves?
      • thaumasiotes 6y ago
        The simplest possible algorithm would be: 1. Shuffle the list of participants. 2. Put the front half of the list into one half of the trial, and the back half of the list into the other half. Generalizing this to more than two groups is straightforward. This algorithm is mentioned sidethread, by sterlind, with the (meaningless) modification of splitting the list even-and-odd instead of front-and-back. As I mentioned there, you can only do this if the list of participants is fixed before the beginning of the study, which is not in general the case.
• cosmotic 6y ago
  If journals retract, they lose reputation. Thus they have incentives to ignore the problems, which explains the authors results.
  • SubiculumCode 6y ago
    but mostly likely they'll never publish that author again.
    • ALittleLight 6y ago
      That doesn't seem to be the case in the story in the OP. The problematic author is still publishing at a great pace.
      • nxpnsv 6y ago
        Which is weird... I guess reviewers are lacking in stats education.
        • SubiculumCode 6y ago
          I mean, those are pretty low tier journals but...
      • SubiculumCode 6y ago
        I did mean at the same journal.
• Johnny555 6y ago
  The article mentions Brandolini’s Law: “The amount of energy needed to refute bullshit is an order of magnitude larger than to produce it.” It's the first time I've heard it, but it's a very appropriate observation in today's world where misinformation travels faster and wider than correct information. If you're just making stuff up, it's much faster than looking up sources.
  • SulfurHexaFluri 6y ago
    Its no wonder there is a trend now of just outright rejecting information presented when our "trusted" sources of information are very susceptible to malice and error without any real tools to combat it. My current view is that academic research should not be used as proof of anything and only as the starting point for your own research. And by your own research I mean your own actual tests. The papers can point you in the right direction but their findings should not be taken as fact.
    • obscura 6y ago
      I don't see how this is practical. You could spend a lifetime testing the work of others and still not get through it all, let alone get to working on anything original. Progress is made by building on the work of others.
      • canofbars 6y ago
        The point is not to test everything ever published, its that when you want to do X, you look for papers on X understanding that they are likely flawed but better than starting from scratch.
        • kenjackson 6y ago
          This still don’t make sense. For example, I want to paint my house with a less toxic paint. I can’t trust any academic research. I have to now research what is toxic in paint? Then I have to find ways to measure various chemicals and gases? Etc... This seems like a complete utter waste of time. In real life most life impacting academic research is much more right than wrong. You are far better served assuming so. Unless you want to waste your time going back to basic science and rebuilding all the academic knowledge in most things you wish to do.
• greesil 6y ago
  I really like this work. Her methodology is remarkably simple. Which begs the question, is peer review so broken that simple things like this can't be caught? Should we have a government ministry of academic papers?
  • dnautics 6y ago
    that's kind of what we have already, though responsibility is divided between DARPA, NSF, NIH, DOE, NCI.
• fiftyacorn 6y ago
  I remember doing chemistry at university and lab results quite frequently didn't match the expected results. So the first time you submit the results you report what you've found and try and explain it, and get marked down Lesson learned in future you give them what they want and attach large error bars I changed course after that as part of science should be explaining bad results
• dnautics 6y ago
  In graduate school, in my lab there was a grad student who was kind of an unlikely "professor's pet". He was tall and had surfer's long hair with a bit of a hippie aesthetic. Anyways, he was also really completely clueless about how to do science correctly, but also, I guess, really good about playing politics (there was a time when he asked me to put some bacterial plasmid DNA on my mammalian cells. I told him "it doesn't work that way", but I did it anyways and handed over the cells, and he got the observation he was expecting). On his main project he was teamed up with a super sketchy foreign postdoc that I was convinced would say anything to get high profile papers out. So they did a series of experiments and reported results that screamed "artefact". On one of them, for example, the postdoc got trained to use the electron microscope and they went through thousands and thousands of images to pick out the one that had "just the right morphology" (I am pretty sure they were snapping photos of salt crystals). On another, they reported that their research subject protein was so fast at the process we were studying that everything occurred IN MIXING TIME. That to me, screams "you are not doing your experiments carefully". Meanwhile I was sweating balls working on a very careful preparation of similarly finicky proteins (you agitate them and they do bad things since they're metastable) and finally got it to produce reproducible results. I suggested they adapt my preparation to their protein but they couldn't give a damn, they had already published their paper and had moved on to sexier proteins. But then an intern was put on the project, and she could not reproduce their results, after working on it for six months (she is careful and honest). At the end, I felt so bad for her, I offered to train her on my technique, but she passed. I think she was burned out on the project. I asked if I could get a sample of the protein that she had prepped, and she agreed. I ran the protein through my preparatory technique and observed that there was a contamination that could have seeded the kinetics of their process. Upon isolating an uncontaminated sample, I carefully but briskly rushed the sample over to the machine. Nothing. Curious, I jacked the temperature up to get it going faster. Nothing. I left it in the machine overnight. Nothing. Finally, convinced that I had likely done something wrong, I dropped the sample in a shaker at temperature, came back the next day and recorded amazingly high signal. In short, the observation that it was "super fast" was entirely an artefact. As I, too, was trained on the Electron Microscope, I quickly spotted my sample onto an EM disc, reserved some time and hopped on the 'scope. The first grid sector I looked at, there was literally TEXTBOOK morphology in front of my eyes. I stapled together my results, gave it to the grad student, and told him that the general gist of his paper was probably still correct, but that he should be careful about characterizing his protein as exceptional. I then said it was in his hands to do the right thing. What do you think he did? Nothing, of course. He kept on the talks circuit, still talking about how exceptional his discovery was, and to date there have been no retractions. He even won the NIH grad student of the year award. The epilog is that after a decade of floundering I realized that even though I am pretty good at science, I was no good at playing academic politics and quit the pursuit; I drove for lyft/uber for a bit, and now I'm a backend dev. I am certain that my experiences are not unique. Amazingly the intern returned to our lab, and had her own three-year stint chasing ghosts that turned out to be overoptimistic interpretation of results reported by a postdoc. Oh. What happened to the grad student? He's a professor in the genomics department at UW.
• qqj 6y ago
  Chinese science, lmao.
• raister 6y ago
  What about Editor's ethics or lack thereof? I got accepted in a Chinese-oriented journal (i.e. most of the Editorial Board were Chinese) - I am not just 'saying' this, I'm saying because the OP mentioned "it's a Chinese thing" over results and datasets, whatever, I digress. On the last revision round, the Editor told me that I was lacking some references, which he promptly send me. Turned out that 6 out 6 of his 'recommendations' were papers HE WAS ONE OF THE AUTHORS. Since the paper was not OFFICIALLY accepted, I caved in and cited the guy (3 times), to my UTTER DISMAY. If you don't play the game, other Chinese are playing the game and having the results. I don't mean to insult Chinese people, but this is what is happening...
  • lrem 6y ago
    Oh, this is not a China thing. I've had a paper have a bunch of reviewers suggest a bunch of references each. Every bunch had every paper share at least one author. Every bunch was pairwise disjoint in the author sets. Draw your own conclusions. Edit: just to be clear: I didn't at the time read that as "submission tax". More of, trying to be helpful and using things they personally were familiar with. Most, if not all, of the extra references would make our paper better... If we weren't fighting that damned page limit, that is.
    • jacquesm 6y ago
      More and more evidence that the magazine based publication route is a net negative to science.
      • DarkWiiPlayer 6y ago
        Question is, why don't scientists just put everything on public platforms (read: github) and call it a day? Is it only a matter of funding, or do other factors also play a role in that?
        • Vinnl 6y ago
          Because nobody reads it there and, more importantly, funders don't recognise the work you've done there. The "prestige" (as indicated by the scientific-looking but mostly inaccurate "Impact Factor") of the journal you publish in determines how good they think your work is. I wrote about that a while ago here: https://medium.com/flockademic/the-ridiculous-number-that-can-make-or-break-academic-careers-704e00ae070a https://medium.com/flockademic/the-ridiculous-number-that-ca...
• Ace17 6y ago
  > I was curious to see how the self-correcting mechanisms of science would respond [...] > I was disappointed by the response from Southwest University. Their verdict has protected [a fraudulent researcher] and enabled him to continue publishing suspicious research at great pace. The self-correcting mechanisms of science can only correct knowledge. Those mechanism work mainly by requiring the research works to be checkable by others. Self-correctness emerges by the accumulation of checks on the same topic, all leading to the same conclusion, and by the progressive retractation of bad research ... not by the elimination of "bad researchers". Efficiently "correcting" people, whatever that means, is a different beast. Such a mechanism belongs to an administrative entity who can emit decisions - and, by construction, who can make errors.
  • marcus_holmes 6y ago
    How does bad research get retracted and corrected then? As the author points out, the "data" in these papers is large enough to contaminate meta-analyses for years to come. And if the Bad Scientist continues to produce more of them, then decades to come. The consensus of the entire discipline will be swayed. Self-correcting this will be very difficult, require lots of data, and be unrewarding. It probably won't happen. Politicians consulting The Science on this subject will get erroneous conclusions and make erroneous decisions. The Scientific Method is self-correcting. Academia, not so much.
    • SiempreViernes 6y ago
      The usual way is someone writing a better paper and everyone going "yeah, that's a better reading of the data". Not sure why you say correcting an established consensus would be unrewarding? Sure, for unimportant details getting a correction out isn't much fun, but correcting an important point is basically the career goal of every scientist precisely because it is important and rewarding.
      • marcus_holmes 6y ago
        From the author, who specifically stated that this task of correcting bad science is unrewarding.
• runsWphotons 6y ago
  There have been several discussions about this topic this year and there are lots of interesting anecdotes in the responses. Is anyone aware of a good longer form review of the difficulties obstructing better reproducibility and more honesty in academic publishing? Something attempting to collect the different theories and observations people make in discussions like these?
• KirillPanov 6y ago
  > They said, no, "It's a China thing." This needs to stop being an acceptable answer. Not just in academia, but also in politics, in business, in the naming of viral strains, ...
  • marcus_holmes 6y ago
    It's tricky. There are huge differences in culture that we don't appreciate. What we think of as ethical and honest doesn't match what that culture thinks of as ethical and honest. And there's no reason for them to think that we're right and they're wrong.
    • physicsguy 6y ago
      If you want to publish in western science journals, then you should be held to western standards for science, whether that's convenient or not. Not that we're perfect by any means!
    • sn41 6y ago
      Please do not bring relativism into science. There is no eastern science and western science. I am an Indian, and work very hard to conform to high standards of scruples and ethics. There are asymmetries like paucity of travel opportunities, lab equipment shortages etc. but we struggle just the same to provide good and trustworthy results. Please encourage everyone to do the same.
  • Traster 6y ago
    I don't know if it's true that this is a particularly chinese thing. But I don't think it's impossible that a country with a very different culture has a very different attitude towards fraud and cheating and that manifests. If theere is more scientific fraud in one group of publishers we need to be aware and tackle that.
• cccc4all 6y ago
  Unfortunately, social politics infect group dynamics, even in supposed scientific settings. When people ask about historical scientific issues, like how did historical scientific consensus conclude the sun revolving around the earth. And it took Copernicus to right the wrongs. Simply look at the kind of scientific shenanigans happening now, false results, outright fraud, huge reproducibility issues in scientific studies. And many scientific communities just going along with the shenanigans. Explains many things in science.
• UShouldBWorking 6y ago
  As a PhD student I found some data that made no sense. Basically a compound added to a plate of cells EXACTLY matched the increase in yield, the former student had just multiplied the yield by the same amount of the added compound. I reported it to my advisor but (of course) she swept it under the rug and did nothing. I tried to contact the format student but also nothing. There were a few more similar instances before I became completely disillusioned and left the phd program after 4 years, totally burned out with little to show. To this day I hate that lab and the whole institution. Rotten to the core.
• rurban 6y ago
  You are not fighting science, you are fighting politics. China obviously wants to forbid violent movies and games for minors, so they come up with these fantasy studies. Very similar phenomenons also just happened in the west, where news and politics wanted to suppress critical scientists. News was stronger. You also have to fight corruption all the time. Paid studies are constantly published to support some companies goals, with much better tricks and not so obvious flaws. Best is just to study the background of the authors and only accept independent research.
  • hoseja 6y ago
    Finally someone is talking about the content of Zhang's "studies". Seems motivated to support predetermined policy, probably so the CCP can boast about their decisions being backed by science.
  • morpheuskafka 6y ago
    Makes sense overall, but why would they care about getting it published in an English-language Western journal? If they just want to convince their own people they can publish it in state-owned media, and it's not a democracy so it's not like they really need to convince anyone.
    • rurban 6y ago
      You underestimate chinese people. Not everybody is successfully brainwashed and believes everything what chinese media or science tell them. They are extremely critical internally, but would never admit that publicly. Their privacy standards are much higher than in the west for a good reason. They are really spied upon, in a much grander scale. Eg their social media profiles are never with real names, and are invite-only. Facebook or Google realname policies won't fly there. So independent confirmation from westerners has its value.
• pototo666 6y ago
  Hell. This is disgusting. I used to think of joining academic world, my interests was in Political Science. But I had many doubts over the academic mechanics in China, many bad news spreaded.(Besides I doubt my passion and intelligence). So I choose to join business world, then startup world. Still, reading such news disgutsts me. There should not be such thing as Fake it until you make it in academics.
• fareesh 6y ago
  What if you had a webpage with the paper at the top and a comments section at the bottom where anyone in the world could post a rich-media post of their findings that cast doubt on some findings with regard to the original paper. We currently treat academic research by a metric of "citations", why not have a new metric where the world can be cautioned of potential issues with the result. You could use the bug bounty model and invite crowdfunded contributions to pay out bounties to those who participate in debunkings. This does not need to be accepted as canonical by the scientific community at large.
  • pjc50 6y ago
    > What if you had a webpage with the paper at the top and a comments section at the bottom where anyone in the world could post a rich-media post of their findings that cast doubt on some findings with regard to the original paper. That just enables your field to be destroyed by cranks who themselves have no accountability.
  • ramraj07 6y ago
    That's already a thing - https://pubpeer.com/static/about https://pubpeer.com/static/about It has definitely brought out many demons in papers but it's still not widely prescribed to or refered from in academia.
  • throwaway2245 6y ago
    > ...anyone in the world could post... This would be ultimately filled with trolls, nonexperts, and people with an ulterior motive or grudge: so any such effort would be less trustworthy than the original publication. If you want to find mistakes, you can find one in every paper. The character of the mistakes is important - is it fraud, is it incompetence, does it negate the results? Or is it good science being done by a human? An open website doesn't seem to me likely to be able to draw that out.
    • grumple 6y ago
      > any such effort would be less trustworthy than the original publication. The idea that scrutiny would be less reliable than blind trust is absurd. The question in the OP, for example, could have been in the comments sections of these papers.
• ssn 6y ago
  Downvoted due to the "clickbait style" headline. You can rewrite it when posting to Hacker News.
• physicsguy 6y ago
  There's a group in Asia that worked in the same area I did my PhD in. In particular, there's a guy who published 18 papers during his two years Master's degree. Now, most of these papers were tiny. They effectively were "Run one simulation, get one interesting but tiny result, publish". To me, that's 'salami slicing', and journals should not accept papers that should have been larger studies. But he's carried on with this, has now completed a PhD and has a permanent position at a Japanese University.
• osamagirl69 6y ago
  This reminds me of the story from R. Trebino about trying to get a correction published--in 123 easy steps! (spoiler alert -- the comment didn't end up being published after the 123 steps) [1] https://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Comment-in-123-Easy-Steps.pdf https://frog.gatech.edu/Pubs/How-to-Publish-a-Scientific-Com...
  • syockit 6y ago
    Not really my field, but I got curious and followed the rabbit hole. The article in question would be (in DOI) 10.1364/OL.32.003558, followed by erratum 10.1364/OL.33.002854, and then the comment by Trebino 10.1364/OL.34.002602, which is replied to in 10.1364/OL.34.002603.
• terranstyler 6y ago
  A fried of mine reported scientific misconduct (p-hacking) and, together with a few colleagues, left the research group, due to moral harassment by the head of that group. The university removed all of them from the research group and said they could continue working on the data because it belongs to the university. 3 months later: - investigations of scientific fraud against the people leaving (neglecting authorship because the data could after all not be used and the head wanted a say in the articles, i.e., change them completely). Also some random other allegations that didn't stick. - police investigation of defamation (because they reported the scientific misconduct and some other misleading statements used by the head in sales for a research-related product) - the university now expects them to contact the head of the ex-research group to clarify questions of authorship - the head meanwhile continues as before
• andi999 6y ago
  I do not think that receiving the raw data gives any rights to analyse the data and publish it on the own webpage.
• lanevorockz 6y ago
  The belief that any research is automatically true is so bogus and so abused that industries and lobbyists came to rely on it. It’s sad that then people blindly push as “it’s science”. Main issue is the sheer amount of papers being published and the lack of capacity of the body of experts to read all of it. I guess it’s the professionalisation of research. People publish papers to improve their rankings and not because it’s relevant.
  • tgv 6y ago
    There actually is more than enough capacity to peer review (). It's just that nobody wants to do it. It costs time and money. Not compensated by the publisher, of course. () edit: that's raw body count. I wouldn't know how many people could actually spot the errors mentioned in the OP.
    • dubbel 6y ago
      From the article: > For example, one paper reported mean task scores of 8.98ms and 6.01ms for males and females, respectively, but a grand mean task score of 23ms. A 9th grader should be able to find that inconsistency, if you give them the table and tell the to find the number that is wrong. (the other stuff is harder to detect, and I fully understand that you can't request and re-process the raw data for every paper you peer review. Some of these numbers....)
  • mjburgess 6y ago
    This comment really clarified an issue: This is a slow-moving disaster for scientific credibility, and therefore for national safety and security. There's going to be a point within two decades where "reproducibility crisis" is not a localised phenomenon, and "expert" misconduct is paraded out by the papers. Totally destroying our societies ability to govern itself based on expert information. The early stages are already here (anti-climate, anti-vax, etc.).
    • rdtwo 6y ago
      I think the outcome is more likely to be that papers from the US are just assumed to be highly suspect in quality sort of how papers from China and India are now.
• throw14082020 6y ago
  I can't imagine how tiring it is to do the investigation the author performed. He had to subscribe to literally garbage papers (from Qian Zhang) to investigate them. He had to do the policing. Because there is no quality control in research. Just citation counts and people-you-know.
• known 6y ago
  I think there should be a bounty for revealing the truth
  • motohagiography 6y ago
    Was just thinking that if sci-hub were more like a git repo or wiki with editors and peer comments, it would add a great deal of transparency, and pretty much destroy this anti-pattern. FOSS still has a lot of vulnerabilities, but it has also caught a lot of vulnerabilities and more people know what to look for. Perhaps this is why there is so much resistance to sci-hub, because there are so many compromised editors and academics who risk exposure?
    • jononor 6y ago
      sci-hub has not tried to move anything in that direction. I would guess that opposition to sci-hub is mainly because it could reduce barriers to entry, and de-value traditional journals. For established researchers the high level of gatekeeping caused by journals is in their favor, since they know how to play the game, and many other want-to-be-researchers do not.
• yason 6y ago
  Scientists are people. Taking the path of least resistance and skipping over details is the nature of some people, and thus of some scientists as well. It's not unbeknownst in other professions. Anecdotally, I haven't had my car serviced in a shop for years because mechanics even with supposedly good reputation would fail to do even trivial maintenance jobs properly, invariably requiring myself to partially redo it myself to ensure the quality of the installation which defeats the point of having a job done by a paid professional in the first place. It's the boring details and following the process carefully that these people skip, in hope for getting results quicker and moving on sooner. I just take it that some people are like that, and the lower barrier of entry to sciences than before means there are more corner-cutters in academia as well. The question is where in the process of the academic journey from student, master, grad-student to doctor are people qualified for the work as for their psychology and personality, not only knowledge and intelligence? The danger signs with this sort of personality are undoubtedly visible in early stages. The market pressure to maintain one's reputation doesn't seem to work in the academia as illustrated by the article. Thus, it would be better to start explicitly culling this attitude off the field before these people get to establish themselves despite their bad workmanship.
• hmwhy 6y ago
  > Science is supposed to be self-correcting. I admire the author's belief, not least because I used to be like that, but I personally think that couldn't be further form the truth for contemporary scientific research, and it's no better in evidence-base physical sciences. I personally know many people who used to be, or still are, in scientific research who wouldn't hesitate to agree with me that scientific research is mostly just a job for most people that's not too different to any other job that earns you a salary. I always ended up not posting my comment in related topics, but since this is getting so much traction, I might as well try not to appear to be bitter about my own experience and give my anecdote another go. If nothing else, at least this will become (albeit insignificant) a piece of history that stays on the Internet. I long time ago I received a prestigious postdoctoral fellowship to work with someone very well-known in the field on studying the mechanisms of a then relatively new type of chemical reactions. I spent a couple of months to meticulously prepare everything I needed for the study, and when finally I got everything ready, I began by reproducing the first break-through that was produced in the group that started it all -- and it didn't work. Since I was new to that particular type of chemistry at that time, I spent the next few months trying to reproduce the reaction while getting others, both within and without the group, to check my work. Nobody seemed to be able to figure out what I did wrong but one particularly thing stood out at that time: nobody I have spoken to actually tried to reproduce the results of the "first" reaction, ever, which was super strange to me. I had also spoken with my advisor then, who basically became well-known because of that first reaction, and he couldn't offer any solutions and the conversation always ended up being something completely unrelated to the irreproducible results. I spent most of that time blaming myself and suffering from some form of imposter syndrome, too, simply because I have the tendency to do that. Up till that point I had been following the procedure published in a journal article, but I thought I would dig up the first author's PhD thesis to check what I had done wrong. I started by casually scrolling through the experimental section and an C-13 NMR spectrum of the catalyst that I was working with caught my eye immediately because of some very unnatural signal truncation that I thought was only possible with data manipulation, and sent the data to a few of my friends who are experts in NMR and they also confirmed that those "artifacts" are most certainly unnatural. I immediately e-mailed my advisor about it, but he never responded -- and that was the only e-mail from me (which obviously required a response) that he never responded to. I did find a few manipulated spectra in the same PhD thesis, but none of that really helped because I still couldn't reproduce the results that nobody has ever mentioned anything wrong about. Then one night, when I was drinking with the group, someone working on a different floor I don't usually talk to about my work asked me how things were going; after I told him my problems he immediately said that he'd met someone from industry at a conference complaining to him that the reaction doesn't work. He also said that a few people who came before me also tried to reproduce that reaction but none of them got it to work. At that point I was just angry because *I thought "science is supposed to be self-correcting"* and there is no way that this stuff was in the literature for 10 years and nobody ever said anything about it. In fact, it's impossible for my advisor to not know that something is wrong with it because he is very well connected to both academia and industry, and so many people in the 10 years before I arrived must have worked on it. During the time I was unable to get anything to work, I was constantly assigned work that seem somewhat related to what I do but wouldn't help me with my career in any way. In the end I had a hunch on what was really happened and determined that the procedures in the original paper and the PhD thesis that first reported the reaction were all out by a factor of 10. I was already on anti-depressants at that point and was drunk every night but was working 10+ hours a day, which was well-known in the group. When I had finally gotten the reaction to "work" (and had explained to people I trust and had them double check my work) and brought it to my advisor, he said "that's great"; I don't remember too well what else he'd said in between because none of it was neither an apology nor a solution, but he said at the end that maybe I should have deferred my fellowship because of my depression (which, frankly, wasn't affecting my ability to work). This is not an isolated case, and not the only type of academic misconduct. The thing that upsets us the most is that at the end of the day, it's not about how good and meticulous you are: for most of us it's mostly about how well you are at gaming the system. The way we fund scientific research is mostly broken, the way we disseminate scientific research is mostly broken, the way we assess potentially great scientists and appoint them is also mostly broken. It's only natural that, for most people, the experience is nothing but shit. Edit: typo.
• realradicalwash 6y ago
  We are dealing with 5+ papers that are fraudulent and another 5+ newer papers that are most likely fraudulent, too. That is, there have been 20, 25+ reviewers looking at those papers. Their job was to carefully read them and double check the numbers. All of them gave those papers a pass. I am at a loss here. The authors' behaviour is outrageous, but this story is also about a broken reviewing process, partly due to wrong incentives.
  • rapht 6y ago
    ^ This exactly! "Peer-reviewed" by whom?...
    • chalst 6y ago
      Journals typically list their referees per issue, but they do not say who reviewed which paper.
  • remus 6y ago
    > Their job was to carefully read them and double check the numbers. That's the theory. The reality is that there is no in-depth review. You're lucky if a reviewer actually reads the paper all the way through, let alone checks the numbers and applies a level of critical thought to the methodology, analysis and conclusions.
  • roel_v 6y ago
    "Peer review" is not "have someone else re-do the experiment". That's just not feasible, especially since reviews are done without pay. It's not realistic to expect people to spend more than a few hours reviewing a paper. That amount of time is barely enough to check for overall conceptual issues and maybe flag some really glaring deficiencies. (And then conclude with 'accepted with minor revisions', those 'revisions' preferably being 'add these three citations to my paper, that'll push your paper into 'acceptable' territory'.)
    • driverdan 6y ago
      But there were glaring deficiencies in the stats. Anyone reviewing the papers should have caught them.
• cornel_io 6y ago
  Chinese researcher is full of shit and fabricates results, news at 11. For the haters, this is not racism but nationalism, China super incentivizes bullshit research at a high level these days, and it's gotten bad enough that we're starting to distrust any "work" that comes out of it. I don't know what the solution is, other than to subject Chinese submissions to more stringent and specifically non-Chinese review. That's absolutely nationalist, and arguably racist, but it's also smart.
  • 12ian34 6y ago
    >China super incentivizes bullshit research at a high level these days Just as you'd expect from a "Chinese researcher", you're going to have to qualify this statement for your point to hold any weight.
    • remram 6y ago
      In particular the incentives are set up the same way in the US, and reproducibility is a problem in many fields.
  • Blikkentrekker 6y ago
    Why would it be racist? I have noticed in English-language discourse that often, shall-we-call-it, “non-white countries” are “races” but “white countries” are “nations”. Also, Christianity and Judaïsm are religions, but Islām is a race. Explain me that.
    • captain_price7 6y ago
      > Islām is a race. Muslim here, that sounds absurd. In fact one of my biggest annoyances is when people view all Muslims around the world as single entity. Every stupid trait of every Muslim majority culture gets blamed on entire Muslim world.
      • Blikkentrekker 6y ago
        I find it absurd too, but it is often how it is phrased in English-language discourse, even in Dutch discourse the anti-Islām branch phrases it as such: the language suggests that one can identify a “Muslim” by some kind of physical phænotype of his body. The difference is that in general in Dutch discourse, such statements are considered racist or betraying such a mentality, and frequently protested, but, in English-language literature, even the “left” that claims to champion the causes of all these “races” and “religions” still very often writes in a way that betrays a mentality that some religions and countries are “races” and others are not.
• cies 6y ago
  Many stories like this and people wonder why the general public is less "believing" in science lately. I think sadly the general public is right not to believe as strongly in science as it maybe once did, the results though are dramatic in "baby bathwater" kind of proportions. I think science should fix itself. Just publishing paper should not be the metric to reward. A retraction should seriously reward the flaw finder (like sometimes with exploits), and really harm the flaw author/publisher: both scientist and journal.
  • mam2 6y ago
    > Many stories like this and people wonder why the general public is less "believing" in science lately. I. Eh im not sure bad studies is the cause. Scientists, especially doctors, wanting to use their authority i some debates while 2 of them can be saying completely opposed things maybe, however, contribute...
    • speeder 6y ago
      Problem is more serious than that. What is happening is that the bad studies are being used for policymaking. Examples: the "nutrition pyramid" that encouraged carbohydrates and blamed health issues on animal-based food, was later found out to be based on research that was blatantly corrupt, with researchers getting bribes from food industry to manipulate or hide results (a case of hiding results: one researcher that found out that vegetable oil causes decrease of blood cholesterol, also found WHY it happened, but omitted that part from his paper... the reason is that cholesterol is needed for cell maintenance, and consuming only vegetable oils cause a deficit from it, the body pulls cholesterol from the blood to repair itself, and even that might not be enough, with some people suffering damage). Or a lot of pharma circlejerking that turns into law or regulations. Or the paper mentioned in the article, that was about video-games and aggression, with many countries passing laws regulating video game consumption based and such papers. Or the original reason Cannabis was banned (long story short: part of the reason is that they wanted to ban hemp fibers, that was being an obstacle to some newly invented synthetic fibers, some of the government people involved, had stocks of Dupont and other fiber companies, and "accidentally" banned hemp fibers while "trying" to ban the drug, based on manipulated and fraudulent science). Or more seriously: the papers that recommended "Austerity" and basically destroyed the livehoods of millions of people, later were found out to have math errors that changed the conclusion completely. And the list goes and goes on.
• LockAndLol 6y ago
  How are we, the laymen, supposed to trust published, peer-reviewed papers? They seem to be just a bunch of words now with little meaning. Before, I used to consider scientific papers... well, scientific and would try to base an informed opinion on the abstracts and conclusions. After reading blog entries like these and actually reading some papers (especially soft science papers which are easier to understand), even as a layman, some glaring mistakes can be spotted. Popular scientists like Dawkins and that black astronomer are happy to point out the glaring problems in other areas of life, but it's looking more and more like the scientific field doesn't have its shit together either. On a scale of bullshit to trustworthy, where stuff like Breitbart and ThePinkNews live in swamps of bullshit, scientific publications and papers seem to barely reach "believable". One always has to question "who payed for this research", "who reviewed it", "which country is this from", "what reasons could there have been to do this research", "are these results too good to be true", "who would benefits from these results", etc. It seems like one really cannot trust anybody or anything and has to constantly keep their wits about themselves. Will we ever be able to clean up our act? What can we do?
  • psim1 6y ago
    For one, consider the institutions and nations that the research is from. It's too bad that journals are not more discriminating, but we as consumers of articles can be.
  • searine 6y ago
    >How are we, the laymen, supposed to trust published, peer-reviewed papers? You are not supposed to. The laymen really isn't the intended audience of academic publications. The literature is always in flux and inherently unreliable as discoveries are claimed and over decades, proven true or false. Taking a snapshot of the literature at any one time is to accept that a proportion of the claimed truth will be false. Unfortunately the laymen doesn't get this, and they believe that published=true. As a layman, you should be looking to sources of information that have been vetted for truth, like textbooks. Textbooks are made to distill the most reliable information from the literature by a team of experts. One paper isn't truth, a dozen independent papers, all pointing to the same thing is. That is what we call the "scientific consensus".
• dandare 6y ago
  Two of the mentioned studies seem to be related to the question whether violent games and movies influence our behaviour. Assuming Qian Zhang's work is fraudulent, what was his agenda? That violent games are OK, or the opposite?
  • JW_00000 6y ago
    His agenda could also just be to advance his career, and make as much money with as little work as possible.
    • dandare 6y ago
      Of course, but in doing so he would choose a certain course, I doubt his studies would contradict each other.
• Method5440 6y ago
  My best friend went to UCSD for his PhD in biology. He was brilliant and had a nearly unique depth of insight into knotty problems and an incredible drive to progress the field. Unfortunately he was also a bit of an idealist with little real political sense. In grad school he selected a difficult problem in the cancer space and worked on it in the lab for 6 years. His advisor thought he was on track for a Nature paper. Around the end of year 6 a very famous scientist who was on his larger committee decided he wanted the research for himself (apparently). He had one of his floater grad students (in their last years of grad school without any research of their own to publish) literally steal his data from his desk. They eventually published their ‘stolen’ paper in Nature themselves, before my friend could have finished writing it up by himself. My friend found he was unable to compete with the reputation of this scientist and was repeatedly told to just move on - even his own advisor suggested that there was nothing to do about it and complaining to the university ethics committee would only hurt his career. He tried anyway, entirely unsuccessfully. My friend was not able to move on. He left grad school with an exit Masters. He spent a few years in his parents house lost, then in institution really really lost. He eventually got himself together and built up the courage to try again (roughly 10 years later). He got into a good bioinformatics program on the opposite coast of the country. Eventually the same exact thing started happening to him again. Things got bad. I left work early one day to go cheer him up and long story short I found his body in the bathtub of his apartment. He was just not ready to go through it all again. I still think about him every single day more than a year out from his funeral. I find myself unable to understand why some humans treat each other the way that they do or how they are able to get away with it. I’ve asked around and it seems like this is a fairly common occurrence, especially in circles around the original ‘famous’ scientist. These people basically killed my friend, don’t know it and probably wouldn’t care. They likely rationalize their behavior as the cost of doing science. The system is absolutely disgustingly broken and much of published and celebrated science is, in one way or another, a lie. We need to stop making scientists into rockstars, especially those who somehow publish more papers in a year than physically possible. Each one of these untouchable individuals is followed by an unseen trail of ruined careers and ruined lives. The field would not have suffered. My friend’s work would still have been published. The difference is that it wouldn’t have added to the myth of exceptionalism of this particular scientist - and maybe the floater grad student would not have gotten her PhD... but in the end my friend didn’t get his PhD either and now he isn’t here any more. Scientific prestige is not a limited resource and should not be subject to the tragedy of the commons. My young daughter asks about him a lot and I have no idea what to say.
• avsteele 6y ago
  Good article. For more on this subject try: https://fantasticanachronism.com/2020/08/11/how-many-undetected-frauds-in-science/ https://fantasticanachronism.com/2020/08/11/how-many-undetec... https://www.goodreads.com/book/show/52199285-science-fictions https://www.goodreads.com/book/show/52199285-science-fiction...
• Aeolun 6y ago
  > ... a swift two-month process ... O, M, G. I’m really happy I’m not in a research field. In my experience, China will do a lot to look good in research. Apparently up to and including falsifying data. My experience is mostly in gaming university ranking mechanisms though (though arguably publishing lots of bogus articles helps there too), the increase in “research” output from China has been nothing short of amazing.
• toxik 6y ago
  Unrelated to the subject, but titles like these are immensely unhelpful to the reader.
• noisy_boy 6y ago
  It should be a requirement to be able to provide the anonymized raw data for publication in peer reviewed journals and multiple retractions should have an effect on the credibility of the authors and their chances of being able to publish such research. Without the penalty, there is no chance of reduction in such fraud.
• leephillips 6y ago
  An inspector general in the U.S. Navy informed me that the Navy does not have a rule against plagiarism. Take this as career advice if you want.
• archi42 6y ago
  I have friends (now M.Sc.s in CS) who did support people writing psychology stuff (PhD & Masters thesis mostly, as well as papers) at my alma mater. The key take away was: Many of the students at that dep were either bad in statistics, or outright abused them to "prove" the desired effect via manipulation of the data or intentionally using the wrong method. Both faculty and students would not listen to experts telling them that their statistical method was weak. The most amusing/saddening point was when one of them was fired because he said he couldn't solve the Halting Problem for them (& I mean that literally, it was a crucial part of an experiment). Now a family member works as a data scientist, supporting students with statistical analysis (for thesis/papers). Same thing there, a lot of students seek her help because they're bad with statistics (well, at least they don't fabricate data...), some want their thesis written by her (she drops that kind of job) and some expect her to hammer the data until it fits their hypothesis (which seems to be the most annoying/exhausting, because she has to convince them their method is wrong and the result pointless). Overall take away: I'm sorry, but for some fields simply have to classify a PhD as worthless unless I've read the work myself :(
• salamanderman 6y ago
  In a graduate product design class I took, our semester project was to design and build and make cost estimates for development of an IOT product. "Internet of things" wasn't a phrase yet, but that's what you'd call it today. We had to incorporate these ultra low power sensor/processor things the professor had his name on and he was a big promoter of. At the beginning of the semester his grad assistant presented her invention from a previous year, which she won awards for and had presented and written about and was part of her PhD work. It was a home health monitoring device and she showed plots of a month of data from sampling herself (pointing out she was the only woman on her project). It was very inspiring and I was very impressed by her. Jump to mid-semester, I randomly have a team of MBA students and me; the three of them were going to do all of the writing and I just had to do all of the engineering by myself (yay). I'm battling in the lab for hours trying to get the damn thing to read a voltage. I keep putting time of the GA's calendar for help, and she keeps blowing me off or passing me in the hall and saying "ummm maybe try this?" or she'd give me another device to see if the last was defective. In principle, she should have been able to point out whatever I was doing wrong in 15minutes or less, but weeks of this avoidance went on. Eventually, after asking everyone in the department where she was and letting people know I was trying to meet with her and just sitting at her desk at our appointed time for over an hour waiting, she caught me in a hall, conspicuously looked both ways to see that nobody was around, and said "look, the things don't work. They've never worked. My device never worked. I made up the plots based on what they theoretically should have been if the product worked. I'm grading the projects. Just focus about the write-up of the business plan." So my work was done. And at the end of the semester, nobody's product worked, but most people acted like theirs did. Ours obviously didn't work, but we made up some shit about it being a mock-up because we didn't have the budget for some of the components. ... A professional photographer took shots of my team that were used in promotional material for the school.
• liminal 6y ago
  How do we prevent fraud? Is there a way to change the incentives? Statistical analysis of results to detect bogus numbers? Only cite research that has been independently replicated and apply a "provisional" label to the first results? Then only cite the duplicating lab and not the first one, so the incentive is there for doing the grunt work?
  • leovailati 6y ago
    I generally like the idea of rewarding replication studies. Replication/validation could be a required process that runs concurrently to peer review. The researchers who run the experiments to replicate results could rewarded by being added as contributors to the original paper, so they also get the citations. And like you suggest, the paper would then be marked as "validated" or something similar. I wonder if any journals out there are already doing something like this. There is, of course, the danger of collusion among original authors and validators. Hopefully the fear of having your results rebuked would prevent people from trying to publish bullshit in the first place. Another problem is logistics. Research labs have their own ideas they want to push forward, so spending time and resources proving or (even worse) disproving some else's idea doesn't sound that great. Also, even if it gives you citations, it probably wouldn't help you with your thesis. Anyways, it's a tough problem to solve.
• johncessna 6y ago
  Oh My Science! I don't understand where this ideal that Science is infallible and beyond corruption, influence, and politics comes from.
• ricksunny 6y ago
  I’m wondering how many folks are aware that a similar case of research misconduct is actually affecting the covid origin investigation? Essentially a lot of scientists in 2020/2021 cite the same two research papers on impounded pangolins to support that covid-19’s virus, SARS-CoV-2, had a close cousin(s) infecting pangolins. However, this analysis from an MIT Broad Institute genomics researcher, Alina Chan PhD, implicates research misconduct on the part of those two articles’ authors. https://twitter.com/ayjchan/status/1320344055230963712?s=21 https://twitter.com/ayjchan/status/1320344055230963712?s=21 Turns out the authors of the pangolin papers can’t provide the complete pangolin-infecting coronavirus sample genome (i.e. they can’t provide ‘the source code’ if you will), and they profess not having coordinated with each other or even knowing each other even though authors from both papers published a paper (notably also a pangolin cov genome oriented) together just a couple months before the outbreak came out. Mainstream virologists like Angela Rasmussen PhD now call the pangolin cov genomes ‘a mess’, https://twitter.com/angie_rasmussen/status/1349841489384284160?s=21 https://twitter.com/angie_rasmussen/status/13498414893842841... and yet these papers continue to get cited to help prop up the natural origin line. U.S. Right to Know published the email traffic between the Nature Medicine & PLOS Pathogens editors and the two sets of authors of the research papers in question: https://twitter.com/ayjchan/status/1354455267656785925?s=21 https://twitter.com/ayjchan/status/1354455267656785925?s=21 . . . and after all that those authors still come up short. In other words, there’s ‘weirdness’ around the provenance of those pangolin cov datasets, and the lack of formal retractions from Nature and PLOS Pathogens (despite those journals’ editors’ posted Q&A with the authors) means that the natural origin line continues to gain unearned steam (beyond being reasonably treated as simply the pandemic origin’s null hypothesis).
• drummer 6y ago
  It is difficult to trust scienctific writings these days especially in medicine. Just listen to what Karry Mullis (inventor of PCR) explains here: https://archive.org/details/nobelprizewinnerchallengesthemythsaboutaids https://archive.org/details/nobelprizewinnerchallengesthemyt...
• throwawayfrauds 6y ago
  My previous account on HN was banned by dang after I called out a researcher for scientific misconduct. The researcher contacted dang and complained, and he banned me. So I guess calling out scientific misconduct didn’t go so well here on HN either.
• barmstrong 6y ago
  This is part of what we created https://www.researchhub.com https://www.researchhub.com to improve on. We should be getting weighted ratings (peer reviews) in real time on all research.
• signaru 6y ago
  Academic institutions produce a lot of graduates far faster than replacing the people who run them. If they're not keeping up in skills, which is highly likely due to teaching load and other non technical time consuming tasks, some of them sadly resort to misconduct to hold on to their jobs by faking competence. A more subtle form of misconduct is not with the technical results, but with fraudulent activities. A close friend of mine got his professor kicked out due to fraud. Furthermore, the professor was using university research grants to cover expenses of his poorly performing startup via bogus reimbursement claims and was also using the university researchers' labor and results to be products (that unsurprisingly didn't sell and didn't benefit the researchers). All this while the professor is at the comfort of his home even before work-from-home was a thing (because staying at home made transportation tax deduction profitable). The professor has a habit of plagiarizing his researcher's manuscripts so he can attend miscellaneous conferences (field-trips). In some cases, dropping the researcher from the co-authors of the new plagiarized work. There is a shared excel file of massively self citing scientists shared a couple years back, and that professor is in the list. As far as the world is concerned, this professor is a prolific scientist with many (last-authored) publications and prestigious talks (that should have been presented by the researchers) who happened to recently "transfer" to a different university and stopped publishing.
• tonfreed 6y ago
  Psychology is undergoing a severe replication crisis at the moment, it doesn't surprise me that there's systemic fraud that the academy refuses to address. Until we drag the ivory tower down a few pegs it's not going to get any better.