5 ms·
The blogger who spotted problems in Dana-Farber cancer papers
- deleted 3y ago[deleted]
- Wistar 3y agoSee also the 2021 New Yorker piece about Stanford researcher and doctored image spotter Elisabeth Bik. "How a Sharp-Eyed Scientist Became Biology’s Image Detective" https://www.newyorker.com/science/elements/how-a-sharp-eyed-scientist-became-biologys-image-detective https://www.newyorker.com/science/elements/how-a-sharp-eyed-...
- lelandbatey 3y agoFor a few more example images/videos of the the image manipulations being caught, see here: https://www.nytimes.com/interactive/2022/10/29/opinion/science-fraud-image-manipulation-photoshop.html https://www.nytimes.com/interactive/2022/10/29/opinion/scien... Or see her mastodon with photos: https://med-mastodon.com/@ElisabethBik https://med-mastodon.com/@ElisabethBik
- 1970-01-01 3y ago>design some file-organizing system that involves giving research images a sensible name. And then when it comes to checking your paper before you publish it, you need to trace back all the images to the raw data and check them against the metadata. For example, if you have a photo labelled as ‘Day three’, does that correspond to the date the photo was taken or that the experiment happened on? Blockchain, but with human-readable metadata? Sure sounds like it is do-able.
- LordShredda 3y agoNo, a notepad.
- 1970-01-01 3y agoData is taken from other papers, it cannot be centralized.
- TylerE 3y agoSince we're talking about immutable data...sure it can. If you really need to prove provenance of the metadata, surely a checksum is enough?
- stonogo 3y agoYou just photograph the notepad and release it as an artifact alongside the paper, if you don't feel like typing it up.
- bafe 3y agoOr you skip the notepad and use a tablet and an electronic lab notebook. I know of a few researcher doing this successfully, but for this to work at scale senior scientists and group leaders need to force everyone in their labs to do the same
- 3y ago
- riedel 3y agoHow does funding this work work? Is it donation? Are there bug bounty programs for research out there? (I quitely wonder if this would not add a lot of credibility to research institutions again. I just wonder if we as state funded institution would be allowed to pay this)
- sudosysgen 3y agoThe blog does ask for donations at the end of the article, so it is at least partially donation-funded
- beastman82 3y ago> I used to write letters to the editor [at journals], but it’s a very infuriating process. This is a shame
- deleted 3y ago[deleted]
- lightsighter 3y agoIt feels like there should be some kind of tool analogous to valgrind/ASAN/TSAN/UBSAN for code that is just run on all the images of any kind of paper before it is published to check for these kinds of things. Some of them just seem to be honest mistakes like linking the same image in two different places and this would fix that so nobody needs to bother checking for it in the future. It would also have the byproduct of catching all the malicious image doctoring that some people do which is just horrific for science in general.
- opwieurposiu 3y agoSuch tools exist, ex: https://imagetwin.ai/ https://imagetwin.ai/
- sparky_z 3y agoWouldn't that just give the fakers a tool that lets them easily tweak their fakery until it can't be easily caught? The saving grace here is that many of these scammers don't have the sophistication or expertise in the wide possibility space of fraud discovery techniques to know where they're exposing themselves. Every type of check that we make automatable and trivially repeatable by anyone will immediately cease to detect anything except the most lazy or incompetent scammers.
- dudus 3y agoPublications should be responsible for the veracity of the content they publish. Instead they just shift the blame. They are 100% about profits and provide zero advantage, no wonder they are being replaced by free alternatives. But it doesn't solve the problem with fake papers and plagiarism.
- ethanbond 3y ago> Publications should be responsible for the veracity of the content they publish That is ridiculous and betrays a serious misunderstanding of the role of publications. The publications are a forum for discussion. People say incorrect things in discussions all the time, and they can’t be corrected until they’ve been said. Where do you propose they get said? How do you propose the publication “know” what’s true or not? There’s no way to do it, otherwise why would you be publishing what you’re publishing? The whole point is that people don’t already know what you’re telling them. Scientific publishing is super broken in many ways but this is a terrible solution.
- dr_kiszonka 3y agoOne of my greatest professional worries is publishing a paper with an honest data analysis error and being accused of fabrication. For this reason, I spend an "unreasonable" amount of time triple-checking everything (my colleagues are annoyed by that). I wish there was software that would help me spot mistakes by looking at my code and/or write up, but I also recognize that it would be abused by those committing fraud.
- karencarits 3y agoThis is also one of my fears, and my strategy has been openness - I share my analysis/code with colleagues/Github/as supplement. Maybe they can spot errors or reuse some of it, and I feel that there is some "protection" in it
- chaxor 3y agoIt should be a requirement that every paper be completely reproducible, for any software part of the paper. On any OS or systém, it should work. Similar to requiring statistics to be reported a certain way, there should be guarantees that the code should be available and work in all circumstances.
- analog31 3y agoUnfortunately, I don't think this is realistic. Is it provable? Does it require use of an "official" programming language and libraries, that are guaranteed to be maintained? Is Turbo Pascal for MS-DOS still supported? That's what I used for my dissertation, in 1993. It's a far cry beyond what's expected in branches of science that are not mainly about programming.
- jaredsohn 3y agoPerhaps require making a docker container and writing a test script that includes inputs and expected outputs. There could be a website that loads those docker images and shows that the tests pass. Tests could be referenced within the paper. Would still require scrutiny to make sure that code doesn't hard-code the outputs.
- cs702 3y agoWhy call Sholto David a "blogger" when he is properly a scientist? Someone please give him a budget. And also to Elisabeth Bik!
- serial_dev 3y ago> The DFCI (...) had already been investigating some of the papers in question before David’s blogpost was published, says the centre’s research-integrity officer, Barrett Rollins. (...) (Rollins is a co-author of three of the papers that David flagged and is not involved in investigations into them, says DFCI spokesperson Ellen Berlin.) How naive do they think people are?
- donor20 3y agoDoubly so because comments on some had been out for a while. My guess is nothing was actually going to happen prior to the blog post.
- greazy 3y agoOdd to call a molecular biologist a 'blogger' in the title when the reality is they're highly trained experts. Really shows the lengths some academics go to falsify data.
- nvm0n2 3y agoIt's not just Dana-Farber and that Alzheimer's lab. This type of fraud is endemic in biology. Occasionally it gets noticed. Behold, another identical thread on the same site about the scientists of the Breast Cancer Research Foundation from four years ago: https://forbetterscience.com/2020/01/20/the-wizard-men-curing-breast-cancer/ https://forbetterscience.com/2020/01/20/the-wizard-men-curin... Featuring a scientist who was rumbled for image cloning, and then submitted still more Photoshopped images in his corrections (accepted by the journal, of course, because why wouldn't you let a detected fraudster have a second try with higher quality fraud). First comment: "I was at Cornell….you are still missing at least my ex-Boss and collaborators."
- TwoNineFive 3y agoAs another commenter here put it, this is "endemic in biology," unfortunately. I once had a job at a top research lab doing engineering on their HPC clusters. There are a lot of big-ego fraudsters in science. I think the most egregious thing I knew about was that payments, I mean donations, would come in from the nearby Roman Catholic Diocese to certain researchers. A few months later, something like a research paper supporting the idea that genital mutilation (male circumcision) reduces AIDs transmission rates would come out. Things like faking, I mean "correcting" datasets and just making stuff up wasn't uncommon at all. You had to publish or perish.