5 ms·
Medical journal says the case reports it has published for 25 years are fiction
- october8140 8mo agoI think research should be assumed fiction until it’s peer reviewed.
- contubernio 8mo agoThere is not good evidence that peer review improves quality and there is perhaps some to the contrary (many predatory journals are peer reviewed). The arxiv (unreviewed) is among the most reliable sources available.
- ranger_danger 8mo agoWhat do you suggest instead? Certainly not giving up I hope.
- observationist 8mo agoYeah, it's almost like science is better when the scientific method is applied to everything, instead of delegating validation to some third party based on credentials or authority or social status.
- deleted 8mo ago[deleted]
- moi2388 8mo agoIndependently replicated. Reviewed says pretty much nothing.
- kergonath 8mo agoPeer review is a sniff test. It cannot guarantee that the results are correct and the conclusions are right. It is just designed to limit some kinds of errors. Replication is important.
- roywiggins 8mo agoCase studies can't be replicated. They aren't experiments.
- em-bee 8mo agoyou can find multiple cases that are comparable. one case study is an anecdote. multiple studies for the same kind of case...
- ambicapter 8mo agoTough to replicate an isolated case study?
- moi2388 8mo agoEspecially if they are made up.
- Rallen89 8mo agoI think it's a bit different considering the goal was a teaching tool of well recognised conditions >all or almost all were cases of very well recognized conditions [...] where a single case report would not generate any interest or ever be cited.
- readthenotes1 8mo agoThat is an ironic proviso given that the article clearly states "The peer-reviewed articles don’t state anywhere the cases described are fictional." Peer review by peers who are trained by non-replicable science is not helpful...
- newzino 8mo ago[flagged]
- deleted 8mo ago[deleted]
- SiempreViernes 8mo agoDoesn't sound like these works were "full" articles, but rather something more like short review articles.
- programmertote 8mo agoSpeaking this as a spouse of a medical doctor -- case reports are sometimes a good way to increase the bullet point count in your CV if you are a medical resident. A lot of residents do that just for the sake of beefing up their CVs (to apply for fellowship for example).
- copperx 8mo agoI don't see anything wrong with that by itself; with the amount of patients doctors see there should be one once in a while that is worth reporting. Or are such cases so rare that the doctor is incentivized to lie?
- sxg 8mo agoI think you may have missed the original commenter's point. Residents (and medical students) are highly incentivized to publish unrealistic numbers of papers and case reports. One case report doesn't cut it—you need literally dozens of publications to match into some of the most competitive residency and fellowship programs. The NRMP (match organizer) publishes a document every 2 years that summarizes all of these stats. The 2024 version is in the link below, and page 12 supports what I'm saying. https://www.nrmp.org/wp-content/uploads/2024/08/Charting_Outcomes_MD_Seniors_2024-2.pdf https://www.nrmp.org/wp-content/uploads/2024/08/Charting_Out...
- digitalPhonix 8mo agoThis is another example of Goodhart's law in action, right? Weirdly Pediatrics (chart 7) skews the other way (less publications tended to get into residency programs)? Are those doctors/administrators/programs somehow seeing through the nonsense?
- husarcik 8mo agoI wonder if it's because pediatrics is not competitive unless applying to a top program.
- 8mo ago
- deleted 8mo ago[deleted]
- sourcegrift 8mo agoIn the era of GitHub etc, if you're not giving out every single data point of your research, it should be assumed it's fake.
- avs733 8mo agoout of context that makes sense...but in the context of a case report how do you implement that? The patients have privacy rights and the authors/doctors have a responsibliity to protect them. That doesn't justify this but it does force a conversation about what 'every single data point' means. Does it mean the patient's real name and social security number? their complete medical chart? Case reports are descriptive not determinative and should be treated as such by other scholars. They are 'I saw this' not 'this is generalizably true'. They can (and often are) replicated or countered but they are not per se research as you are thinking about it. Whether it is fictitious or not, other scholars should be cautious in citing them as proof/evidence in papers that fit into the 'research' mold.
- nradov 8mo agoFrom a legal perspective, journal article authors can implement this by following the official HHS guidance for de-identification. This applies to any use of protected health information (PHI), not just case reports. https://www.hhs.gov/hipaa/for-professionals/special-topics/de-identification/index.html https://www.hhs.gov/hipaa/for-professionals/special-topics/d... The IRB for a particular organization can impose additional restrictions.
- fsh 8mo agoThe article is about case reports, not about empirical studies. Putting a fake case report on GitHub wouldn't make it any less fake.
- qwertox 8mo ago> Putting a fake case report on GitHub wouldn't make it any less fake. Much easier to review for whomever wants to review it.
- helsinkiandrew 8mo ago> The articles usually start with a case description followed by “learning points” that include statistics, clinical observations and data from CPSP. I can see the reason where fictional cases could be used here as teaching aid - based on real cases/ilnesses but simplified to make the learning points succinctly, but surely if the cases are being cited elsewhere someone should have raised the issue earlier?
- SiempreViernes 8mo agoSince it was for teaching I expect the case studies were always showing typical features of real cases, so there's nothing in the case vignette itself to give it away unless the author picks a funny name or something like that. Rather it would be the entire form of these short highlight articles that would make you keep searching for a proper citation, unless you're lazy or pressed for time.
- ultropolis 8mo agoWouldn't citing actual cases be a HIPAA violation? I can see why they would invent example cases, based on real ones, especially if they are fairly pedestrian cases. I mean. Except if your pedestrian example does not reflect reality, then that is bad.
- reenorap 8mo agoHIPAA is American, not Canadian.
- fc417fc802 8mo agoIt's a privacy violation to reveal information that identifies the patient. It is not a violation (and is extremely common) to recount details without noting names, places, or even dates. Unless you already have access to a database of records you won't be able to track it down. It's even common during talks to display diagnostic images that have had any identifying marks redacted.
- learingsci 8mo ago“Pics or it didn’t happen,” goes a long way in my book.
- MarkusQ 8mo agoYou may want to update that, given recent advances in generative AI. No idea what you should update to, mind you, but the old era of photographic evidence is on its last jpgs.
- krisoft 8mo agoWhat a mess. > One author of a case report was surprised to learn of the correction — because the case described in her article is true. So they managed to mess up even the correction of their giant mess. > correcting the correction "would be difficult." I bet. That's why they should have got it right in the first place. I would be absolutely ballistic if they would be libelling my work like that.
- SiempreViernes 8mo agoYeah, they seem to have been quite sloppy with these vignettes. Thought note that in the situation of the mislabeled real case, the formal solution is could be a retraction of the entire highlight article since it is against the (poorly implemented) policy to have a real case study. Don't know how patient consent for being used in a case study works, did this author get a perpetual license, did they just copy something from another article they wrote, or from an article someone else wrote?
- smelendez 8mo agoYou can see the full article here: https://www.cpsp.cps.ca/uploads/publications/pxy155-Teething.pdf https://www.cpsp.cps.ca/uploads/publications/pxy155-Teething... It looks like it has a short intro paragraph that talks about a specific case with no identifying details (beyond "a previously healthy 4-month-old boy"), citing this report by other doctors: https://pubmed.ncbi.nlm.nih.gov/27503268/ https://pubmed.ncbi.nlm.nih.gov/27503268/ followed by further discussions of physician reports and survey data. The correction is explicitly listed as applying to that article (https://academic.oup.com/pch/article-abstract/24/2/132/5164289?redirectedFrom=fulltext&login=false https://academic.oup.com/pch/article-abstract/24/2/132/51642...), which itself seems false since that article doesn't seem to include a fictional vignette.
- andrewflnr 8mo agoIt looks like they labelled all of them fiction based on a single instance of one of the authors fabricating their case, a gross overcorrection. I wonder if they flinched at the prospect of actually assessing the validity of all of them and decided it was safer to just disclaim them.
- deleted 8mo ago[deleted]
- kittikitti 8mo agohttps://onlinelibrary.wiley.com/doi/10.1111/jpc.14206 https://onlinelibrary.wiley.com/doi/10.1111/jpc.14206 Maybe we should revisit the routine practice of infant male genital mutilation?
- SiempreViernes 8mo agoI think this is mainly a case of the common "didn't notice when crucial literature for own published content was retracted, get caught with pants down when the replication police come knocking". Obviously the poor labelling is bad, but 9 bad citations per year isn't the end of science and better labelling wouldn't discourage all the lazy authors who chose to cite these highlight articles, it'll just shift whos is to blame. The real problem is hosting a review article about research that was retracted, and it sounds like they aren't moving very quickly on taking that piece down.
- insane_dreamer 8mo agoI don't mind the fact that the case reports were fictional -- actual cases can be problematic in terms of privacy as it may be easy to ascertain the patient's identity from the details -- but not putting a notice that it was fictional (or altered from a real case for privacy), for teaching purposes, is pretty bad.
- snapetom 8mo agoOriginal HN discussion about the case: https://news.ycombinator.com/item?id=46789205 https://news.ycombinator.com/item?id=46789205
- skyberrys 8mo agoThank you, this really adds the missing context to this update about fictional case studies. The original read was compelling and also alarming.
- TomMasz 8mo agoThis is fine, though somewhat belated. But it does nothing to deal with the public's growing distrust of science in general, and medical science in particular.
- jfengel 8mo agoThe "growing distrust" is due to a concerted disinformation campaign which is independent of the facts. There was indeed much negative information that the public was not aware of, and they should perhaps have held more skepticism than they did. But the gleeful acceptance of outright anti-science lies implies that they were never really in a position to make a sound judgment one way or the other. In those circumstances I'll settle for people reaching the correct action: that practically all accepted medicine is correct and they should follow their doctor's advice. If they choose to over-inflate the importance of things that do indeed go wrong, then they are the ones failing to reach valid conclusions.
- tw85 8mo ago[flagged]
- jfengel 8mo agoLike I said: every word out of your mouth there is a lie. Yes, I know the links you're about to hand me, to right-wing disinformation sites and actual news articles that don't say what you're pretending they say. These are straight out falsehoods, collected for you deliberately, which you are repeating because you didn't even pretend to examine them critically. There is no way to discuss the actual mistakes made during the pandemic when it takes me ten times as long to refute the lies you're spreading.
- tw85 8mo agoYou're very quick to throw around unsubstantiated accusations of spreading misinformation while providing nothing of substance to back it up. Pounding on the keys forcefully doesn't carry an argument. Come back when your temper tantrum subsides.
- damnesian 8mo agoToo late, it's already in the bloodstream, LLMs will be recommending things to pediatric doctors and families from fabricated archives for years, probably.
- deleted 8mo ago[deleted]
- sekuraai 8mo agoThey had access to ChatGPT for last 25 years!
- deleted 8mo ago[deleted]
- Bratmon 8mo agoSerious question: Why do doctors change their practice so much based on one case study? Surely, even if there isn't any malice, a doctor can make a mistake?
- myiosaccount765 8mo agoI wondered this too after reading the original New Yorker article a few weeks back and was quite surprised. However, the article also made me think that once a practice is adopted it’s hard for it to change even if the evidence support changing. (Which is how I expected it to be from the outside) I figured there was some context that I was missing as to why some things are quicker to adopt and others less so. Maybe because adopting this change was seen to be “saving” lives by being more cautious about the how medicines and feeding interact - and reverting the change is “risky” in case there is truth to it.
- aaron695 8mo ago[dead]
- cfu28 8mo agoCase studies are used in medical decision making only when there is no better form of evidence available, or there is a gap in current evidence. It is not the first place to look
- Bratmon 8mo agoBut in this case, doctors across the US and Canada stopped giving pain meds to new mothers because of this one study! Even if the study didn't get retroactively declared to be fictional, that seems like an extreme level of decision-making over one case.