14 ms·
Evidence of Fabricated Data in a Vitamin C trial by Paul E Marik et al in CHEST
- begueradj 5y agoUnbelievable but sadly true. Thank you for sharing
- deleted 5y ago[deleted]
- DantesKite 5y agoFor context: “Remember that Vitamin C cures sepsis paper that could never be replicated in 9 RCTs? Turns out there is a good reason why: it’s very likely fraudulent. More brilliant statistical sleuthing by @K_Sheldrick.” https://twitter.com/nickmmark/status/1506402015936581635?s=21&t=1cbeN1bAO1W-qYYFwA7c2A https://twitter.com/nickmmark/status/1506402015936581635?s=2...
- ALittleLight 5y agoWhat a horrible way to waste medical research resources.
- rmbyrro 5y agoI see it from another angle: it was resources spent to save the public from a lie. It's good investment. Desiring a world where no scientist lies or falsifies data is wishful thinking, in my opinion.
- hn_throwaway_99 5y agoI thought this was very well explained, but the tests that K Sheldrick did were not actually particularly complicated. I remember learning about the Fisher Exact Test in an introductory college stats class. I have to wonder why any of the nine other groups that tried to replicate the results didn't ask for a statistician to look at the original results first. This is flat out fraud, and if the research used any public funds the guilty party/parties deserves criminal prosecution.
- dane-pgp 5y ago> if the research used any public funds the guilty party/parties deserves criminal prosecution. That sort of prompts the question though: Should it be a crime for any scientist to deliberately publish falsified data? Obviously there would need to be a very high bar for proving that it was deliberate and not an honest mistake (or even complete incompetence), and we wouldn't want juries to have the power to stop new scientific theories emerging, but other professions are subject to criminal penalties if they fail at their job.
- lapsis_beeftech 5y agoScientific fraud is deeply immoral because it has the potential to subvert our sapience, harming our ability to act rationally and morally. Such behaviour is criminal from basic principles of justice and human laws should reflect that.
- smabie 5y agoI guess just lying in general should be illegal then?
- pigeonhole123 5y agoLying for financial gain can sometimes be fraud which is already illegal.
- ArnoVW 5y agoIt is? Fraud, libel, perjury. I guess academic fraud is fraud in the legal sense. The problem is that the 'fraud' statute requires 'damages' and 'intent' to have standing, and they're difficult to prove in academic fraud.
- Maursault 5y agoNearly always, lies are protected speech. The Supreme Court has decided, rightly I think, that the government is not capable of determining what is opinion and what is fact.
- nextos 5y agoBig effect sizes tend to be fabricated. It's really disturbing. I do lots of biostatistics in academia and professors openly acknowledge many trials cherrypick data. The outcome is that if you are honest it's hard to get funded because your results look bad. Then, if we talk about honest results only, crappy statistics also leads to crazy optimistic effect estimates. Regularizing priors and pooling should be the norm. Andrew Gelman posts a lot about this: https://statmodeling.stat.columbia.edu/ https://statmodeling.stat.columbia.edu/
- civilized 5y agoPaul Marik is also the co-founder of the Front Line COVID Critical Care Alliance, the foremost group of doctors promoting ivermectin https://www.medpagetoday.com/special-reports/exclusives/96522 https://www.medpagetoday.com/special-reports/exclusives/9652...
- moistly 5y agoIt’s people like that who make me think tar and feathering wasn’t a bad idea.
- WaxedChewbacca 5y ago
- mikeyouse 5y agoI hope (but doubt) that the repeated fraudulent studies, the lack of replicable results followed by very strong negative results from more ethical scientists, and the self-serving grift that the HCQ/IVM/Zinc promoters have engaged in is enough to get people to really step back and consider why they believed these charlatans in the first place. Plenty of ~intelligent people on HN and elsewhere, who would never sign up for a Gwyneth Paltrow detox regime, talked themselves into believing that there was some forbidden knowledge being exposed by these quacks. It was always so transparently silly.
- Maursault 5y ago
- Khelavaster 5y agoDichloroacetate apparently cures cancer as hard as smoking 'cures' meat--kills too many normal cells to be safe to do in living people. https://xkcd.com/1217/ https://xkcd.com/1217/
- Khelavaster 5y ago
- IAmGraydon 5y agoSingle data point, but my grandfather was treated for a systemic infection at Sentara Norfolk General by Dr. Marik using his Vitamin C protocol and it brought him back from the brink of death when nothing else worked. Is the study fraudulent? Could be, and his fight to use Ivermectin on Covid patients always struck me as quackery as well. That said, it seems to me that there is something about the Vitamin C protocol that should not be dismissed.
- civilized 5y agoIt's incredibly unlikely that the Marik Protocol is what made the difference. Following this fraudulent paper, the protocol was tested in 9 randomized controlled trials and they were all negative. And as a statistician I can tell you with absolute certainty this paper is fraudulent.
- bsdetector 5y agoAs a non-statistician I can tell you this blog is absolutely unconvincing, because it doesn't explain anything about what the test is. Apparently this test is how likely it is for the same number of people to have the primary diagnosis in each group because all the 1.0 have equal number of patients with that diagnosis or off by 1. Like if they said, we have 8 people with COPD in the first group and then waited for COPD patients to show up at the ER for the second group until they had 8 then you would get a perfect 1.0 match since they made that variable the same? And then the other variables would be randomish, but maybe correlated so more than 0.5? Maybe you can explain how this layman take is wrong, but if all that's going on is they selected patients with the same diagnoses and didn't make that clear in the paper then I don't understand what the big deal is.
- jmount 5y agoI'll give it a try. The study is two small populations that are assumed to be similar. If this were the case, then knowing the study ID (1 or 2) would sometimes wrongly look predictive for different conditions. However none of the measurements have this flaw, and that it self is very very unlikely. The claim is the data looks like it was designed to not have any imbalance from between sets 1 and 2, to an extend that is itself unlikely due to sampling.
- edgyquant 5y agoSo this is a recent study but surely there’s prior ones as well? How many people actually go and check studies like these for validation? I know there’s reproducibility issues with certain social sciences but how wide spread is this? Is anything being done to address such a glaring dependability issue?
- deleted 5y ago[deleted]
- Khelavaster 5y agoThe protocol is vitamin c + thiamine + hydrocortisone. I'm 100% convinced hydrocortisone is bioactive is sepsis patients, like it is in anyone else . Also 100% convinced vitamin C boosts antioxidant levels in sepsis patients who're vitamin-C-depleted.
- pen2l 5y agoThis is the same Marik behind efforts to treat COVID with Ivermectin. The paper he co-authored about treating COVID was found to have major issues and was retracted. As for this particular paper about the Vitamin C trial (which hasn't been retracted yet): numerous other groups tried replicating it and none could. Millions and millions and millions of dollars, countless hours wasted on conducting controlled trials all because of this guy's fraud. My two takeaways: 1) Kudos to Sheldrick, we absolutely need more people like him. We need statisticians to pick apart data in papers and see if it smells funny. This should have been caught earlier. 2) There should be a higher price to pay for this kind of fraud. For the confusion and mistrust that ensued, for the money and human capital that was wasted, he needs to be behind bars for a long, long time.
- rmbyrro 5y agoI don't think it was wasted. It's well invested capital and time to find out whether something is true or not That's how science work. No matter what we do to discourage fraud, we'll always invest tons of capital and time to find and filter it out.
- linspace 5y agoIt would be better that independent studies tried to detect errors, not fraud. Sometimes it is difficult to distinguish between both but of there is a track record of fraud no reputable journal should accept more papers from that source. Edit: As I re-read my comment I worry because research should not really depend on "prestige", it should be based on cold facts. On the other hand I don't see any other way to deal with malicious actors.
- rmbyrro 5y agoThere will always be fraud and we will always need to invest in trying to reproduce everything in order to sort them out. It's absurd to think there will be a way to know which studies need reproduction or not. The whole purpose of reproducing is to find out which ones are true and which are bogus
- 5y ago
- Sunspark 5y agoI am only a sample size of 1, but twice now ascorbic acid about 1 or 2 teaspoons worth drank seems to have stopped a visual migraine after 15 minutes.
- secretsatan 5y agoI am a sample size of 1 and ibuprofen stopped a visual migraine after 15 minutes Funnily enough, if you should just go online and check the typical duration of a visual migraine, the results may shock you!
- tgv 5y agoVitamin C intake can't even have an effect in 15 minutes, can it? What would be the path from ingestion to some effect on neural oscillations anyway?
- deleted 5y ago[deleted]
- eecc 5y agoWow. Wouldn’t believe it, if not because of life experience. I’ve had this issue, arguing with bad faithed opportunists, whose approach was to just bury competing ideas in FUD. Seems someone summarized it in this succinct statement: [“The amount of energy needed to refute bullshit is an order of magnitude larger than is needed to produce it.”][1] [1]: https://en.m.wikipedia.org/wiki/Brandolini%27s_law https://en.m.wikipedia.org/wiki/Brandolini%27s_law
- rscho 5y agoCall me old-fashioned, but I suspect Sheldrick doesn't really take the measure of the risks. It's like walking on a wire. Very impressive, but just one misstep and you're joining the crocodiles (MD profs) below. Some of them really have unsuspected reach and power. Actually, the sheer dishonesty and nastiness of the MD hierarchy is probably what surprised me the most at the onset of my career. Good luck to this man!
- mathieubordere 5y agoThe world needs more idealists.
- roenxi 5y ago> ...I have not requested access to the raw data or contacted the authors for explanation as the case is audacious... Happily from the sidelines - it probably is fraud and this isn't really an article targeted at the general audience. But the statistical evidence proves beyond a doubt that the data wasn't collected by the methods described in the paper. That doesn't prove it is fraudulent because formally speaking he can't rule out that they just did a very bad job of explaining their methods. Just from life experience, any time anyone goes in guns blazing based on statistical evidence - without actually talking to the people involved - it is often a mistake. Just because someone can't think of an alternative reason says more about their imagination than the reality of the situation. Particularly coming in hot off a twitter argument. That being said, these are unusual and unexpected patterns that show irregularities in the data, and I would certainly share his concerns.
- ai_ja_nai 5y agoI'd like to question how poorly the article was peer reviewed before publication
- JoachimS 5y agoYou need to consider the historical context too. The referenced authors has been found performing statistical fraud before. P(Mistake) is probably lower.
- ImaCake 5y agoNo there is no reason to be skeptical of the author here because he based his evidence on statistics. As the author describes, the p-values listed are effectively impossible to get from real data. Because of the magic of mathematics, we know exactly how p-values are meant to behave when you calculate them repeatedly - uniform random between 0 and 1. There are a battery of tests you can use to interrogate this very well understood distribution. Those tests, as the author states, prove without any doubt that the p-values in table 1 are effectively impossible. You can't make it to those p-values by mistake.
- roenxi 5y ago
- hrgiger 5y agoRemind me the OTC company I have invested year ago, one of the cofounder had a really decent record, I am not expert in the bio research area but i had an impression that he was one of the respected authority in stem cell research. Then I try to find answers for "why stocks are so cheap", turns out his group confidently started fast trials at the beginning of COVID and resulted some deaths. It's hard to judge due to nature of pandemic executing a fast trial but his way of handling the aftermath was totally disaster for his company
- yakak 5y agoIn light of recent trends toward many more authors, I think it is worth revisiting scientific analysis of Ouija boards to look at how many participant processes arrive at a conclusion they find satisfactory without being aware of their own bias.
- soldeace 5y agoOnce I had an argument with a PhD biologist at my university because she was doing a Fisher's exact test in the most insane way I had ever seen: taking floating point ratios instead of integers, and because the stats program she used didn't take floating points as valid inputs for the test---obviously---, her workaround was to multiply these numbers by 100 to get integers. She refused to back down on this madness because "her supervisor did it like this, and his supervisor before him". In the end, her paper was peer reviewed and published and it was the first in a series of let downs that made me decide not to pursue a career in academia. Sometimes it's not like researchers want to tamper with their results. It's just that statistical ignorance is widespread and affect even the reviewers that were supposed to be the first line of defense. Which is a huge problem for an environment that relies too much on p-values and not so much on explaining them.
- verisimi 5y agoBut this is just par for the course. Most studies are flawed. https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis "A 2016 poll of 1,500 scientists reported that 70% of them had failed to reproduce at least one other scientist's experiment (50% had failed to reproduce one of their own experiments)." or at least - this is what it used to say! This has been memory holed. and https://www.bbc.com/news/science-environment-39054778 https://www.bbc.com/news/science-environment-39054778 "According to a survey published in the journal Nature last summer, more than 70% of researchers have tried and failed to reproduce another scientist's experiments."
- bigbillheck 5y agoThere's still plenty of verbage on wiki about this, including more recent work. (Or maybe that Nature survey failed to reproduce?)
- verisimi 5y agoThey have altered the page to be about psychology, economics and water resource management as well as medicine. If you look at this page, you feel as if you have stumbled into some psychological/sociological discussion. This, to me, is an example of how wiki editors are trying to dampen down the implications of the replication crisis. Take a look at the current version against previous versions, eg: https://en.wikipedia.org/w/index.php?title=Replication_crisis&oldid=937393753 https://en.wikipedia.org/w/index.php?title=Replication_crisi... The strong "Overall" statement early in the article is gone, and instead we have lots of dull, makeweight context and background.
- upsidesinclude 5y agoIt seems poignant to acknowledge that while this type of potential deception or fraud hiding in statistical analysis may be to the detriment of society or patients at large, a greater fraud is being perpetrated on the whole of humanity in the 'one size fits all', 'gold standard' of clinical trials. The human body is unique between individuals, drug interaction is unique to body chemistry. This isn't up for debate nor is it absurd to call for genotype specific studies of efficacy. We, collectively, have allowed real miraculous discoveries to flounder as a result of this failure to appreciate personalized medicine. We will look back on this Era in awe with the ineptitude of our methods.
- ImaCake 5y agoRandomised trials are for removing bias. You can certainly run precision medicine genomics randomised clinical trials. They are rarely done because so far our ‘omics tools haven’t been good enough to use, but that is changing now.
- InfiniteRand 5y agoHow much of this type of stat checking in scientific papers could be automated with a well designed AI system
- ImaCake 5y agoQuite a bit. Table 1 is standard fare in human clinical trials and you could certainly check the summary stats in it for sanity. But you would find a lot of edge cases where your automated system would struggle. This is sorta done for meta-analyses that aggregate the results of several studies systematically. I think there are people trying to automate more of it.
- Khelavaster 5y agoShame on Chest Magazine's editors for not catching this either way. Most likely, one report-writer didn't understand that these were MATCHED consecutive patients. The data is so clearly matched, it's hard to believe it's fraudulent.
- ravel-bar-foo 5y agoThis is addressed by the author. Even if matched, these are impressively matched datasets. To the extent that it is questionable the hospital had the number of potential study participants to match them this well.