21 ms·

The illusion of evidence based medicine

324 points by pueblito 5y ago

tomohawk 5y ago
It's not just corruption, it is an idea that seems very correct, but which is subtly wrong. Take a substance like chlorine. It is put into municipal drinking water because it kills things - things that can kill us or harm us. It is credited with making municipal water a lot safer. It has been shown to be safe and effective for most people. The key is most people. Some people, at the edges of the bell curve, cannot tolerate chlorine. If they drink water with it, or bathe in water with it, it harms them. I have family members who have this sensitivity. Since moving to a place with well water, their health has improved immensely. If you were to be guided only by studies that show the safety and effectiveness of the chlorine treatment, then you could overlook the fact that some people are outliers. Evidence based medicine will guide you to things that will work for most people in a given situation. However, for a specific patient, you may need to depart from that evidence. Limiting doctors to only evidence based approaches prevents them from treating specific patients.
- auslegung 5y ago
  I think you’re getting down-voted because yourcomment indicates you think this article says that doctors should only be able to use evidence-based approaches. That’s not at all with this article is about.
- stonemetal12 5y ago
  What do you think evidence based means? There is evidence that some people are sensitive to chlorine. In an evidenced based approach it would be considered a possible cause until there is evidence to rule it out. If other issues present similar symptoms and are more likely then they should be tested and ruled out first.
  - obscur 5y ago
    Might be wrong, but I think they just mean that when you do large scale trials you aggregate over the population, and determine if the medication had a statistically significant effect. You can get evidence that a subgroup does not react well, but you might not, if that is not your focus. A key concept and solution here is 'precision medicine', tailoring medicine based on more precise information about the individual.
- nonameiguess 5y ago
  The EPA guidelines limit chlorine content in potable water to 1 PPM. That is well below the level at which even the most sensitive person is affected. The problem is this guideline has no legal enforcement mechanism, so whether or not your municipal water district follows it is up to them. If they're not, they're not practicing evidence-based anything. They're ignoring evidence-based guidelines for some other reason. Whether any US water district is really doing this seems debatable anyway. Maybe this is just a consequence of the poor state of search engines, but every source I can find claiming specific US water supplies contain unsafe levels of chlorine seem to be exclusively limited to companies trying to sell you expensive filtration systems you almost certainly do not need.
- rscho 5y ago
  That some people are sensitive to chlorine should then be scientifically studied, to expand the knowledge on that topic. Counting on individual practitioners to cook up artisanal individualized treatment on their own is both unscientific and the antithesis of what you (and everyone) want.
  - tricky777 5y ago
    Evidence is king. But hyphoteticaly... if I'm an outlier, and blind test is made on only me, and reveals that I am sensitive to chlorated water. ...I would count that as evidence based.
  - tomohawk 5y ago
    Where did I say a practitioner should "cook up artisanal treatments"? That's a strawman, for sure. Doctors have a lot of education, and gain a lot of experience over time. What I pointed out is that the studies are not enough. Where they leave off is where a doctor needs to be able to apply their education, experience, and knowledge of the case to make progress. Doctors are trained to apply scientific approaches to bridge the gap between the broad science and the individual patient. The big health co's do not like doctors doing this because it takes time and is expensive. So, they tie their hands because they only want the doctors to focus on the easiest fixes that solve the most problems.
    - rscho 5y ago
      I am a doctor. That's not a strawman at all. >Doctors are trained to apply scientific approaches to bridge the gap between the broad science and the individual patient. No. Really, I mean it. Doctors are trained in textbook approaches, not in the scientific method. The less creative, the better the doctor. That's precisely what evidence-based medicine is, and it's what allowed us to leave the Dark Ages of paternalistic medicine. > The big health co's do not like doctors doing this because it takes time and is expensive. So, they tie their hands because they only want the doctors to focus on the easiest fixes that solve the most problems. We tie our own hands. With guidelines. Yes, the system is tuned to a limited investment of time and money in each patient for distributive justice reasons, but guidelines are (mostly) there to discourage crackpot medicine.
- dekhn 5y ago
  chloramine is replacing chlorine in municipal treatment. I'd be amazed if somebody was really sensitive to chlorine, to the point where typical municipal water caused rashes. After all, all of us contain 80grams of it and consume 500mg/day.
- celdon25 5y ago
  > Take a substance like chlorine. It is put into municipal drinking water because it kills things - things that can kill us or harm us. It is credited with making municipal water a lot safer. > It has been shown to be safe and effective for most people. And yet it wasn't until I was well into my 30s that I learned that not cleaning your shower head often enough or thorough enough can block the effects of the chlorine and spray aerosolized bacteria everywhere. It doesn't matter what kind of medicine you have when there are plenty of problems that would be prevented if not for the obscene asymmetry of information and poor cultural health habits. But as long as big tech continues to believe that steering people toward political propaganda is what's best for society, that will be the narrative of the day rather than real progress.
  - manwe150 5y ago
    And I was today years old (not much younger than you were)
qnsi 5y ago
Medical industry is corruption prone. Another example: Industry sponsored conferences. Doctors are invited to a 3 day 4 star hotel stay to attend a conference. Technically, talks that are paid shilling are specially marked as sponsored talks. But are the other doctors, who are getting paid to speak there 100% independent? Will you be a 100% independent doctor after getting free presents? Maybe you will. But industry probably found its profitable if they spent millions on these kinds of conferences. Also lie repeated multiple times makes you believe in, there are psychological studies. So when you spend hour listening how drug X is gold, you might start believing its gold
- Enginerrrd 5y ago
  >Will you be a 100% independent doctor after getting free presents? >Maybe you will. But industry probably found its profitable if they spent millions on these kinds of conferences. So... theoretically the industry putting on the conference is also pushing the state of the art pharmaceutically and believe that they have an improvement on the existing standard of care. I don't think it's got anything to do with the presents as much as the message.
  - pessimizer 5y ago
    Nah. Speakers fees are a way to launder direct contributions. There was at least one famous case where the "conference" was done in one of the speakers' living rooms. Imagine getting a speaking fee for speaking extemporaneously in a Florida hotel room with two pharma reps and one other doctor five minutes before you all leave for golf.
    - nobody9999 5y ago
      >Nah. Speakers fees are a way to launder direct contributions The comparison to money laundering is specious, IMHO. In the US, at least, there is a database[0] of any value (including monetary and in-kind) exchange between doctors and medical companies. As such, there is no "laundering" at all. All moneys and in-kind (hotel rooms, meals, swag, etc.) payments are documented and detailed. Is there an incentive for doctors to favor a particular pharma or medical device company based on those payments? Perhaps. But since (again, at least in the US) such payments are documented and publicly available, it isn't some sort of secret set of payments designed to surreptitiously co-opt doctors. Sure, some payments (my brother, a physician, received ~US$20,000 in 2020, mostly (~$13,000) from a single consulting fee. Is my brother favoring the company who paid him $13,000 in 2020? Maybe. But if and only if their products/devices have clear beneficial effects over other products/devices. What's more, that $20,000 (aside from the consulting fee, it was food/beverage and other in-kind stuff) is a small fraction of his annual income and doesn't make a significant difference in his quality of life. As such, while there may certainly be doctors who are co-opted/corrupted by medical companies, assuming that the majority of doctors are swayed by such things is iffy at best. [0] https://openpaymentsdata.cms.gov/ https://openpaymentsdata.cms.gov/
twofornone 5y ago
>Scientific progress is thwarted by the ownership of data and knowledge because industry suppresses negative trial results, fails to report adverse events, and does not share raw data with the academic research community. Not to derail the thread but this nicely encapsulates my reservations toward the COVID vaccines, and why I think mandates are egregiously unethical. The raw clinical trial data is still effectively a secret between pharma and the FDA, and both are under immense financial/political pressure to release a "safe and effective" product, not to mention the incestuous relationship between the ostensibly independent FDA and big pharma.
- sebmellen 5y ago
  I find it very discomfiting that anyone suspicious of the COVID-19 vaccine (by that I mean the COVID vaccine/spike protein in particular) is branded as anti-vax. I say this as a vaccinated person who caught a heavy bout of COVID. I suppose I'm grateful for the vaccine, but mandates scare me.
  - specialist 5y ago
    Do you school-aged children? Pets?
  - tricky777 5y ago
    as chistians say, "turn the other cheek". (labeling did not made much harm, dont get too ivested in that) reality is, that we can't have policy/public sentiment that is nuanced. (I heard it breaks the timecontinuum) Find the silver lining and move on. I'm just glad that, a lot of people are in pro-vaccine side overall. While in your case it is a bit of mob mentality (mostly appeal to authority + optimism + jugde_dissagreeing_side), in the long run "pro vaccines" is (even if only on average) leaps and bounds and couple of parsecs better than "no vaccines".
  - deleted 5y ago
    [deleted]
- simonh 5y ago
  To a point that's understandable, based just on trials it's reasonable to be skeptical of the results. However once the vaccines have been in mass distribution and there is copious evidence of their efficacy and the extreme rarity of adverse affects, I don't think it's reasonable anymore. The mandates only came in once enough of the population had been inoculated to make a requirement possible. At that point the initial trials are a footnote compared to the results from mass vaccinations. IMHO it's just not reasonable for that objection to carry any weight. There may be other reasons, of course.
cosmotic 5y ago
This should not be considered evidence that other quack medicines have any superiority over modern medicine.
- trulyme 5y ago
  True. It is just evidence that sometimes modern medicine doesn't have any superiority over quack medicines either.
smohare 5y ago
One of the issues I’ve seen with EBP is that it’s often not necessarily science based. That is to say, prior plausibility derived from basic science isn’t fully incorporated, ergo undo weight is given to noise. You see with in almost any study of pseudoscientific modalities, where actual efficacy would require a drastic reformation of our fundamental, very well tested models of the natural world. Why should a particularly strong one-off result for my made up pain treatment be afforded the same stature as that of an pharmacological agent with clear physiological pathways?
- rscho 5y ago
  Evidence-based medicine does incorporate prior evidence for causal evaluation. Check out the "Bradford criteria". They're the basis of causal evaluation in epidemiology.
chicob 5y ago
After reading this article I find only one problem with it, and that is the title. The problem the authors have so perfectly characterized has nothing to do with the emphasis on evidence that medicine requires, but on the distortion imposed on the system by large pharmaceutical companies. The reality of the obstacles standing in the way of rigorous testing for drug or therapy effectiveness does not make evidence-based medicine an illusion. In fact, the authors actually make a strong argument in favor of evidence-based medicine, by stating the need to acknowledge these biases. And that the reason for all these biases is greed.
- pmoriarty 5y ago
  The article shows how what passes for evidence based medicine is an illusion. The "evidence" is biased and corrupted by corporations, as are many of the people who evaluate it, set the standards, and make decisions.
- xmprt 5y ago
  I think you're misunderstanding the title. It's not that evidence based medicine has problems. It's that there are problems with how evidence based medicine is currently practiced which gives it the appearance of evidence based medicine while actually not being anything close, that is, an illusion.
  - chicob 5y ago
    Well, yes. My point was simply that the title misrepresents the article, because it is ambiguous. "The illusion of evidence based medicine" can be interpreted as a statement of how the aim of evidence based medicine is illusory. The short introductory paragraph, "Evidence based medicine has been corrupted by corporate interests [etc]" furthers this idea, having a somewhat definite tone, as if this corruption were fatal to any proper scientific inquiry. The rest of the article dissipates this confusion rather quickly. Stepping back a little, how would one react to a title such as "The illusion of medicine", making the same general argument?
- ouid 5y ago
  Stop calling it greed. It's corruption and regulatory capture. Both of these, unlike greed, are problems that can be solved.
rscho 5y ago
Evidence-based medicine, more than the indeed important issues raised in the article, is mostly impaired by the absolutely abysmal average quality of clinical research papers. The absolutism of higher-ups combined with their unrivalled (in the scientific world) statistical ignorance is hurting science more than anything else. And there's a reason to that: MDs are not scientists. We're highly-skilled workers, and more of the blue-collar kind.
- dr_dshiv 5y ago
  Disagree. Think your standards are too high.
  - throwoutway 5y ago
    Your standards are too low. Human lives are at stake in medicine. Do you want a doctor treating you when they are misinformed by fake science?
    - dr_dshiv 5y ago
      I want doctors to not treat studies like “science sez” and instead recognize that “Science is messy”. They should use their training and experience to heal, not the latest publication.
    - webmaven 5y ago
      I'm not sure that's any worse than a doctor misunderstanding real science. Note that the root cause (unfamiliarity with the tools of science, especially probability and statistics) is largely the same.
      - rscho 5y ago
        > I'm not sure that's any worse than a doctor misunderstanding real science It is worse. A highly experienced practitioner who would've done the right thing might get led astray by new bogus research. Precisely because most practitioners don't have the ability to judge scientific work on their own.
slibhb 5y ago
I think profit motive and regulatory capture are real problems but I don't think they're the only problems and maybe not the biggest. For example, the FDA is generally agreed to have dropped the ball by dragging its feet on approving covid tests. I don't think profit motive or regulatory capture explain that. What does? More transparency (as the authors recommend) sounds great but giving more power to regulatory bodies is no panacea.
- mikekij 5y ago
  > For example, the FDA is generally agreed to have dropped the ball by dragging its feet on approving covid tests. I don't think profit motive or regulatory capture explain that. What does? I'm no fan of over-regulation, but the FDA finds itself in an impossible situation: approve tests quickly to slow the spread of Covid, only to later find that the tests don't work, or test the products rigorously, while patients suffer. It's a very difficult balance to strike, and I would argue they've done as good a job as one could hope.
- ouid 5y ago
  Big Pharma derives its oligopoly from the barriers to entry that the FDA imposes. Presumably they take every opportunity to lobby for more rigorous extensive testing. Not because they want to pay more to run tests, but because they want their smaller competitors to pay more to run studies for drugs that might compete. This is, of course, not an explanation for how they have variously captured the components of the FDA, but it does explain how such a capture is an explanation for foot-dragging.
yes_man 5y ago
Another great argument I have heard against EBM (or maybe rather against being religious about it) is that the studies and papers held to highest regard are made in circumstances that do not apply to most of healthcare: the studies might be done in the best hospitals like Johns Hopkins, with practically limitless human resources for the study, and very carefully selected patient groups. In a normal hospital situation you won’t have the time or the resources to dedicate the same care for each patient, and the patients might have other conditions (which would rule you out of the big studies). Still, as a layman who is not a doctor, evidence based medicine sounds like a good idea, but as with everything, taking it to extreme is probably harmful
caycep 5y ago
granted, there is the day-to-day human physician "bayesian" evidence-based-medicine...for better or worse...
TeeMassive 5y ago
Here in Québec the vaccine passport and the curfew was touted as the pinnacle of "following the science"(tm). Anyone contradicting their usefulness as effective sanitary measures were accused of "spreading misinformation"(tm). The curfew was implemented twice. The vaccine passport was only recently removed. Turns out, one FOIA request later, that they never consulted the data scientists of the public health institute. They made it all up.
tensor 5y ago
Drug company bias is definitely a well known issue. But there are even bigger issues with science based medicine. * MDs are not PhDs for one, though people often equate the two. * Things like surgery is in fact hard to test. There are ethical issues around it. E.g. are you really going to do a shame surgery "for science" if you think there is a chance it might save the patients life? On the flip side, surgery has a lot of risk, and performing surgery that doesn't work exposes patients to unnecessary risk. * There are entire fields where pseudo-science is common. Notably a lot of physical therapy and chiropracty is not science based.
- willhinsa 5y ago
  On the difficulty of testing lifesaving treatments, there's a great paper on this topic: "Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials" [0] It has some great gems like: - No randomised controlled trials of parachute use have been undertaken - The basis for parachute use is purely observational, and its apparent efficacy could potentially be explained by a “healthy cohort” effect - Individuals who insist that all interventions need to be validated by a randomised controlled trial need to come down to earth with a bump [0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/
  - pmyteh 5y ago
    There was, subsequently, published a Christmas BMJ article reporting the results of an RCT of parachute use jumping from planes. Apart from a few minor confounding problems with selecting the participants, it's very good science! Found no evidence of benefit, sadly... https://www.bmj.com/content/363/bmj.k5094 https://www.bmj.com/content/363/bmj.k5094
    - willhinsa 5y ago
      So funny! > RESULTS Parachute use did not significantly reduce death or major injury (0% for parachute v 0% for control; P>0.9). This finding was consistent across multiple subgroups. Compared with individuals screened but not enrolled, participants included in the study were on aircraft at significantly lower altitude (mean of 0.6 m for participants v mean of 9146 m for non-participants; P<0.001) and lower velocity (mean of 0 km/h v mean of 800 km/h; P<0.001). > CONCLUSIONS Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention. However, the trial was only able to enroll participants on small stationary aircraft on the ground, suggesting cautious extrapolation to high altitude jumps. When beliefs regarding the effectiveness of an intervention exist in the community, randomized trials might selectively enroll individuals with a lower perceived likelihood of benefit, thus diminishing the applicability of the results to clinical practice.
medymed 5y ago
Science marches on one funeral at a time, medicine marches on one expired patent at a time.
- rscho 5y ago
  No. For medicine, both are true because funerals of both patients and professors are a signal for progress.
deleted 5y ago
[deleted]
tsoukase 5y ago
EBM suggests that swimming in a heated swimming pool (controlled studies) can be the same as swimming in open sea/ocean (applying the result in any possible case)
- FabHK 5y ago
  The distinction you allude to is so well known and understood that there are not only papers about it, but also special words to distinguish them: efficacy (performance of an intervention under ideal and controlled circumstances) and effectiveness (performance under ‘real-world' conditions). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3726789/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3726789/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3912314/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3912314/
  - tsoukase 5y ago
    The problem is there is no EBM without p-values and there is no real world without Bayesian reasoning. So, the two are fundamentally disconnected
deleted 5y ago
[deleted]
rafaelero 5y ago
There is one simple solution: replication. Stop complaining and start replicating.
Terry_Roll 5y ago
As someone who recently complained to the UK Royal College of Veterinary Surgeons and got palmed off because the UK legislation doesnt quantify "reasonable care", the whole system is one giant authoritarian money making scam. I dont know what professionals are being taught in med/vet/law school now a days, but technology is increasingly demonstrating that professionals could be out of date by the end of their course! So that problem of trying to stay current in IT is spreading out into other professions and I dont see them tackling this problem like the IT sector has/does.
- throwawaylinux 5y ago
  > As someone who recently complained to the UK Royal College of Veterinary Surgeons and got palmed off because the UK legislation doesnt quantify "reasonable care", Pretty hard to quantify. "Reasonable" is a widely used concept in the law for this reason. What were you expecting them to do? How would you define "reasonable care", exactly? > I dont know what professionals are being taught in med/vet/law school now a days, but technology is increasingly demonstrating that professionals could be out of date by the end of their course! Surely nobody ever thought otherwise. Doctors aren't superhumans, none will ever know everything and there are good and bad ones. Exactly the same as any other person in any other field. > So that problem of trying to stay current in IT is spreading out into other professions and I dont see them tackling this problem like the IT sector has/does. The IT sector does? How? I thought IT was pretty mediocre at it. And the challenge of staying current with new developments predates the computer by a few millennia doesn't it? This isn't something new to IT. Have you actually looked to see how the medical field tackles this problem? I'm not in the medical field but some family members are. The amount of training, certifications, exams, etc they are required to do is staggering. Far more than any software developer I've ever known.
  - Terry_Roll 5y ago
    All fair points, but medicine is highly cautious, and most would prefer to go with the larger entities opinions/research. So Merck (https://en.wikipedia.org/wiki/Merck_Veterinary_Manual https://en.wikipedia.org/wiki/Merck_Veterinary_Manual) have a massive influence which introduces lag, although some do read pubmed to access more up to date studies, but (google) scholar can be a better search engine to use over pubmed. I'm also aware of the Eli Pariser Filter Bubble which will affect what the vets see and what I see. I also have to do repeated searches sometimes over a few weeks, months or years to see what the search results are throwing up which it didnt previously. > Have you actually looked to see how the medical field tackles this problem? Yeah and they dont tackle the problem to my satisfaction instead using older technology/medicine. No one wants to put their neck on the line because they dont want to look at human studies where biological similarities exist, or different species are used, like the good old rat. So some in-vitro studies exist because its impossible to study in-vivo, the tech just doesnt exist for in-vivo and that is an area where they are highly cautious and there does seem to be a general dislike for using non patentable chemicals!
sammalloy 5y ago
One thing that has always stuck with me in regards to this topic is the lack of medical research in areas that need it but don’t get it because there isn’t any money in it. One area that comes to mind is feline urinary tract disease. Many years ago, I had a cat who kept coming down with this, and the vet(s) were at a total loss as to how this kept happening. So I decided to open up the literature and plough into it as a layman with no knowledge about animal medicine. Within the space of a single hour I quickly learned that many of the most common diseases facing animals have very little research behind them and a lot of unknowns. Which brings me back to one of the main points of the author. If the neoliberal approach to research is only going to focus on what is profitable to treat, then medical science as a whole has backed itself into a dark corner.
- dayofthedaleks 5y ago
  I'm sorry but veterinary medicine is an entirely different problem domain with little overlap beyond capitalism and mammalian biology.
  - StreamBright 5y ago
    No it is not. Maybe you have heard about animal testing before. If what you say is true we hadn't been using animal testing.
    - wins32767 5y ago
      Animal testing is to test human drugs/treatments/cosmetics on animals to see if they are safe enough to test on humans.
      - caddemon 5y ago
        A lot of basic research is also done in animals, it's a pretty important part of the process that led to many drugs being tested in the first place. But it's true an understanding of the animal is usually treated as an unimportant tangent. Like you might introduce a mutation into a mouse gene that is analogous to a mutation thought to be relevant in humans and then look at how that affects some other biomarker of interest in the mouse. But you're not going to be studying some issue that arises naturally in mice. Hell you're only using the mouse because it's an established lab model, you don't give a shit about mice. Also, all the mice are hella inbred and grow up in a sterile cage, so they're hardly even reflective of IRL mice. So even if someone did want to fund lots of research on mouse veterinary science, it would still be a long road (technically and culturally) to get that integrated with basic medical research. I'm not saying this is the way it should be, but it's the way it is.
DarkmSparks 5y ago
Massive, major flaw in their argument. "without seeing the raw data" is not true. At least in the UK. All raw study data, published and un published must be provided before a medicine will be reimbursed. It is reviewed by teams who do their utmost to prove the med is worthless. Same for the FDA afaik. All the data is generally "commercial in confidence" so any old twitteraty can not download and misinterpret it. All trials must be registered on Clinicaltrials.gov or they cannot be used as evidence. Publication bias is a well know issue, resolved for more than a decade. No idea how this article made it into the BMJ, smells like click bait.
- rscho 5y ago
  > Publication bias is a well know issue, resolved for more than a decade. How is that resolved? This is not what I see in my meta-analyses.
  - DarkmSparks 5y ago
    requiring registering with clinicaltrials.gov and regulators doing their own analysis on all the data. Managed entry agreements and conditional reimbursement (dont pay for patients that dont do better than the average of current practice) Requirements for "real world data" collection and submission post approval. A whole host of other mechanisms I cant remember off the top of my head. Also "evidence based medicine" doesnt mean the evidence is infallible (e.g. rare disease meds have a huge issue with collecting any meaningful data) - it just means actually using evidence rather than just taking the industries and acedemias word for it that it works like the old days.
    - rscho 5y ago
      From personal experience, I think you are massively overestimating the reach and effectiveness of regulations.
      - DarkmSparks 5y ago
        I've worked with nice. several of my students still do. We've kicked out of the NHS quite a few meds the industry and academics were wrong about, based on the evidence, not what the industry/academics said before they were shown to be not correct. The whole posit of this article is wrong. "acedemics said so" is not evidence based medicine, it is one small facit of it, and the author matters, no one trusts "sponsored by" papers, NIHR is constantly publishing counter evidence. Some of that even ends up in court (e.g. Avastin) https://www.reuters.com/article/us-novartis-bayer-britain-idUSKCN1M12EC https://www.reuters.com/article/us-novartis-bayer-britain-id...
threads2 5y ago
There's so much money in cancer research - when I was a pharmacology/genetics researcher I was told to try to tie everything back to cancer for research papers and grant proposals. And the oncology lab next door had the best equipment. We always had to borrow their stuff. Idk maybe this is a good argument against private charity. Not to say that cancer research isn't important but from what I could tell, cancer researchers had an embarrassment of riches.
conorh 5y ago
My wife and I were just talking about this yesterday. She is an extremely specialized surgeon and she is horrified by the academic publishing in her area of expertise. It is often utterly wrong and conflicts directly with her experience. I'm not sure she'd agree with the reasons mentioned in this article here as the main issue - although we have personally seen less ethically motivated doctors publishing research to help line their pockets via the industry. In her opinion the main issue is that academic doctors HAVE to publish in order to advance their careers. So you end up with a never ending stream of terrible research, churned out by doctors that are not experts in doing research and that actually spend most of their time doing other things. This happens even at the best centers; at least in her area of expertise. I don't know what the solution is, but I do know that whenever she has needed treatment for herself she generally ignores most of the research and talks to the most experienced surgeon she can find.
- rscho 5y ago
  This is all true. Welcome to the real world. > academic doctors HAVE to publish in order to advance their careers Indeed. But the issue is what criteria will you use for promotion if not that? Don't even think of evaluating clinical results, that's even worse than what we have. > This happens even at the best centers This happens ESPECIALLY at the best centers. Because centers are in good part ranked according to citation counts. > I don't know what the solution is, but I do know that whenever she has needed treatment for herself she generally ignores most of the research and talks to the most experienced surgeon she can find. After 10+ years in the system, I'm in the same boat. And although I've given it a good thought, I don't see a solution either.
  - derbOac 5y ago
    The truth this article touches on is that it's not even really about citation counts anymore, it's about dollars. Via drug companies, private grants, federal grants, any sorts of money coming in from outside the university or hospital. I've sat on promotion meetings where people aren't publishing, but are bringing in 2 million dollar grants and "it's all ok". This incentive structure distorts research. What to do about it? I'm not exactly sure but having university administrations who see research as a means to make money, rather than an end in itself, is not the way.
jrapdx3 5y ago
Over the years I've practiced medicine the idea of "evidence-based" care has had its ups and downs. Part of the problem lies in the question of what is "evidence". From what I've seen, heard and read, the question is broader than pharmaceutical research though that's certainly an important element. The evidence literature spans making diagnoses, applying drug and non-drug therapies, deciding what is a successful outcome of treatment, etc. What I see is that clinicians are quite willing to use relevant and effective evidence-based methods. One problems applying evidence to real-world practice arises out of the aggregate bases of evidence. Even if evidence-based approaches could work in typical cases, many patients vary enough from norms that following evidence-based recommendations doesn't work. Furthermore it shouldn't be surprising when low-quality evidence gains corporate or governmental backing. When such promoted "standards" defy clinical observation it tarnishes the idea of evidence-based practice. An example is fairly recent recommendations that men over 70 shouldn't have routine PSA lab tests. Urologists will tell you about the hundreds of men they see with early stage prostate cancer detected by periodic PSA tests. Following the "evidence" would result in far more deaths and disability so it's ignored. As the article points out corruption of "evidence-based medicine" has destroyed much of the promise the idea once held. Insurers, pharma houses, governments contribute to abuse of evidence to serve their own agendas. No doubt some fraction of practitioners have added to problems and that needs to be halted. But IMO regulators are quick to point fingers at all clinicians for failures of health care systems. Rather most providers would prefer to avoid corrupt "evidence" whereas real evidence is welcome to workers toiling in the trenches.
- wolverine876 5y ago
  Thanks for sharing your perspective. > An example is fairly recent recommendations that men over 70 shouldn't have routine PSA lab tests. Urologists will tell you about the hundreds of men they see with early stage prostate cancer detected by periodic PSA tests. Following the "evidence" would result in far more deaths and disability so it's ignored. Isn't that how any system works - the rules provide guidance (with some teeth), and the people working hands-on apply their judgment? I recently was reading about the US Army's process. Many think of the military as highly regimented, but the US Army encourages - in fact, requires - subordinates to ignore orders that don't apply to their current situation; they are required to fulfill the 'commander's intent'. Only broken, authoritarian or totalitarian systems try to apply the theoretical centralized control to practice, where it is so rigid that it is bound to fail at scale (like another army in the news, apparently). What central authority is intelligent enough, has the information for, and the vision for anticipating every situation now and in the future, knowing the best solution, and writing rules for it?
faangiq 5y ago
We all live inside our own illusions.
refurb 5y ago
It would appear that the authors are selling a book on this topic, which explains the BMJ article. And I'm not sure their premise is all that accurate. If they think the FDA is in the pocket of pharmaceutical companies, that runs entirely counter to my experience. I can recall plenty of times where we'd suggest a change to a filing and the FDA response was "nope, changes denied". If you think the low level reviewers of submissions are secretly pulling the strings for Pfizer, you'd be wrong. And in terms of the quality of publications - this is nothing new. There has always been a robustness to clinical trials. That's why the NCCN guidelines for cancer treatment actually rate the clinical evidence from Category 1 (strongest) to Category 3 (no evidence whatsoever).
t0bia_s 5y ago
https://news.ycombinator.com/item?id=30761070#30796020 https://news.ycombinator.com/item?id=30761070#30796020 I put that article three days ago here, isn't there any moderation for double submissions?
mola 5y ago
This is how dark ages start. Actual knowledge is watered down by corporate nonsense. Society loses confidence in the scientific establishment. The organizing coordinate that took center stage in the age of enlightment will be lost. Society still need some way to organize its cognition. Power and superstition will take center stage again.
- WA 5y ago
  This is how conspiracy theories start. Take a thing that has some truth to it and extrapolate it all the way up, dismissing that the whole system is probably more complex.
IG_Semmelweiss 5y ago
The problem is misaligned incentives. You need to go after the "research trolls". Same as you would with patent trolls. Hit them where their money is. Pass a law that anyone anyone ultimately funding or profiting off research can be sued by the public for double damages x cost of the research if : - paper won't replicate - paper sponsors & costs are not disclosed. - paper does not include instructions to make data requests - researcher does not release data within 30 days of request. - paper publisher (whomever that could be, even faculty) failing to flag prominently within 30 days, any paper that has lost in court. So if a university employs 1 researcher at X salary, and time spent on research is Y, the university can be sued for Y * X * 2 by Joe Schmo. Maybe this would be small claims court. Same would apply to any research created by big AG, Pharma, etc. If researcher uploads paper in his own personal website and forgets to flag it as flawed, he now pays the price too. It would be necessary to cap disgorgement at 10x initial research cost. The point is to let the crowd hunt down garbage research, not bankrupt anyone for honest mistakes. I would also default to making the defense be rewarded by 2x incurred legal costs if they win case to ensure lawsuits have some merit. This would immunize students if they are not paid to make research (but not PHDs that live off stipends), or random people doing research, out of goodwill. Professionals would be very careful to publish garbage lest be fired from their jobs, and companies & researchers would think twice before pushing propaganda. No more research mills of BS.
yolo69420 5y ago
science is a pretty long term process. in practice results and consensus stabilizies after around 50 years or so. the reason it takes so long is that the turnaround time for a single result in science is half a year to a year. imagine discussing something with someone and every sentence you speak could only be uttered after a year. articles like OP always hinge on some short or medium understanding of science. they try to point out that the whole approach is wrong based on the intermediate results of the process. OP itself is part of this process and the reason why the results will be better and more stabilized a couple of decades from now.
- chithanh 5y ago
  > consensus stabilizies after around 50 years or so One other reason for science being such a long-term process is Planck's Principle: https://en.wikipedia.org/wiki/Planck%27s_principle https://en.wikipedia.org/wiki/Planck%27s_principle Basically, scientific innovations spread by the old orthodoxy retiring/dying and replaced by successors who are familiarized with the new ideas (and then proceed to form the new orthodoxy).
- dgb23 5y ago
  But there is also the social problem. Incentives and power structures matter, sometimes to a comical degree. Openness and collaboration are fundamental to progress.
YeGoblynQueenne 5y ago
I have no idea if any of this is true, but it rings true and it reminds me of similar distortions of the integrity of scientific inquiry by the interest of industry, in my own field of study: artificial intelligence and machine learning. And perhaps that's no surprise. It's the same universities that produce both kinds of research (and researcher) after all.
jokoon 5y ago
In france we have this research director, Didier Raoult, who pushed for the use of Chloroquine as a treatment against COVID. He was the head of a research center dedicated to tropical diseases. This guy literally argued that the smaller the test sample, the better the result. This guy lost his job of director (not to mention investigation of research fraud), but he mades it very difficult for uneducated people to trust the vaccine. Not to mention that I dated a biology scientist and teacher who did not want to get vaccinated. I heard 2 other scientist lost their jobs in a research lab in my city because of not wanting to get vaccinated. So no matter what people will tell me, it seems that science is rife with people who should not have their credentials, despite high level education, universities, "merit based" grades, etc. It reinforce my cynicism, because I had some scientific education, but I dropped and never reached a degree. It's what Pierre Bourdieu talks about, that degrees are a way of maintaining social classes. It's exactly what you hear in the movie "Good will hunting", that a degree is just a piece of paper, while people can just educate themselves by reading science books.
user3939382 5y ago
I've been more or less of this opinion since watching this talk years ago by Dr. Jason Fung: https://www.youtube.com/watch?v=z6IO2DZjOkY https://www.youtube.com/watch?v=z6IO2DZjOkY
phtrivier 5y ago
Cursory reading is : EBM would be great, but we can't trust the "E" in EBM because people selling the "M" in EBM have a vesting interest in producing fake data to justify selling more to the institutions hoping to do the "B". It seems like the author think more "independant" parties would help (regulators ? white-label labs to do the testing instead of pharma labs ?) All I can tell from the last two years is that I doubt people would really trust a "government agency" (or, anyone "paid by the taxpayers") more than "the corporate" lab. (Case in point: the EU agencies that banned or suspended or restricted usage of some COVID-19 vaccines, at various points in the last year, are still mistrusted on the ground that they did not ban or suspend _one_ of the vaccines - which happens to be the most popular - hence "they must be in the pocket of lab X".)
theknocker 5y ago

324 points by pueblito 5y ago

It's not just corruption, it is an idea that seems very correct, but which is subtly wrong. Take a substance like chlorine. It is put into municipal drinking water because it kills things - things that can kill us or harm us. It is credited with making municipal water a lot safer. It has been shown to be safe and effective for most people. The key is most people. Some people, at the edges of the bell curve, cannot tolerate chlorine. If they drink water with it, or bathe in water with it, it harms them. I have family members who have this sensitivity. Since moving to a place with well water, their health has improved immensely. If you were to be guided only by studies that show the safety and effectiveness of the chlorine treatment, then you could overlook the fact that some people are outliers. Evidence based medicine will guide you to things that will work for most people in a given situation. However, for a specific patient, you may need to depart from that evidence. Limiting doctors to only evidence based approaches prevents them from treating specific patients.

auslegung 5y ago
I think you’re getting down-voted because yourcomment indicates you think this article says that doctors should only be able to use evidence-based approaches. That’s not at all with this article is about.
stonemetal12 5y ago
What do you think evidence based means? There is evidence that some people are sensitive to chlorine. In an evidenced based approach it would be considered a possible cause until there is evidence to rule it out. If other issues present similar symptoms and are more likely then they should be tested and ruled out first.
- obscur 5y ago
  Might be wrong, but I think they just mean that when you do large scale trials you aggregate over the population, and determine if the medication had a statistically significant effect. You can get evidence that a subgroup does not react well, but you might not, if that is not your focus. A key concept and solution here is 'precision medicine', tailoring medicine based on more precise information about the individual.
nonameiguess 5y ago
The EPA guidelines limit chlorine content in potable water to 1 PPM. That is well below the level at which even the most sensitive person is affected. The problem is this guideline has no legal enforcement mechanism, so whether or not your municipal water district follows it is up to them. If they're not, they're not practicing evidence-based anything. They're ignoring evidence-based guidelines for some other reason. Whether any US water district is really doing this seems debatable anyway. Maybe this is just a consequence of the poor state of search engines, but every source I can find claiming specific US water supplies contain unsafe levels of chlorine seem to be exclusively limited to companies trying to sell you expensive filtration systems you almost certainly do not need.
rscho 5y ago
That some people are sensitive to chlorine should then be scientifically studied, to expand the knowledge on that topic. Counting on individual practitioners to cook up artisanal individualized treatment on their own is both unscientific and the antithesis of what you (and everyone) want.
- tricky777 5y ago
  Evidence is king. But hyphoteticaly... if I'm an outlier, and blind test is made on only me, and reveals that I am sensitive to chlorated water. ...I would count that as evidence based.
- tomohawk 5y ago
  Where did I say a practitioner should "cook up artisanal treatments"? That's a strawman, for sure. Doctors have a lot of education, and gain a lot of experience over time. What I pointed out is that the studies are not enough. Where they leave off is where a doctor needs to be able to apply their education, experience, and knowledge of the case to make progress. Doctors are trained to apply scientific approaches to bridge the gap between the broad science and the individual patient. The big health co's do not like doctors doing this because it takes time and is expensive. So, they tie their hands because they only want the doctors to focus on the easiest fixes that solve the most problems.
  - rscho 5y ago
    I am a doctor. That's not a strawman at all. >Doctors are trained to apply scientific approaches to bridge the gap between the broad science and the individual patient. No. Really, I mean it. Doctors are trained in textbook approaches, not in the scientific method. The less creative, the better the doctor. That's precisely what evidence-based medicine is, and it's what allowed us to leave the Dark Ages of paternalistic medicine. > The big health co's do not like doctors doing this because it takes time and is expensive. So, they tie their hands because they only want the doctors to focus on the easiest fixes that solve the most problems. We tie our own hands. With guidelines. Yes, the system is tuned to a limited investment of time and money in each patient for distributive justice reasons, but guidelines are (mostly) there to discourage crackpot medicine.
dekhn 5y ago
chloramine is replacing chlorine in municipal treatment. I'd be amazed if somebody was really sensitive to chlorine, to the point where typical municipal water caused rashes. After all, all of us contain 80grams of it and consume 500mg/day.
celdon25 5y ago
> Take a substance like chlorine. It is put into municipal drinking water because it kills things - things that can kill us or harm us. It is credited with making municipal water a lot safer. > It has been shown to be safe and effective for most people. And yet it wasn't until I was well into my 30s that I learned that not cleaning your shower head often enough or thorough enough can block the effects of the chlorine and spray aerosolized bacteria everywhere. It doesn't matter what kind of medicine you have when there are plenty of problems that would be prevented if not for the obscene asymmetry of information and poor cultural health habits. But as long as big tech continues to believe that steering people toward political propaganda is what's best for society, that will be the narrative of the day rather than real progress.
- manwe150 5y ago
  And I was today years old (not much younger than you were)

qnsi 5y ago

Medical industry is corruption prone. Another example: Industry sponsored conferences. Doctors are invited to a 3 day 4 star hotel stay to attend a conference. Technically, talks that are paid shilling are specially marked as sponsored talks. But are the other doctors, who are getting paid to speak there 100% independent? Will you be a 100% independent doctor after getting free presents? Maybe you will. But industry probably found its profitable if they spent millions on these kinds of conferences. Also lie repeated multiple times makes you believe in, there are psychological studies. So when you spend hour listening how drug X is gold, you might start believing its gold

Enginerrrd 5y ago
>Will you be a 100% independent doctor after getting free presents? >Maybe you will. But industry probably found its profitable if they spent millions on these kinds of conferences. So... theoretically the industry putting on the conference is also pushing the state of the art pharmaceutically and believe that they have an improvement on the existing standard of care. I don't think it's got anything to do with the presents as much as the message.
- pessimizer 5y ago
  Nah. Speakers fees are a way to launder direct contributions. There was at least one famous case where the "conference" was done in one of the speakers' living rooms. Imagine getting a speaking fee for speaking extemporaneously in a Florida hotel room with two pharma reps and one other doctor five minutes before you all leave for golf.
  - nobody9999 5y ago
    >Nah. Speakers fees are a way to launder direct contributions The comparison to money laundering is specious, IMHO. In the US, at least, there is a database[0] of any value (including monetary and in-kind) exchange between doctors and medical companies. As such, there is no "laundering" at all. All moneys and in-kind (hotel rooms, meals, swag, etc.) payments are documented and detailed. Is there an incentive for doctors to favor a particular pharma or medical device company based on those payments? Perhaps. But since (again, at least in the US) such payments are documented and publicly available, it isn't some sort of secret set of payments designed to surreptitiously co-opt doctors. Sure, some payments (my brother, a physician, received ~US$20,000 in 2020, mostly (~$13,000) from a single consulting fee. Is my brother favoring the company who paid him $13,000 in 2020? Maybe. But if and only if their products/devices have clear beneficial effects over other products/devices. What's more, that $20,000 (aside from the consulting fee, it was food/beverage and other in-kind stuff) is a small fraction of his annual income and doesn't make a significant difference in his quality of life. As such, while there may certainly be doctors who are co-opted/corrupted by medical companies, assuming that the majority of doctors are swayed by such things is iffy at best. [0] https://openpaymentsdata.cms.gov/ https://openpaymentsdata.cms.gov/

twofornone 5y ago

>Scientific progress is thwarted by the ownership of data and knowledge because industry suppresses negative trial results, fails to report adverse events, and does not share raw data with the academic research community. Not to derail the thread but this nicely encapsulates my reservations toward the COVID vaccines, and why I think mandates are egregiously unethical. The raw clinical trial data is still effectively a secret between pharma and the FDA, and both are under immense financial/political pressure to release a "safe and effective" product, not to mention the incestuous relationship between the ostensibly independent FDA and big pharma.

sebmellen 5y ago
I find it very discomfiting that anyone suspicious of the COVID-19 vaccine (by that I mean the COVID vaccine/spike protein in particular) is branded as anti-vax. I say this as a vaccinated person who caught a heavy bout of COVID. I suppose I'm grateful for the vaccine, but mandates scare me.
- specialist 5y ago
  Do you school-aged children? Pets?
- tricky777 5y ago
  as chistians say, "turn the other cheek". (labeling did not made much harm, dont get too ivested in that) reality is, that we can't have policy/public sentiment that is nuanced. (I heard it breaks the timecontinuum) Find the silver lining and move on. I'm just glad that, a lot of people are in pro-vaccine side overall. While in your case it is a bit of mob mentality (mostly appeal to authority + optimism + jugde_dissagreeing_side), in the long run "pro vaccines" is (even if only on average) leaps and bounds and couple of parsecs better than "no vaccines".
- deleted 5y ago
  [deleted]
simonh 5y ago
To a point that's understandable, based just on trials it's reasonable to be skeptical of the results. However once the vaccines have been in mass distribution and there is copious evidence of their efficacy and the extreme rarity of adverse affects, I don't think it's reasonable anymore. The mandates only came in once enough of the population had been inoculated to make a requirement possible. At that point the initial trials are a footnote compared to the results from mass vaccinations. IMHO it's just not reasonable for that objection to carry any weight. There may be other reasons, of course.

cosmotic 5y ago

This should not be considered evidence that other quack medicines have any superiority over modern medicine.

trulyme 5y ago
True. It is just evidence that sometimes modern medicine doesn't have any superiority over quack medicines either.

smohare 5y ago

One of the issues I’ve seen with EBP is that it’s often not necessarily science based. That is to say, prior plausibility derived from basic science isn’t fully incorporated, ergo undo weight is given to noise. You see with in almost any study of pseudoscientific modalities, where actual efficacy would require a drastic reformation of our fundamental, very well tested models of the natural world. Why should a particularly strong one-off result for my made up pain treatment be afforded the same stature as that of an pharmacological agent with clear physiological pathways?

rscho 5y ago
Evidence-based medicine does incorporate prior evidence for causal evaluation. Check out the "Bradford criteria". They're the basis of causal evaluation in epidemiology.

chicob 5y ago

After reading this article I find only one problem with it, and that is the title. The problem the authors have so perfectly characterized has nothing to do with the emphasis on evidence that medicine requires, but on the distortion imposed on the system by large pharmaceutical companies. The reality of the obstacles standing in the way of rigorous testing for drug or therapy effectiveness does not make evidence-based medicine an illusion. In fact, the authors actually make a strong argument in favor of evidence-based medicine, by stating the need to acknowledge these biases. And that the reason for all these biases is greed.

pmoriarty 5y ago
The article shows how what passes for evidence based medicine is an illusion. The "evidence" is biased and corrupted by corporations, as are many of the people who evaluate it, set the standards, and make decisions.
xmprt 5y ago
I think you're misunderstanding the title. It's not that evidence based medicine has problems. It's that there are problems with how evidence based medicine is currently practiced which gives it the appearance of evidence based medicine while actually not being anything close, that is, an illusion.
- chicob 5y ago
  Well, yes. My point was simply that the title misrepresents the article, because it is ambiguous. "The illusion of evidence based medicine" can be interpreted as a statement of how the aim of evidence based medicine is illusory. The short introductory paragraph, "Evidence based medicine has been corrupted by corporate interests [etc]" furthers this idea, having a somewhat definite tone, as if this corruption were fatal to any proper scientific inquiry. The rest of the article dissipates this confusion rather quickly. Stepping back a little, how would one react to a title such as "The illusion of medicine", making the same general argument?
ouid 5y ago
Stop calling it greed. It's corruption and regulatory capture. Both of these, unlike greed, are problems that can be solved.

rscho 5y ago

Evidence-based medicine, more than the indeed important issues raised in the article, is mostly impaired by the absolutely abysmal average quality of clinical research papers. The absolutism of higher-ups combined with their unrivalled (in the scientific world) statistical ignorance is hurting science more than anything else. And there's a reason to that: MDs are not scientists. We're highly-skilled workers, and more of the blue-collar kind.

dr_dshiv 5y ago
Disagree. Think your standards are too high.
- throwoutway 5y ago
  Your standards are too low. Human lives are at stake in medicine. Do you want a doctor treating you when they are misinformed by fake science?
  - dr_dshiv 5y ago
    I want doctors to not treat studies like “science sez” and instead recognize that “Science is messy”. They should use their training and experience to heal, not the latest publication.
  - webmaven 5y ago
    I'm not sure that's any worse than a doctor misunderstanding real science. Note that the root cause (unfamiliarity with the tools of science, especially probability and statistics) is largely the same.
    - rscho 5y ago
      > I'm not sure that's any worse than a doctor misunderstanding real science It is worse. A highly experienced practitioner who would've done the right thing might get led astray by new bogus research. Precisely because most practitioners don't have the ability to judge scientific work on their own.

slibhb 5y ago

I think profit motive and regulatory capture are real problems but I don't think they're the only problems and maybe not the biggest. For example, the FDA is generally agreed to have dropped the ball by dragging its feet on approving covid tests. I don't think profit motive or regulatory capture explain that. What does? More transparency (as the authors recommend) sounds great but giving more power to regulatory bodies is no panacea.

mikekij 5y ago
> For example, the FDA is generally agreed to have dropped the ball by dragging its feet on approving covid tests. I don't think profit motive or regulatory capture explain that. What does? I'm no fan of over-regulation, but the FDA finds itself in an impossible situation: approve tests quickly to slow the spread of Covid, only to later find that the tests don't work, or test the products rigorously, while patients suffer. It's a very difficult balance to strike, and I would argue they've done as good a job as one could hope.
ouid 5y ago
Big Pharma derives its oligopoly from the barriers to entry that the FDA imposes. Presumably they take every opportunity to lobby for more rigorous extensive testing. Not because they want to pay more to run tests, but because they want their smaller competitors to pay more to run studies for drugs that might compete. This is, of course, not an explanation for how they have variously captured the components of the FDA, but it does explain how such a capture is an explanation for foot-dragging.

yes_man 5y ago

Another great argument I have heard against EBM (or maybe rather against being religious about it) is that the studies and papers held to highest regard are made in circumstances that do not apply to most of healthcare: the studies might be done in the best hospitals like Johns Hopkins, with practically limitless human resources for the study, and very carefully selected patient groups. In a normal hospital situation you won’t have the time or the resources to dedicate the same care for each patient, and the patients might have other conditions (which would rule you out of the big studies). Still, as a layman who is not a doctor, evidence based medicine sounds like a good idea, but as with everything, taking it to extreme is probably harmful

caycep 5y ago

granted, there is the day-to-day human physician "bayesian" evidence-based-medicine...for better or worse...

TeeMassive 5y ago

Here in Québec the vaccine passport and the curfew was touted as the pinnacle of "following the science"(tm). Anyone contradicting their usefulness as effective sanitary measures were accused of "spreading misinformation"(tm). The curfew was implemented twice. The vaccine passport was only recently removed. Turns out, one FOIA request later, that they never consulted the data scientists of the public health institute. They made it all up.

tensor 5y ago

Drug company bias is definitely a well known issue. But there are even bigger issues with science based medicine. * MDs are not PhDs for one, though people often equate the two. * Things like surgery is in fact hard to test. There are ethical issues around it. E.g. are you really going to do a shame surgery "for science" if you think there is a chance it might save the patients life? On the flip side, surgery has a lot of risk, and performing surgery that doesn't work exposes patients to unnecessary risk. * There are entire fields where pseudo-science is common. Notably a lot of physical therapy and chiropracty is not science based.

willhinsa 5y ago
On the difficulty of testing lifesaving treatments, there's a great paper on this topic: "Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials" [0] It has some great gems like: - No randomised controlled trials of parachute use have been undertaken - The basis for parachute use is purely observational, and its apparent efficacy could potentially be explained by a “healthy cohort” effect - Individuals who insist that all interventions need to be validated by a randomised controlled trial need to come down to earth with a bump [0] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC300808/
- pmyteh 5y ago
  There was, subsequently, published a Christmas BMJ article reporting the results of an RCT of parachute use jumping from planes. Apart from a few minor confounding problems with selecting the participants, it's very good science! Found no evidence of benefit, sadly... https://www.bmj.com/content/363/bmj.k5094 https://www.bmj.com/content/363/bmj.k5094
  - willhinsa 5y ago
    So funny! > RESULTS Parachute use did not significantly reduce death or major injury (0% for parachute v 0% for control; P>0.9). This finding was consistent across multiple subgroups. Compared with individuals screened but not enrolled, participants included in the study were on aircraft at significantly lower altitude (mean of 0.6 m for participants v mean of 9146 m for non-participants; P<0.001) and lower velocity (mean of 0 km/h v mean of 800 km/h; P<0.001). > CONCLUSIONS Parachute use did not reduce death or major traumatic injury when jumping from aircraft in the first randomized evaluation of this intervention. However, the trial was only able to enroll participants on small stationary aircraft on the ground, suggesting cautious extrapolation to high altitude jumps. When beliefs regarding the effectiveness of an intervention exist in the community, randomized trials might selectively enroll individuals with a lower perceived likelihood of benefit, thus diminishing the applicability of the results to clinical practice.

medymed 5y ago

Science marches on one funeral at a time, medicine marches on one expired patent at a time.

rscho 5y ago
No. For medicine, both are true because funerals of both patients and professors are a signal for progress.

deleted 5y ago

[deleted]

tsoukase 5y ago

EBM suggests that swimming in a heated swimming pool (controlled studies) can be the same as swimming in open sea/ocean (applying the result in any possible case)

FabHK 5y ago
The distinction you allude to is so well known and understood that there are not only papers about it, but also special words to distinguish them: efficacy (performance of an intervention under ideal and controlled circumstances) and effectiveness (performance under ‘real-world' conditions). https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3726789/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3726789/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3912314/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3912314/
- tsoukase 5y ago
  The problem is there is no EBM without p-values and there is no real world without Bayesian reasoning. So, the two are fundamentally disconnected

deleted 5y ago

[deleted]

rafaelero 5y ago

There is one simple solution: replication. Stop complaining and start replicating.

Terry_Roll 5y ago

As someone who recently complained to the UK Royal College of Veterinary Surgeons and got palmed off because the UK legislation doesnt quantify "reasonable care", the whole system is one giant authoritarian money making scam. I dont know what professionals are being taught in med/vet/law school now a days, but technology is increasingly demonstrating that professionals could be out of date by the end of their course! So that problem of trying to stay current in IT is spreading out into other professions and I dont see them tackling this problem like the IT sector has/does.

throwawaylinux 5y ago
> As someone who recently complained to the UK Royal College of Veterinary Surgeons and got palmed off because the UK legislation doesnt quantify "reasonable care", Pretty hard to quantify. "Reasonable" is a widely used concept in the law for this reason. What were you expecting them to do? How would you define "reasonable care", exactly? > I dont know what professionals are being taught in med/vet/law school now a days, but technology is increasingly demonstrating that professionals could be out of date by the end of their course! Surely nobody ever thought otherwise. Doctors aren't superhumans, none will ever know everything and there are good and bad ones. Exactly the same as any other person in any other field. > So that problem of trying to stay current in IT is spreading out into other professions and I dont see them tackling this problem like the IT sector has/does. The IT sector does? How? I thought IT was pretty mediocre at it. And the challenge of staying current with new developments predates the computer by a few millennia doesn't it? This isn't something new to IT. Have you actually looked to see how the medical field tackles this problem? I'm not in the medical field but some family members are. The amount of training, certifications, exams, etc they are required to do is staggering. Far more than any software developer I've ever known.
- Terry_Roll 5y ago
  All fair points, but medicine is highly cautious, and most would prefer to go with the larger entities opinions/research. So Merck (https://en.wikipedia.org/wiki/Merck_Veterinary_Manual https://en.wikipedia.org/wiki/Merck_Veterinary_Manual) have a massive influence which introduces lag, although some do read pubmed to access more up to date studies, but (google) scholar can be a better search engine to use over pubmed. I'm also aware of the Eli Pariser Filter Bubble which will affect what the vets see and what I see. I also have to do repeated searches sometimes over a few weeks, months or years to see what the search results are throwing up which it didnt previously. > Have you actually looked to see how the medical field tackles this problem? Yeah and they dont tackle the problem to my satisfaction instead using older technology/medicine. No one wants to put their neck on the line because they dont want to look at human studies where biological similarities exist, or different species are used, like the good old rat. So some in-vitro studies exist because its impossible to study in-vivo, the tech just doesnt exist for in-vivo and that is an area where they are highly cautious and there does seem to be a general dislike for using non patentable chemicals!

sammalloy 5y ago

One thing that has always stuck with me in regards to this topic is the lack of medical research in areas that need it but don’t get it because there isn’t any money in it. One area that comes to mind is feline urinary tract disease. Many years ago, I had a cat who kept coming down with this, and the vet(s) were at a total loss as to how this kept happening. So I decided to open up the literature and plough into it as a layman with no knowledge about animal medicine. Within the space of a single hour I quickly learned that many of the most common diseases facing animals have very little research behind them and a lot of unknowns. Which brings me back to one of the main points of the author. If the neoliberal approach to research is only going to focus on what is profitable to treat, then medical science as a whole has backed itself into a dark corner.

dayofthedaleks 5y ago
I'm sorry but veterinary medicine is an entirely different problem domain with little overlap beyond capitalism and mammalian biology.
- StreamBright 5y ago
  No it is not. Maybe you have heard about animal testing before. If what you say is true we hadn't been using animal testing.
  - wins32767 5y ago
    Animal testing is to test human drugs/treatments/cosmetics on animals to see if they are safe enough to test on humans.
    - caddemon 5y ago
      A lot of basic research is also done in animals, it's a pretty important part of the process that led to many drugs being tested in the first place. But it's true an understanding of the animal is usually treated as an unimportant tangent. Like you might introduce a mutation into a mouse gene that is analogous to a mutation thought to be relevant in humans and then look at how that affects some other biomarker of interest in the mouse. But you're not going to be studying some issue that arises naturally in mice. Hell you're only using the mouse because it's an established lab model, you don't give a shit about mice. Also, all the mice are hella inbred and grow up in a sterile cage, so they're hardly even reflective of IRL mice. So even if someone did want to fund lots of research on mouse veterinary science, it would still be a long road (technically and culturally) to get that integrated with basic medical research. I'm not saying this is the way it should be, but it's the way it is.

DarkmSparks 5y ago

Massive, major flaw in their argument. "without seeing the raw data" is not true. At least in the UK. All raw study data, published and un published must be provided before a medicine will be reimbursed. It is reviewed by teams who do their utmost to prove the med is worthless. Same for the FDA afaik. All the data is generally "commercial in confidence" so any old twitteraty can not download and misinterpret it. All trials must be registered on Clinicaltrials.gov or they cannot be used as evidence. Publication bias is a well know issue, resolved for more than a decade. No idea how this article made it into the BMJ, smells like click bait.

rscho 5y ago
> Publication bias is a well know issue, resolved for more than a decade. How is that resolved? This is not what I see in my meta-analyses.
- DarkmSparks 5y ago
  requiring registering with clinicaltrials.gov and regulators doing their own analysis on all the data. Managed entry agreements and conditional reimbursement (dont pay for patients that dont do better than the average of current practice) Requirements for "real world data" collection and submission post approval. A whole host of other mechanisms I cant remember off the top of my head. Also "evidence based medicine" doesnt mean the evidence is infallible (e.g. rare disease meds have a huge issue with collecting any meaningful data) - it just means actually using evidence rather than just taking the industries and acedemias word for it that it works like the old days.
  - rscho 5y ago
    From personal experience, I think you are massively overestimating the reach and effectiveness of regulations.
    - DarkmSparks 5y ago
      I've worked with nice. several of my students still do. We've kicked out of the NHS quite a few meds the industry and academics were wrong about, based on the evidence, not what the industry/academics said before they were shown to be not correct. The whole posit of this article is wrong. "acedemics said so" is not evidence based medicine, it is one small facit of it, and the author matters, no one trusts "sponsored by" papers, NIHR is constantly publishing counter evidence. Some of that even ends up in court (e.g. Avastin) https://www.reuters.com/article/us-novartis-bayer-britain-idUSKCN1M12EC https://www.reuters.com/article/us-novartis-bayer-britain-id...

threads2 5y ago

There's so much money in cancer research - when I was a pharmacology/genetics researcher I was told to try to tie everything back to cancer for research papers and grant proposals. And the oncology lab next door had the best equipment. We always had to borrow their stuff. Idk maybe this is a good argument against private charity. Not to say that cancer research isn't important but from what I could tell, cancer researchers had an embarrassment of riches.

conorh 5y ago

My wife and I were just talking about this yesterday. She is an extremely specialized surgeon and she is horrified by the academic publishing in her area of expertise. It is often utterly wrong and conflicts directly with her experience. I'm not sure she'd agree with the reasons mentioned in this article here as the main issue - although we have personally seen less ethically motivated doctors publishing research to help line their pockets via the industry. In her opinion the main issue is that academic doctors HAVE to publish in order to advance their careers. So you end up with a never ending stream of terrible research, churned out by doctors that are not experts in doing research and that actually spend most of their time doing other things. This happens even at the best centers; at least in her area of expertise. I don't know what the solution is, but I do know that whenever she has needed treatment for herself she generally ignores most of the research and talks to the most experienced surgeon she can find.

rscho 5y ago
This is all true. Welcome to the real world. > academic doctors HAVE to publish in order to advance their careers Indeed. But the issue is what criteria will you use for promotion if not that? Don't even think of evaluating clinical results, that's even worse than what we have. > This happens even at the best centers This happens ESPECIALLY at the best centers. Because centers are in good part ranked according to citation counts. > I don't know what the solution is, but I do know that whenever she has needed treatment for herself she generally ignores most of the research and talks to the most experienced surgeon she can find. After 10+ years in the system, I'm in the same boat. And although I've given it a good thought, I don't see a solution either.
- derbOac 5y ago
  The truth this article touches on is that it's not even really about citation counts anymore, it's about dollars. Via drug companies, private grants, federal grants, any sorts of money coming in from outside the university or hospital. I've sat on promotion meetings where people aren't publishing, but are bringing in 2 million dollar grants and "it's all ok". This incentive structure distorts research. What to do about it? I'm not exactly sure but having university administrations who see research as a means to make money, rather than an end in itself, is not the way.

jrapdx3 5y ago

Over the years I've practiced medicine the idea of "evidence-based" care has had its ups and downs. Part of the problem lies in the question of what is "evidence". From what I've seen, heard and read, the question is broader than pharmaceutical research though that's certainly an important element. The evidence literature spans making diagnoses, applying drug and non-drug therapies, deciding what is a successful outcome of treatment, etc. What I see is that clinicians are quite willing to use relevant and effective evidence-based methods. One problems applying evidence to real-world practice arises out of the aggregate bases of evidence. Even if evidence-based approaches could work in typical cases, many patients vary enough from norms that following evidence-based recommendations doesn't work. Furthermore it shouldn't be surprising when low-quality evidence gains corporate or governmental backing. When such promoted "standards" defy clinical observation it tarnishes the idea of evidence-based practice. An example is fairly recent recommendations that men over 70 shouldn't have routine PSA lab tests. Urologists will tell you about the hundreds of men they see with early stage prostate cancer detected by periodic PSA tests. Following the "evidence" would result in far more deaths and disability so it's ignored. As the article points out corruption of "evidence-based medicine" has destroyed much of the promise the idea once held. Insurers, pharma houses, governments contribute to abuse of evidence to serve their own agendas. No doubt some fraction of practitioners have added to problems and that needs to be halted. But IMO regulators are quick to point fingers at all clinicians for failures of health care systems. Rather most providers would prefer to avoid corrupt "evidence" whereas real evidence is welcome to workers toiling in the trenches.

wolverine876 5y ago
Thanks for sharing your perspective. > An example is fairly recent recommendations that men over 70 shouldn't have routine PSA lab tests. Urologists will tell you about the hundreds of men they see with early stage prostate cancer detected by periodic PSA tests. Following the "evidence" would result in far more deaths and disability so it's ignored. Isn't that how any system works - the rules provide guidance (with some teeth), and the people working hands-on apply their judgment? I recently was reading about the US Army's process. Many think of the military as highly regimented, but the US Army encourages - in fact, requires - subordinates to ignore orders that don't apply to their current situation; they are required to fulfill the 'commander's intent'. Only broken, authoritarian or totalitarian systems try to apply the theoretical centralized control to practice, where it is so rigid that it is bound to fail at scale (like another army in the news, apparently). What central authority is intelligent enough, has the information for, and the vision for anticipating every situation now and in the future, knowing the best solution, and writing rules for it?

faangiq 5y ago

We all live inside our own illusions.

refurb 5y ago

It would appear that the authors are selling a book on this topic, which explains the BMJ article. And I'm not sure their premise is all that accurate. If they think the FDA is in the pocket of pharmaceutical companies, that runs entirely counter to my experience. I can recall plenty of times where we'd suggest a change to a filing and the FDA response was "nope, changes denied". If you think the low level reviewers of submissions are secretly pulling the strings for Pfizer, you'd be wrong. And in terms of the quality of publications - this is nothing new. There has always been a robustness to clinical trials. That's why the NCCN guidelines for cancer treatment actually rate the clinical evidence from Category 1 (strongest) to Category 3 (no evidence whatsoever).

t0bia_s 5y ago

https://news.ycombinator.com/item?id=30761070#30796020 https://news.ycombinator.com/item?id=30761070#30796020 I put that article three days ago here, isn't there any moderation for double submissions?

mola 5y ago

This is how dark ages start. Actual knowledge is watered down by corporate nonsense. Society loses confidence in the scientific establishment. The organizing coordinate that took center stage in the age of enlightment will be lost. Society still need some way to organize its cognition. Power and superstition will take center stage again.

WA 5y ago
This is how conspiracy theories start. Take a thing that has some truth to it and extrapolate it all the way up, dismissing that the whole system is probably more complex.

IG_Semmelweiss 5y ago

The problem is misaligned incentives. You need to go after the "research trolls". Same as you would with patent trolls. Hit them where their money is. Pass a law that anyone anyone ultimately funding or profiting off research can be sued by the public for double damages x cost of the research if : - paper won't replicate - paper sponsors & costs are not disclosed. - paper does not include instructions to make data requests - researcher does not release data within 30 days of request. - paper publisher (whomever that could be, even faculty) failing to flag prominently within 30 days, any paper that has lost in court. So if a university employs 1 researcher at X salary, and time spent on research is Y, the university can be sued for Y * X * 2 by Joe Schmo. Maybe this would be small claims court. Same would apply to any research created by big AG, Pharma, etc. If researcher uploads paper in his own personal website and forgets to flag it as flawed, he now pays the price too. It would be necessary to cap disgorgement at 10x initial research cost. The point is to let the crowd hunt down garbage research, not bankrupt anyone for honest mistakes. I would also default to making the defense be rewarded by 2x incurred legal costs if they win case to ensure lawsuits have some merit. This would immunize students if they are not paid to make research (but not PHDs that live off stipends), or random people doing research, out of goodwill. Professionals would be very careful to publish garbage lest be fired from their jobs, and companies & researchers would think twice before pushing propaganda. No more research mills of BS.

yolo69420 5y ago

science is a pretty long term process. in practice results and consensus stabilizies after around 50 years or so. the reason it takes so long is that the turnaround time for a single result in science is half a year to a year. imagine discussing something with someone and every sentence you speak could only be uttered after a year. articles like OP always hinge on some short or medium understanding of science. they try to point out that the whole approach is wrong based on the intermediate results of the process. OP itself is part of this process and the reason why the results will be better and more stabilized a couple of decades from now.

chithanh 5y ago
> consensus stabilizies after around 50 years or so One other reason for science being such a long-term process is Planck's Principle: https://en.wikipedia.org/wiki/Planck%27s_principle https://en.wikipedia.org/wiki/Planck%27s_principle Basically, scientific innovations spread by the old orthodoxy retiring/dying and replaced by successors who are familiarized with the new ideas (and then proceed to form the new orthodoxy).
dgb23 5y ago
But there is also the social problem. Incentives and power structures matter, sometimes to a comical degree. Openness and collaboration are fundamental to progress.

YeGoblynQueenne 5y ago

I have no idea if any of this is true, but it rings true and it reminds me of similar distortions of the integrity of scientific inquiry by the interest of industry, in my own field of study: artificial intelligence and machine learning. And perhaps that's no surprise. It's the same universities that produce both kinds of research (and researcher) after all.

jokoon 5y ago

In france we have this research director, Didier Raoult, who pushed for the use of Chloroquine as a treatment against COVID. He was the head of a research center dedicated to tropical diseases. This guy literally argued that the smaller the test sample, the better the result. This guy lost his job of director (not to mention investigation of research fraud), but he mades it very difficult for uneducated people to trust the vaccine. Not to mention that I dated a biology scientist and teacher who did not want to get vaccinated. I heard 2 other scientist lost their jobs in a research lab in my city because of not wanting to get vaccinated. So no matter what people will tell me, it seems that science is rife with people who should not have their credentials, despite high level education, universities, "merit based" grades, etc. It reinforce my cynicism, because I had some scientific education, but I dropped and never reached a degree. It's what Pierre Bourdieu talks about, that degrees are a way of maintaining social classes. It's exactly what you hear in the movie "Good will hunting", that a degree is just a piece of paper, while people can just educate themselves by reading science books.

user3939382 5y ago

I've been more or less of this opinion since watching this talk years ago by Dr. Jason Fung: https://www.youtube.com/watch?v=z6IO2DZjOkY https://www.youtube.com/watch?v=z6IO2DZjOkY

phtrivier 5y ago

Cursory reading is : EBM would be great, but we can't trust the "E" in EBM because people selling the "M" in EBM have a vesting interest in producing fake data to justify selling more to the institutions hoping to do the "B". It seems like the author think more "independant" parties would help (regulators ? white-label labs to do the testing instead of pharma labs ?) All I can tell from the last two years is that I doubt people would really trust a "government agency" (or, anyone "paid by the taxpayers") more than "the corporate" lab. (Case in point: the EU agencies that banned or suspended or restricted usage of some COVID-19 vaccines, at various points in the last year, are still mistrusted on the ground that they did not ban or suspend _one_ of the vaccines - which happens to be the most popular - hence "they must be in the pocket of lab X".)

theknocker 5y ago