4 ms·

RCTs are high quality, because they are done in a controlled setting, testing against a control group, testing the null hypothesis. RCTs are able to show causal

RCTs are high quality, because they are done in a controlled setting, testing against a control group, testing the null hypothesis. RCTs are able to show causality. The problem with RCTs is that they are very expensive, since it involves keeping people in a clinic, so usually RCTs are short term and can only afford to look at surrogate markers. However most often than not, surrogate markers are enough. Also RCTs tend to coroborate the findings from observational studies in about 70% of the cases. Also for really hot topics, like diabetes management, we do have year-long RCTs too. And drawing conclusions from systematic reviews and meta analyses of RCTs, well, that's basically the best we can do. For assessing the strength of available evidence, I recommend reading the following: https://examine.news/how-to-read-a-study https://examine.news/how-to-read-a-study

waterhouse 6y ago
> The problem with RCTs is that they are very expensive, since it involves keeping people in a clinic This may be a noob question, but why? For something like a vitamin supplement, can't you just give the study participants a box of their 50 possibly-placebo pills and require them to take one per day? If you're worried about compliance, can't you require them to write down the time each day when they take the pill—or even have the pills in a one-pill-per-box calendar arrangement and require them to take a photo with their phone each day? (Doesn't stop malicious noncompliance, but it should cover laziness/forgetting.)
- gojomo 6y ago
  They're not all done in a completely-controlled clinic - indeed the 25,000 person study highlighted here, with followup after around 5 years, of course had the subjects going about their daily lives except during original enrollment & occasional check-ins. But when outcomes are finer-grained than "did the subject die or get diagnosed with a major new disease", or short-term, or need high compliance & regular monitoring, RCTs can become a lot more intense & expensive.
- tgb 6y ago
  There are "smart" pill bottles for that purpose, that record what time they are opened.
JPLeRouzic 6y ago
> RCTs are high quality, because they are done in a controlled setting I have a personal interest in Amyotrophic Lateral Sclerosis, so I am reading patients' Internet forums since two years. Several patients report participating in trials but they also tell that they take other drugs without telling doctors (they do not want to be expelled from the trial). Those drugs could be whatever they fancy, and they change often: A week seems too long for them, if there is no improvement. I expect the situation being the same in every life challenging condition, so I doubt that RCTs are of high quality. Moreover even the pharmaceutical industry seems to be not sure: Sometimes they are repetitively testing the same drug (for example Memantine in ALS): https://clinicaltrials.gov/ct2/results?term=Memantine&cond=Amyotrophic+Lateral+Sclerosis https://clinicaltrials.gov/ct2/results?term=Memantine&cond=A...
- bad_user 6y ago
  Those are anecdotes, the question is how often does it happen? Sure, it's possible, which is why we have reviews aggregating the findings of multiple studies. The results of one study could be an anomaly, which is why replication is important in science. But when those results get replicated in tens or hundreds of such studies, we can start trusting those findings (although when surrogate markers are involved, their interpretation can be up for debate). Also when people say "high quality", it's important to understand what it refers to. It means the authors can show that the study was designed for the declared primary outcomes. It means that adherence was ensured, versus an observational study relying on questionnaires. And it means a control group was used for comparison, which was randomized, thus the study preferably showing a dose dependent response. Having a control group BTW minimizes the impact of the subjects not adhering to the program in the way that you describe, since you'd have rebels in both camps.
  - throwaway6000 6y ago
    People who boost their vitamin D levels with supplements reduce their risk of respiratory tract infections, such as the flu, by up to 12%, according to a new systematic review and meta-analysis study of 25 randomized controlled trial (RCT) studies published in The BMJ. https://bcmj.org/news/vitamin-d-effective-reducing-flu-and-colds-study?inline=true https://bcmj.org/news/vitamin-d-effective-reducing-flu-and-c... Harvard Gazette: Study confirms vitamin D protects against colds and flu https://news.harvard.edu/gazette/story/2017/02/study-confirms-vitamin-d-protects-against-cold-and-flu/ https://news.harvard.edu/gazette/story/2017/02/study-confirm... WebMD: Vitamin D May Cut Risk of Flu https://www.webmd.com/cold-and-flu/news/20100616/vitamin-d-may-cut-risk-of-flu#1 https://www.webmd.com/cold-and-flu/news/20100616/vitamin-d-m... Vitamin D protects against colds and flu, finds major global study https://www.sciencedaily.com/releases/2017/02/170216110002.htm https://www.sciencedaily.com/releases/2017/02/170216110002.h...

waterhouse 6y ago

> The problem with RCTs is that they are very expensive, since it involves keeping people in a clinic This may be a noob question, but why? For something like a vitamin supplement, can't you just give the study participants a box of their 50 possibly-placebo pills and require them to take one per day? If you're worried about compliance, can't you require them to write down the time each day when they take the pill—or even have the pills in a one-pill-per-box calendar arrangement and require them to take a photo with their phone each day? (Doesn't stop malicious noncompliance, but it should cover laziness/forgetting.)

gojomo 6y ago
They're not all done in a completely-controlled clinic - indeed the 25,000 person study highlighted here, with followup after around 5 years, of course had the subjects going about their daily lives except during original enrollment & occasional check-ins. But when outcomes are finer-grained than "did the subject die or get diagnosed with a major new disease", or short-term, or need high compliance & regular monitoring, RCTs can become a lot more intense & expensive.
tgb 6y ago
There are "smart" pill bottles for that purpose, that record what time they are opened.

JPLeRouzic 6y ago

> RCTs are high quality, because they are done in a controlled setting I have a personal interest in Amyotrophic Lateral Sclerosis, so I am reading patients' Internet forums since two years. Several patients report participating in trials but they also tell that they take other drugs without telling doctors (they do not want to be expelled from the trial). Those drugs could be whatever they fancy, and they change often: A week seems too long for them, if there is no improvement. I expect the situation being the same in every life challenging condition, so I doubt that RCTs are of high quality. Moreover even the pharmaceutical industry seems to be not sure: Sometimes they are repetitively testing the same drug (for example Memantine in ALS): https://clinicaltrials.gov/ct2/results?term=Memantine&cond=Amyotrophic+Lateral+Sclerosis https://clinicaltrials.gov/ct2/results?term=Memantine&cond=A...

bad_user 6y ago
Those are anecdotes, the question is how often does it happen? Sure, it's possible, which is why we have reviews aggregating the findings of multiple studies. The results of one study could be an anomaly, which is why replication is important in science. But when those results get replicated in tens or hundreds of such studies, we can start trusting those findings (although when surrogate markers are involved, their interpretation can be up for debate). Also when people say "high quality", it's important to understand what it refers to. It means the authors can show that the study was designed for the declared primary outcomes. It means that adherence was ensured, versus an observational study relying on questionnaires. And it means a control group was used for comparison, which was randomized, thus the study preferably showing a dose dependent response. Having a control group BTW minimizes the impact of the subjects not adhering to the program in the way that you describe, since you'd have rebels in both camps.
- throwaway6000 6y ago
  People who boost their vitamin D levels with supplements reduce their risk of respiratory tract infections, such as the flu, by up to 12%, according to a new systematic review and meta-analysis study of 25 randomized controlled trial (RCT) studies published in The BMJ. https://bcmj.org/news/vitamin-d-effective-reducing-flu-and-colds-study?inline=true https://bcmj.org/news/vitamin-d-effective-reducing-flu-and-c... Harvard Gazette: Study confirms vitamin D protects against colds and flu https://news.harvard.edu/gazette/story/2017/02/study-confirms-vitamin-d-protects-against-cold-and-flu/ https://news.harvard.edu/gazette/story/2017/02/study-confirm... WebMD: Vitamin D May Cut Risk of Flu https://www.webmd.com/cold-and-flu/news/20100616/vitamin-d-may-cut-risk-of-flu#1 https://www.webmd.com/cold-and-flu/news/20100616/vitamin-d-m... Vitamin D protects against colds and flu, finds major global study https://www.sciencedaily.com/releases/2017/02/170216110002.htm https://www.sciencedaily.com/releases/2017/02/170216110002.h...