3 ms·

"Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial" https://www.bmj.com/content/363/bmj.k5094 https://www

"Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial" https://www.bmj.com/content/363/bmj.k5094 https://www.bmj.com/content/363/bmj.k5094 But seriously: this is a recognized problem in medicine and there's already a widely used solution. Whenever you're doing trials of an intervention for a condition which already has an accepted treatment, you run a trial to compare your new intervention to that, and see if your test group has better outcomes. After all, the question shouldn't be whether your treatment is effective; it's whether it's better than existing treatments. Trials against a placebo have a purpose, but they aren't the only way to run a trial.

saalweachter 1y ago
It also helps to actually go back and look at in terms of "rejecting the null hypothesis". If you're talking about a treatment for The Common Cold, the null hypothesis is "the subject got better after awhile because people get better after awhile", and you can't disprove that's what's happening without a very rigorous study with a well designed control. If you're talking about "here's some robot eyes that cure blindness", you don't really need a control group to test if it works. The null hypothesis is they're blind; you just need to demonstrate they can see to disprove the null hypothesis and prove efficacy.
standardUser 1y ago
And years later we may have some useful data, if a study can be conducted ethically in the first place. Meanwhile, the environment around us continues to change at a pace the likes of which humans have never experienced. Whatever this era of LLMs does (or does not) do to improve the situation, I am firmly confident that years-to-decades-long human testing is not the endgame of medical science, but rather a long and inconvenient pitstop. There's a lot of those in the history of medicine.
thayne 1y ago
Ok, suppose the current standard treatment is A. You have a new treatment B that is 100x cheaper than B, and doesn't trigger allergic reactions that some people have to A. You test against A, and the B group has slightly worse outcomes overall than A. Does that mean that B is useless? What if it is almost as effective, but it's lower cost means more people would be able to use it? "better" is not a total order, one treatment may be better in some ways and worse in other ways. Especially if you include things like cost and availability.
- rileymat2 1y ago
  Wouldn’t those factors be detected and reported on in the trial?
- refurb 1y ago
  You just discovered the entire scientific field of cost-effectiveness.
- duskwuff 1y ago
  Even if that's the case, the metric you're interested in isn't "is Treatment B better than doing nothing"; it's "how effective is Treatment B relative to Treatment A". And the most effective way to determine that will be to trial them against each other, not against a placebo.
  - thayne 1y ago
    Ideally, for getting good information, you compare to both. Because you also want to know if your treatment is more effective than a placebo.

saalweachter 1y ago

It also helps to actually go back and look at in terms of "rejecting the null hypothesis". If you're talking about a treatment for The Common Cold, the null hypothesis is "the subject got better after awhile because people get better after awhile", and you can't disprove that's what's happening without a very rigorous study with a well designed control. If you're talking about "here's some robot eyes that cure blindness", you don't really need a control group to test if it works. The null hypothesis is they're blind; you just need to demonstrate they can see to disprove the null hypothesis and prove efficacy.

standardUser 1y ago

And years later we may have some useful data, if a study can be conducted ethically in the first place. Meanwhile, the environment around us continues to change at a pace the likes of which humans have never experienced. Whatever this era of LLMs does (or does not) do to improve the situation, I am firmly confident that years-to-decades-long human testing is not the endgame of medical science, but rather a long and inconvenient pitstop. There's a lot of those in the history of medicine.

thayne 1y ago

Ok, suppose the current standard treatment is A. You have a new treatment B that is 100x cheaper than B, and doesn't trigger allergic reactions that some people have to A. You test against A, and the B group has slightly worse outcomes overall than A. Does that mean that B is useless? What if it is almost as effective, but it's lower cost means more people would be able to use it? "better" is not a total order, one treatment may be better in some ways and worse in other ways. Especially if you include things like cost and availability.

rileymat2 1y ago
Wouldn’t those factors be detected and reported on in the trial?
refurb 1y ago
You just discovered the entire scientific field of cost-effectiveness.
duskwuff 1y ago
Even if that's the case, the metric you're interested in isn't "is Treatment B better than doing nothing"; it's "how effective is Treatment B relative to Treatment A". And the most effective way to determine that will be to trial them against each other, not against a placebo.
- thayne 1y ago
  Ideally, for getting good information, you compare to both. Because you also want to know if your treatment is more effective than a placebo.