5 ms·

just how difficult it is to judge effectiveness of treatments and preventions in even extremely controlled studies. Translation: "just how difficult it is to b

by moldbug 15y ago

just how difficult it is to judge effectiveness of treatments and preventions in even extremely controlled studies. Translation: "just how difficult it is to be absolutely absolutely absolutely * 10^7 sure we're not selling quack medicine." The treatment of Steveoma is effective if Steve gets better. If not, it isn't. The sample size is 1 by definition. Obviously, if Steve gets better and he's taking 30 drugs, no one has any way to know which of the 30 worked, which were unnecessary, and which were actually counterproductive. But you still know more than you do when you started - you know that something worked. Lather, rinse, repeat. And your idea that somehow embracing quackery will lead to effective cancer treatments is just frankly insane. I love being refudiated in the morning!

ricree 14y ago
>The treatment of Steveoma is effective if Steve gets better. If not, it isn't. The sample size is 1 by definition. Well, it's effective if Steve gets better than he would have if he hadn't taken it. That's a rather important distinction, and it's real hard to tell the difference with low sample sizes.
- moldbug 14y ago
  Spontaneous remission, while it happens, is not a significant confounding factor in pancreatic cancer. See above for comments on statistics.
  - intended 14y ago
    But in your sample size of 1, and later in sample sizes of all cancer suffering billionaires (lets also add in close family) a single confounding factor means that the resulting data is useless, and will only result in a wild goose chase for years to come. Basically - any single chaotic factor outside the narrowly defined boundaries you have mentioned, will bork your entire system. While this means that we've reached a similar state to where we are today, the current state is advantageous because it is systematic from the ground up.
    - moldbug 14y ago
      Basically - any single chaotic factor outside the narrowly defined boundaries you have mentioned, will bork your entire system. Sure. If you're stupid. Science isn't for the stupid. There is a common if implicit belief that systematic methods will allow the dull, or worse the dishonest, to advance the frontiers of knowledge. No - the rules of science need to be made for those both brilliant and honest. Everyone else has plenty of other games they can play. Aristotle had a nice word for the intellectual quality it takes to learn from sample sizes of 1: http://en.wikipedia.org/wiki/Phronesis http://en.wikipedia.org/wiki/Phronesis
      - intended 14y ago
        ? > There is a common if implicit belief that systematic methods will allow the dull, or worse the dishonest, to advance the frontiers of knowledge. Ok, I'm taken aback by that - people actually think that people who are dull and dishonest become scientists? I mean I can understand that they become - frauds -. But scientists? Well maybe not in my country, but perhaps where you live. Anyway thats just plain daft - the smartest become scientists. >No - the rules of science need to be made for those both brilliant and honest. Everyone else has plenty of other games they can play. Well, aren't the rules of science the scientific method? Which anyone can use?
        moldbug 14y ago
        Ok, I'm taken aback by that - people actually think that people who are dull and dishonest become scientists? I mean I can understand that they become - frauds -. But scientists? Well maybe not in my country, but perhaps where you live. I'm afraid the problem is that my standards of both intelligence and honesty are much higher than yours. Our society has far too many scientists churning out what Kuhn called "normal science." Look at all the scientists in the '70s and '80s who performed fantastic feats of normal science to sequence some gene or other. Science and society in 2012 are precisely as advanced as they'd be if all these researchers had been driving cabs. Well, aren't the rules of science the scientific method? Which anyone can use? Anyone can use a paintbrush, too. Not everyone can paint a Picasso.
ErrantX 14y ago
I understand where your ideas come from, I wish they were true. But sadly they are misinformed on many levels, and the root of the problem is you are playing a statistical game where reality will win. - There are not enough Billionaires to meet your needs, statistically you'll need quite a few to find any decent cures. Especially with 30 teams in play on each case; figuring out "what worked" is a bad numbers game in that case. - Science shows there is unlikely to be one "magic bullet" for cancer, which makes it hard to cure individuals and then take that science into a wider populace; "something here worked, but does it work generally?" - Cancer treatments are harsh and invasive - by design! 30-40 competing drugs is unsustainable, you would be, to put it mildly, royally fucked. So death could come simply by exhausting the body with treatment, as easily as anything else. - You invoked Godwin's law, which is never a good sign. I agree with your general premise; which is that it takes too long to iterate ideas into actual production. But the reason that scenario exists is not just "quack medicine" and other hopeful (but useless) remedies, it is for the case of "oh shit, I just killed a million people because we didn't investigate long term effects". And that is where your theory really fails; because you are trading an unknown collateral now, for an unknown collateral in the future. I'm not arguing against either as a good or bad thing; but it does highlight the fatal flaw in the argument "you are killing X people right now by not releasing/iterating fast enough"
- Jach 14y ago
  I have some questions about your points... How many people do you think one would need to find any decent cures/treatments? How many patients are current researchers making use of to accomplish the feat? If moldbug had said something along the lines of "you can increase the numbers by having billionaires fund people other than themselves, such as their parents or other people they care about, or just as donations, in ways that don't bankrupt them but still provide lots of experimental data", do you think the increased number could still not possibly be enough, would you still object? If moldbug had started with 5 concurrent treatments instead of 30, would you object to that? (Though considering death is coming I personally would rather die of treatment exhaustion from X treatments than the actual cancer, knowing I tried many potential solutions. (This conditional on a cancer that actually threatens my life.)) Lastly, are the "unknown collaterals" equivalent unknowns? Shouldn't we try to estimate them and decide whether ramping up the release/iterate process is worth it? That people are dying because the research system sucks doesn't seem that controversial to me or very unknown in truth-value. How likely do you think it is that someone kills 1 million people with some remedy, and how likely do you think it is that the same 1 million people would have lived much longer without such a remedy being attempted on them?

by moldbug 15y ago

ricree 14y ago

>The treatment of Steveoma is effective if Steve gets better. If not, it isn't. The sample size is 1 by definition. Well, it's effective if Steve gets better than he would have if he hadn't taken it. That's a rather important distinction, and it's real hard to tell the difference with low sample sizes.

moldbug 14y ago
Spontaneous remission, while it happens, is not a significant confounding factor in pancreatic cancer. See above for comments on statistics.
- intended 14y ago
  But in your sample size of 1, and later in sample sizes of all cancer suffering billionaires (lets also add in close family) a single confounding factor means that the resulting data is useless, and will only result in a wild goose chase for years to come. Basically - any single chaotic factor outside the narrowly defined boundaries you have mentioned, will bork your entire system. While this means that we've reached a similar state to where we are today, the current state is advantageous because it is systematic from the ground up.
  - moldbug 14y ago
    Basically - any single chaotic factor outside the narrowly defined boundaries you have mentioned, will bork your entire system. Sure. If you're stupid. Science isn't for the stupid. There is a common if implicit belief that systematic methods will allow the dull, or worse the dishonest, to advance the frontiers of knowledge. No - the rules of science need to be made for those both brilliant and honest. Everyone else has plenty of other games they can play. Aristotle had a nice word for the intellectual quality it takes to learn from sample sizes of 1: http://en.wikipedia.org/wiki/Phronesis http://en.wikipedia.org/wiki/Phronesis
    - intended 14y ago
      ? > There is a common if implicit belief that systematic methods will allow the dull, or worse the dishonest, to advance the frontiers of knowledge. Ok, I'm taken aback by that - people actually think that people who are dull and dishonest become scientists? I mean I can understand that they become - frauds -. But scientists? Well maybe not in my country, but perhaps where you live. Anyway thats just plain daft - the smartest become scientists. >No - the rules of science need to be made for those both brilliant and honest. Everyone else has plenty of other games they can play. Well, aren't the rules of science the scientific method? Which anyone can use?
      - moldbug 14y ago
        Ok, I'm taken aback by that - people actually think that people who are dull and dishonest become scientists? I mean I can understand that they become - frauds -. But scientists? Well maybe not in my country, but perhaps where you live. I'm afraid the problem is that my standards of both intelligence and honesty are much higher than yours. Our society has far too many scientists churning out what Kuhn called "normal science." Look at all the scientists in the '70s and '80s who performed fantastic feats of normal science to sequence some gene or other. Science and society in 2012 are precisely as advanced as they'd be if all these researchers had been driving cabs. Well, aren't the rules of science the scientific method? Which anyone can use? Anyone can use a paintbrush, too. Not everyone can paint a Picasso.

ErrantX 14y ago

I understand where your ideas come from, I wish they were true. But sadly they are misinformed on many levels, and the root of the problem is you are playing a statistical game where reality will win. - There are not enough Billionaires to meet your needs, statistically you'll need quite a few to find any decent cures. Especially with 30 teams in play on each case; figuring out "what worked" is a bad numbers game in that case. - Science shows there is unlikely to be one "magic bullet" for cancer, which makes it hard to cure individuals and then take that science into a wider populace; "something here worked, but does it work generally?" - Cancer treatments are harsh and invasive - by design! 30-40 competing drugs is unsustainable, you would be, to put it mildly, royally fucked. So death could come simply by exhausting the body with treatment, as easily as anything else. - You invoked Godwin's law, which is never a good sign. I agree with your general premise; which is that it takes too long to iterate ideas into actual production. But the reason that scenario exists is not just "quack medicine" and other hopeful (but useless) remedies, it is for the case of "oh shit, I just killed a million people because we didn't investigate long term effects". And that is where your theory really fails; because you are trading an unknown collateral now, for an unknown collateral in the future. I'm not arguing against either as a good or bad thing; but it does highlight the fatal flaw in the argument "you are killing X people right now by not releasing/iterating fast enough"

Jach 14y ago
I have some questions about your points... How many people do you think one would need to find any decent cures/treatments? How many patients are current researchers making use of to accomplish the feat? If moldbug had said something along the lines of "you can increase the numbers by having billionaires fund people other than themselves, such as their parents or other people they care about, or just as donations, in ways that don't bankrupt them but still provide lots of experimental data", do you think the increased number could still not possibly be enough, would you still object? If moldbug had started with 5 concurrent treatments instead of 30, would you object to that? (Though considering death is coming I personally would rather die of treatment exhaustion from X treatments than the actual cancer, knowing I tried many potential solutions. (This conditional on a cancer that actually threatens my life.)) Lastly, are the "unknown collaterals" equivalent unknowns? Shouldn't we try to estimate them and decide whether ramping up the release/iterate process is worth it? That people are dying because the research system sucks doesn't seem that controversial to me or very unknown in truth-value. How likely do you think it is that someone kills 1 million people with some remedy, and how likely do you think it is that the same 1 million people would have lived much longer without such a remedy being attempted on them?