3 ms·

Your comment is almost completely orthogonal to dekhn's. The point is that, regardless of progress/stagnation in cancer research, success in mouse models is no

Your comment is almost completely orthogonal to dekhn's. The point is that, regardless of progress/stagnation in cancer research, success in mouse models is not a strong indicator of eventual success in humans.

Retric 7y ago
Mouse models are almost the only indicator for future success. So while many fail, most progress comes through research just like this. Just remember, successful means lower the rates of deaths they don’t generally eliminate it. So, yay study 1357 got the survival rate from 5% to 10% that’s real progress. (Reported as curing cancer). Study 2217 got the survival rate from 10% or 15% wow progress. (Reported as curing cancer). Study 3... I mean grated they did help some specific individual within that study so it’s not meaningless, but come on.
- jessriedel 7y ago
  > Mouse models are almost the only indicator for future success. What do you mean by this? The vast, vast majority of things that work in mice do not work in humans. Even if effectiveness in mice is (approximately) necessary for eventual effectiveness in humans (say, because of our research methods), this does not make it a good indicator. "Drug doesn't immediately melt your brain" is also a necessary property, but it doesn't do a good job of predicting effectiveness.
  - Retric 7y ago
    If you only look at everything that has already passed the test then sure the test does not seem important. The set of all possible cancer treatments however are not just the set of treatments that pass animal models. Drugs etc need to go though several hoops before they are even tested in mice, and even more fail in mice. Everything that’s made it to this stage is extremely promising relative to the baseline.
    - jessriedel 7y ago
      If the baseline is success probability 10^-8, raising it to 10^-5 is extremely promising at a relative level, but not at an absolute level.
      - Retric 7y ago
        That’s fair though 10^-5 is wildly off base. About 1/3 of drugs that show promise on mouse models are ever tested in humans. After that Oncology has a 8.3 percent success rate for 2015. So, when looking at ‘cured cancer in mice’ something like 2.5% is about the right ballpark, which I would consider promising as a general baseline. For some comparisons the success rate in clinical trials averages around 14% though it ranges from around 33.4 percent in vaccines for infectious diseases to 3.4 depending on the type of treatment being tested. https://www.centerwatch.com/cwweekly/2018/02/05/new-mit-study-puts-clinical-research-success-rate-14-percent/ https://www.centerwatch.com/cwweekly/2018/02/05/new-mit-stud... Baseline is harder to calculate, but 10^-8 is probably reasonable depending on what’s considered.
        jessriedel 7y ago
        > About 1/3 of drugs that show promise on mouse models are ever tested in humans. Do you have a citation? This seems totally off to me. In so far as the number refers to something real, I suspect "shows promise" must mean something much stricter than the threshold that this particular treatment has passed. That is, I would bet against a 2.5% chance that this particular study leads to an actual drug approved to treat pancreatic cancer.
- dekhn 7y ago
  if your proxy metric correlates less than 0.5 with the system of interest (roughly correct for mouse models), then the indicator is less than useful- it causes systematic and random false negatives and positives at a higher rate than true positives.

Retric 7y ago

Mouse models are almost the only indicator for future success. So while many fail, most progress comes through research just like this. Just remember, successful means lower the rates of deaths they don’t generally eliminate it. So, yay study 1357 got the survival rate from 5% to 10% that’s real progress. (Reported as curing cancer). Study 2217 got the survival rate from 10% or 15% wow progress. (Reported as curing cancer). Study 3... I mean grated they did help some specific individual within that study so it’s not meaningless, but come on.

jessriedel 7y ago
> Mouse models are almost the only indicator for future success. What do you mean by this? The vast, vast majority of things that work in mice do not work in humans. Even if effectiveness in mice is (approximately) necessary for eventual effectiveness in humans (say, because of our research methods), this does not make it a good indicator. "Drug doesn't immediately melt your brain" is also a necessary property, but it doesn't do a good job of predicting effectiveness.
- Retric 7y ago
  If you only look at everything that has already passed the test then sure the test does not seem important. The set of all possible cancer treatments however are not just the set of treatments that pass animal models. Drugs etc need to go though several hoops before they are even tested in mice, and even more fail in mice. Everything that’s made it to this stage is extremely promising relative to the baseline.
  - jessriedel 7y ago
    If the baseline is success probability 10^-8, raising it to 10^-5 is extremely promising at a relative level, but not at an absolute level.
    - Retric 7y ago
      That’s fair though 10^-5 is wildly off base. About 1/3 of drugs that show promise on mouse models are ever tested in humans. After that Oncology has a 8.3 percent success rate for 2015. So, when looking at ‘cured cancer in mice’ something like 2.5% is about the right ballpark, which I would consider promising as a general baseline. For some comparisons the success rate in clinical trials averages around 14% though it ranges from around 33.4 percent in vaccines for infectious diseases to 3.4 depending on the type of treatment being tested. https://www.centerwatch.com/cwweekly/2018/02/05/new-mit-study-puts-clinical-research-success-rate-14-percent/ https://www.centerwatch.com/cwweekly/2018/02/05/new-mit-stud... Baseline is harder to calculate, but 10^-8 is probably reasonable depending on what’s considered.
      - jessriedel 7y ago
        > About 1/3 of drugs that show promise on mouse models are ever tested in humans. Do you have a citation? This seems totally off to me. In so far as the number refers to something real, I suspect "shows promise" must mean something much stricter than the threshold that this particular treatment has passed. That is, I would bet against a 2.5% chance that this particular study leads to an actual drug approved to treat pancreatic cancer.
dekhn 7y ago
if your proxy metric correlates less than 0.5 with the system of interest (roughly correct for mouse models), then the indicator is less than useful- it causes systematic and random false negatives and positives at a higher rate than true positives.