5 ms·

This paper really sounds like it advocates the status quo of attempting to take the "sum of the evidence", which usually leads to the generation of many differe

by nonbel 9y ago

This paper really sounds like it advocates the status quo of attempting to take the "sum of the evidence", which usually leads to the generation of many different weak (low sample size, etc) lines of evidence. They don't seem to get that what made science so successful was its ability to generate simpler models of phenomenon. To do this you need to come up with (most often quantitative) theories that can synthesize various lines of evidence. The practice of "counting heads" seen in fields like medicine is totally inappropriate (eg 2 out of 3 results support the theory so it looks good). The correct thing to do is more like what is seen in physics. Eg, take the idea that "cancer is due to mutations that accumulate as cells divide, therefore cancer rates should be related to number of divisions and mutation rate." The status quo is to do something like: 1) "cancer is positively correlated with number of divisions" 2) "cancer is positively correlated with mutation rate" 3) "cancer is positively correlated with number of accumulated mutations" Instead, we should say "this means the probability of accumulating at least n mutations after d divisions given mutation rate r should follow a law similar to": p(x >= n) = (1 - (1 - r)^d)^n The two approaches are similar in that you measure mutation rate, number of divisions, accumulated mutations and frequency of cancer cells. The difference is that for the latter approach all the lines of evidence combine in a nonlinear fashion. This method allows us to subsume a myriad of factors such as genetic disposition, exposure to toxins, etc to be incorporated via a single value (mutation rate). So rather than an ever expanding apparent complexity as more factors are incorporated, there is an accumulation of knowledge as we estimate the model parameters more and more accurately and either reject, modify, or keep our models of the phenomenon.

SiempreViernes 9y ago
Which paper are you talking about? Do you actually mean the linked article? Because it contains the following: > But replication alone will get us only so far. In some cases, routine replication might actually make matters worse. Consistent findings could take on the status of confirmed truths, when they actually reflect failings in study design, methods or analytical tools. This is a strong criticism against the counting heads method you also object to.
- nonbel 9y ago
  Look at their example. Theory: "One example in which triangulation has helped is in establishing that smoking during pregnancy results in babies with lower birth weights" Lines of evidence: 1) "women who smoke are more likely to have babies who weigh less" 2) "if a woman’s partner smokes during her pregnancy, many of the same confounders apply as in maternal smoking, but the association with lower birth weight is much weaker" 3) "Birth weight can also be analysed according to levels of cigarette taxation across US states, which reduces the effects of confounders." 4) "the birth weights of siblings whose mother smoked during one pregnancy but not another." 5) "In cohorts grouped according to whether or not people carry a genetic variant associated with greater cigarette consumption in those who smoke, mothers who smoke and carry the variant tended to have babies who weighed less; non-smokers with the same variant did not." Conclusion: "Taken together, these studies make it clear that maternal smoking affects birth weight directly" This looks exactly like what I described as "counting heads". There is no synthesis of this evidence, just a disconnected list of things (and who knows whether contrary evidence has been left out or never checked).
aaavl2821 9y ago
that's definitely a good approach in theory, but messier in reality. if we knew enough about which variables impacted disease, and could measure them well enough, and had comprehensive quantitative models of each variable interacts with a biological system, we could do this, but we dont. the reason we have to triangulate is because the types of modeling you do in physics just doesn't translate to fields like life science / medicine there are too many unknowns in life sci and error bars are too high compared to physics to enable this kind of model to be too accurate. collecting all of that data (ie mutation rate, number of divisions, accumuated mutations etc) across whole tumors in enough people to matter would be incredibly costly and time consuming, and the result probably wouldnt be as useful as we'd hope.
- nonbel 9y ago
  >"there are too many unknowns in life sci and error bars are too high compared to physics to enable this kind of model to be too accurate. collecting all of that data (ie mutation rate, number of divisions, accumuated mutations etc) across whole tumors in enough people to matter would be incredibly costly and time consuming, and the result probably wouldnt be as useful as we'd hope." Researchers will never cure cancer if they don't collect basic information like mutation rates, number of divisions since zygote as a function of age, cells in each tissue as a function of age, etc. It is like trying to develop astronomy without recording the positions of the planets... Also, I have seen that an only-slightly-more-complicated model than that extremely simple one (assuming a decaying curve for division rate as a function of age) is able to fit the age-specific incidence of many cancers (data from SEER). Have you checked that such approaches are too difficult or are you assuming so? Can you point me to info on any failed attempts? It really sounds like giving up to me. If that is the attitude of cancer researchers, I suspect eventually people with a less defeatist attitude will take it on (possibly coming from outside the healthcare industry).
  - aaavl2821 9y ago
    I think that new approaches to doing drug R&D are always welcome and if you know of any particular interesting and useful approaches id love to learn more. The "bottoms up" approach you mention seems like it could be really powerful and definitely complementary to whats being done now. I have however seen a lot of approaches "sort of" like that not work that well. I also know of a lot of less quantitative approaches taht actually work really well, and thats how a lot of R&D is done today I disagree w the statement that we'll never cure cancer w/o info on mutation rates. Many of the most impactful medicines we have were discovered without this kind of low level, quantitative info. Many through serendipity. Penicillin is the classic example. We have actually made huge progress in treating many cancers in the last five years. We can turn many melanomas from death sentences into relatively manageable chronic diseases, and a lot of blood cancers are being completely eradicated in some patients (at least for a time). We have a long way to go, but most of these insights come from triangulating various pieces of molecular biological and chemical information to create testable insights There has been a move lately to more "genomic" medicine where you identify a specific genentic mutation, and target that to treat disease. Most diseases aren't that simple though, and these are rarely curative There was also a move in the early 2000s to measure expression levels of hundreds of thousands of cancer genes to try to understand which ones would metastasize, how to treat them, etc. In some cases algorithms got pretty good at predicting rates of metastasis, but most didnt, and even fewer yielded clinically actionable insights. And I dont know of any drugs that were developed based on this work The idea of the moment is the "liquid biopsy", measuring pieces of DNA or proteins in the blood, and using machine learning to correlate these "biomarkers" with risk of getting cancer or other endpoints. The jury is still out on whether this works to my knowledge, though frankly i havent followed the science too closely No one is giving up, not in the least -- in fact more money is being invested into drug research than ever before. Its just really hard, and "triangulation" and insight are more powerful right now than purely quantitative methods. I hope that changes. But for things to change, we probably need better tools for collecting data before algorithms could be really powerful

by nonbel 9y ago

SiempreViernes 9y ago

Which paper are you talking about? Do you actually mean the linked article? Because it contains the following: > But replication alone will get us only so far. In some cases, routine replication might actually make matters worse. Consistent findings could take on the status of confirmed truths, when they actually reflect failings in study design, methods or analytical tools. This is a strong criticism against the counting heads method you also object to.

nonbel 9y ago
Look at their example. Theory: "One example in which triangulation has helped is in establishing that smoking during pregnancy results in babies with lower birth weights" Lines of evidence: 1) "women who smoke are more likely to have babies who weigh less" 2) "if a woman’s partner smokes during her pregnancy, many of the same confounders apply as in maternal smoking, but the association with lower birth weight is much weaker" 3) "Birth weight can also be analysed according to levels of cigarette taxation across US states, which reduces the effects of confounders." 4) "the birth weights of siblings whose mother smoked during one pregnancy but not another." 5) "In cohorts grouped according to whether or not people carry a genetic variant associated with greater cigarette consumption in those who smoke, mothers who smoke and carry the variant tended to have babies who weighed less; non-smokers with the same variant did not." Conclusion: "Taken together, these studies make it clear that maternal smoking affects birth weight directly" This looks exactly like what I described as "counting heads". There is no synthesis of this evidence, just a disconnected list of things (and who knows whether contrary evidence has been left out or never checked).

aaavl2821 9y ago

that's definitely a good approach in theory, but messier in reality. if we knew enough about which variables impacted disease, and could measure them well enough, and had comprehensive quantitative models of each variable interacts with a biological system, we could do this, but we dont. the reason we have to triangulate is because the types of modeling you do in physics just doesn't translate to fields like life science / medicine there are too many unknowns in life sci and error bars are too high compared to physics to enable this kind of model to be too accurate. collecting all of that data (ie mutation rate, number of divisions, accumuated mutations etc) across whole tumors in enough people to matter would be incredibly costly and time consuming, and the result probably wouldnt be as useful as we'd hope.

nonbel 9y ago
>"there are too many unknowns in life sci and error bars are too high compared to physics to enable this kind of model to be too accurate. collecting all of that data (ie mutation rate, number of divisions, accumuated mutations etc) across whole tumors in enough people to matter would be incredibly costly and time consuming, and the result probably wouldnt be as useful as we'd hope." Researchers will never cure cancer if they don't collect basic information like mutation rates, number of divisions since zygote as a function of age, cells in each tissue as a function of age, etc. It is like trying to develop astronomy without recording the positions of the planets... Also, I have seen that an only-slightly-more-complicated model than that extremely simple one (assuming a decaying curve for division rate as a function of age) is able to fit the age-specific incidence of many cancers (data from SEER). Have you checked that such approaches are too difficult or are you assuming so? Can you point me to info on any failed attempts? It really sounds like giving up to me. If that is the attitude of cancer researchers, I suspect eventually people with a less defeatist attitude will take it on (possibly coming from outside the healthcare industry).
- aaavl2821 9y ago
  I think that new approaches to doing drug R&D are always welcome and if you know of any particular interesting and useful approaches id love to learn more. The "bottoms up" approach you mention seems like it could be really powerful and definitely complementary to whats being done now. I have however seen a lot of approaches "sort of" like that not work that well. I also know of a lot of less quantitative approaches taht actually work really well, and thats how a lot of R&D is done today I disagree w the statement that we'll never cure cancer w/o info on mutation rates. Many of the most impactful medicines we have were discovered without this kind of low level, quantitative info. Many through serendipity. Penicillin is the classic example. We have actually made huge progress in treating many cancers in the last five years. We can turn many melanomas from death sentences into relatively manageable chronic diseases, and a lot of blood cancers are being completely eradicated in some patients (at least for a time). We have a long way to go, but most of these insights come from triangulating various pieces of molecular biological and chemical information to create testable insights There has been a move lately to more "genomic" medicine where you identify a specific genentic mutation, and target that to treat disease. Most diseases aren't that simple though, and these are rarely curative There was also a move in the early 2000s to measure expression levels of hundreds of thousands of cancer genes to try to understand which ones would metastasize, how to treat them, etc. In some cases algorithms got pretty good at predicting rates of metastasis, but most didnt, and even fewer yielded clinically actionable insights. And I dont know of any drugs that were developed based on this work The idea of the moment is the "liquid biopsy", measuring pieces of DNA or proteins in the blood, and using machine learning to correlate these "biomarkers" with risk of getting cancer or other endpoints. The jury is still out on whether this works to my knowledge, though frankly i havent followed the science too closely No one is giving up, not in the least -- in fact more money is being invested into drug research than ever before. Its just really hard, and "triangulation" and insight are more powerful right now than purely quantitative methods. I hope that changes. But for things to change, we probably need better tools for collecting data before algorithms could be really powerful