5 ms·

"The findings are yet to be published and reviewed." Are people really ok with this practice of researchers talking about their discoveries without having shar

by nonbel 11y ago

"The findings are yet to be published and reviewed." Are people really ok with this practice of researchers talking about their discoveries without having shared the methods and data (I don't care so much about the review aspect)? I'm certainly not, but apparently many think this is acceptable for some reason because I don't see any pushback.

tominous 11y ago
I'm biased. My wife was diagnosed with metastatic cancer 6 months ago. With current standard treatments there is a 3-year survival rate of practically zero, so to me the most important thing is getting new effective treatments developed and brought into clinical practice as soon as possible. I haven't seen the presentation about the study (a related news briefing is here [1]) so I don't know whether you're correct to say that the researchers haven't shared their methods and data. They seem quite happy to share technical details at the news briefing. However, regardless of that, I still think it is a good thing if researchers talk about their findings at an early stage, especially if the findings are as dramatic and positive as these. If a treatment is available and you have patients out there dying with no other options, then it is just wrong to hide that away. In my opinion we treat these things far too cautiously. It is not optimal for the individual and it is not optimal for society. By sharing /and/ opening up options like this to terminally ill patients we can (a) save individual lives and (b) speed up development and refinement of new treatments to benefit everyone. The problem is that the incentives we have in place treat a cancer death as acceptable, while a treatment-related death is unacceptable for researchers, doctors and drug companies (unless they are shielded by layers of clinical-trial bureaucracy or years of standard practice). The result is that the course of treatment doesn't properly weigh the different sources of risk. Whether the goal is quality-adjusted life years for an individual patient or faster development of effective and safe treatments, the current system is way too conservative. [1] http://www.eurekalert.org/aaasnewsroom/2016/webcast/?b=38 http://www.eurekalert.org/aaasnewsroom/2016/webcast/?b=38
- nonbel 11y ago
  For what it's worth, I used to say I would take some experimental treatment when the time came. Not after doing research myself and seeing first hand what "experimental" really means when it comes to medicine. It means: no one has any way of really knowing what is going on. For example, 1) There will be some studies where a measurement was higher in the treatment group compared to controls, but it is not 100% clear what combination of things is actually being measured. 2) There will be other studies you hear about that got different results, but didn't get published because publishing "negative results" is discouraged and bad for careers. 3) The criteria for publishing a positive result is a p-value below an arbitrary threshold, and most people researching medical treatments really do not understand the meaning of a p-value to begin with. There are all sorts of myths like it is the probability the treatment didn't work, etc. So (at least the implementation of) the basic criteria used to judge success is fatally flawed. It is quite scary and I don't see anything I can do about it but make people aware there is a problem here.
  - tominous 11y ago
    From a sociology point of view, yes you should distrust any signs of p-hacking or selective reporting, especially when there is a low effect size. From a technology point of view, I think we are already well past "no one.. really knowing what is going on." Take the current study. Past researchers had identified the CD19 antigen found (often exclusively) on B cells, and others had found that autologous T-cells could persist in a patient for many years, potentially providing a durable response. The current researchers built on this knowledge to design and engineer T-cells that could wipe out a patient's B cells. So it's a very planned, targeted approach, more akin to engineering than the old approach of testing thousands of random molecules as potential treatments. When the targeted approach works, and especially when it has a large and consistent effect on patients as in this case, then you can be very confident at an early stage. This needs to be recognized and medical practice needs to change as a result. We're entering an era of incredibly detailed knowledge and precision tools for research and development. The pace of development will continue to increase and treatments will become more patient-specific as individual targets are identified. You could consider the large, slow, blunt clinical trials as being like optimization for a particular treatment. When new drugs are rare or not well understood it makes sense to spend time to optimize their use. But when new drugs come frequently and are well targeted this premature optimization costs lives.
- Sven7 11y ago
  If you know the experimental drug\treatment you want to try check overseas. Don't wait for local (I'm assuming) docs to give you the info. Contact centers with a known rep. Good luck!
  - tominous 11y ago
    Thanks, I certainly have my eyes peeled for news about metastatic ACUP (or any related diagnoses) but as far as I'm aware there's no silver bullet anywhere yet. Right now the strategy is to kick the can down the road until there is a solution: traditional chemo, then the blunt immunotherapy of ipilimumab + nivolumab/pembrolizumab. Also (with oncologist's knowledge) adding in very speculative but reasonably safe extras like a COX-2 inhibitor, anti-histamine, and melatonin. (Certainly not the well-targeted approach I talk about in my other comment, unfortunately!)
- GrinningFool 11y ago
  > I'm biased. My wife was diagnosed with metastatic cancer 6 months ago My sincerest condolences. Even attempting to imagine what that must be like is beyond painful; my mind refuses to contemplate it for more than a few moments. I hope that you're able to find something that works out.
mbreese 11y ago
These results were presented at a meeting. The final results will be eventually published in a peer-reviewed journal. This technique is actually fairly well known and other groups are running similar trials. So this isn't some black box. There is a long history of peer-reviewed publications that got us to this point. The basic science behind these techniques has already been published. This is just the next step - presenting the clinical trial data for these studies.
- nonbel 11y ago
  "These results were presented at a meeting. The final results will be eventually published in a peer-reviewed journal." They could put the information on their own webpage for all I care, as long as they are willing to stake their reputations on it. The point is that we, the audience, do not have access to the requisite information (methods, data, background refs they trust) to even start attempting to judge the claims. Even if a conference talk/poster/abstract is available somewhere, that is an extremely limited format of communication.
  - mbreese 11y ago
    With all due respect, in all likelihood, you are not the audience for this talk (I honestly don't know). For those who are in the field, who were at the talk, I'm sure there was more than enough information presented for them to judge the claims (one way or the other). Think of this as a preprint for the paper. It's just a talk. If you want to blame anyone here, you could blame the media for reporting on a talk. But then again, the public likes to hear good news about cancer research, so I think the media is okay on this one.
    - nonbel 11y ago
      >"For those who are in the field, who were at the talk, I'm sure there was more than enough information presented for them to judge the claims (one way or the other)." I can assure you 100% this is not the case. Perhaps they could get that info by extensively talking to the presenter afterwards (which will not be recorded), but no way you can fit it into a 15-60 min talk or onto a poster, or any way that will be shared later. Not physically possible. Edit: Ok, I imagined a bunch of androids like Data (star trek character), and believe it may be physically possible but would require superhuman capabilities to transmit and process information.

by nonbel 11y ago

tominous 11y ago

I'm biased. My wife was diagnosed with metastatic cancer 6 months ago. With current standard treatments there is a 3-year survival rate of practically zero, so to me the most important thing is getting new effective treatments developed and brought into clinical practice as soon as possible. I haven't seen the presentation about the study (a related news briefing is here [1]) so I don't know whether you're correct to say that the researchers haven't shared their methods and data. They seem quite happy to share technical details at the news briefing. However, regardless of that, I still think it is a good thing if researchers talk about their findings at an early stage, especially if the findings are as dramatic and positive as these. If a treatment is available and you have patients out there dying with no other options, then it is just wrong to hide that away. In my opinion we treat these things far too cautiously. It is not optimal for the individual and it is not optimal for society. By sharing /and/ opening up options like this to terminally ill patients we can (a) save individual lives and (b) speed up development and refinement of new treatments to benefit everyone. The problem is that the incentives we have in place treat a cancer death as acceptable, while a treatment-related death is unacceptable for researchers, doctors and drug companies (unless they are shielded by layers of clinical-trial bureaucracy or years of standard practice). The result is that the course of treatment doesn't properly weigh the different sources of risk. Whether the goal is quality-adjusted life years for an individual patient or faster development of effective and safe treatments, the current system is way too conservative. [1] http://www.eurekalert.org/aaasnewsroom/2016/webcast/?b=38 http://www.eurekalert.org/aaasnewsroom/2016/webcast/?b=38

nonbel 11y ago
For what it's worth, I used to say I would take some experimental treatment when the time came. Not after doing research myself and seeing first hand what "experimental" really means when it comes to medicine. It means: no one has any way of really knowing what is going on. For example, 1) There will be some studies where a measurement was higher in the treatment group compared to controls, but it is not 100% clear what combination of things is actually being measured. 2) There will be other studies you hear about that got different results, but didn't get published because publishing "negative results" is discouraged and bad for careers. 3) The criteria for publishing a positive result is a p-value below an arbitrary threshold, and most people researching medical treatments really do not understand the meaning of a p-value to begin with. There are all sorts of myths like it is the probability the treatment didn't work, etc. So (at least the implementation of) the basic criteria used to judge success is fatally flawed. It is quite scary and I don't see anything I can do about it but make people aware there is a problem here.
- tominous 11y ago
  From a sociology point of view, yes you should distrust any signs of p-hacking or selective reporting, especially when there is a low effect size. From a technology point of view, I think we are already well past "no one.. really knowing what is going on." Take the current study. Past researchers had identified the CD19 antigen found (often exclusively) on B cells, and others had found that autologous T-cells could persist in a patient for many years, potentially providing a durable response. The current researchers built on this knowledge to design and engineer T-cells that could wipe out a patient's B cells. So it's a very planned, targeted approach, more akin to engineering than the old approach of testing thousands of random molecules as potential treatments. When the targeted approach works, and especially when it has a large and consistent effect on patients as in this case, then you can be very confident at an early stage. This needs to be recognized and medical practice needs to change as a result. We're entering an era of incredibly detailed knowledge and precision tools for research and development. The pace of development will continue to increase and treatments will become more patient-specific as individual targets are identified. You could consider the large, slow, blunt clinical trials as being like optimization for a particular treatment. When new drugs are rare or not well understood it makes sense to spend time to optimize their use. But when new drugs come frequently and are well targeted this premature optimization costs lives.
Sven7 11y ago
If you know the experimental drug\treatment you want to try check overseas. Don't wait for local (I'm assuming) docs to give you the info. Contact centers with a known rep. Good luck!
- tominous 11y ago
  Thanks, I certainly have my eyes peeled for news about metastatic ACUP (or any related diagnoses) but as far as I'm aware there's no silver bullet anywhere yet. Right now the strategy is to kick the can down the road until there is a solution: traditional chemo, then the blunt immunotherapy of ipilimumab + nivolumab/pembrolizumab. Also (with oncologist's knowledge) adding in very speculative but reasonably safe extras like a COX-2 inhibitor, anti-histamine, and melatonin. (Certainly not the well-targeted approach I talk about in my other comment, unfortunately!)
GrinningFool 11y ago
> I'm biased. My wife was diagnosed with metastatic cancer 6 months ago My sincerest condolences. Even attempting to imagine what that must be like is beyond painful; my mind refuses to contemplate it for more than a few moments. I hope that you're able to find something that works out.

mbreese 11y ago

These results were presented at a meeting. The final results will be eventually published in a peer-reviewed journal. This technique is actually fairly well known and other groups are running similar trials. So this isn't some black box. There is a long history of peer-reviewed publications that got us to this point. The basic science behind these techniques has already been published. This is just the next step - presenting the clinical trial data for these studies.

nonbel 11y ago
"These results were presented at a meeting. The final results will be eventually published in a peer-reviewed journal." They could put the information on their own webpage for all I care, as long as they are willing to stake their reputations on it. The point is that we, the audience, do not have access to the requisite information (methods, data, background refs they trust) to even start attempting to judge the claims. Even if a conference talk/poster/abstract is available somewhere, that is an extremely limited format of communication.
- mbreese 11y ago
  With all due respect, in all likelihood, you are not the audience for this talk (I honestly don't know). For those who are in the field, who were at the talk, I'm sure there was more than enough information presented for them to judge the claims (one way or the other). Think of this as a preprint for the paper. It's just a talk. If you want to blame anyone here, you could blame the media for reporting on a talk. But then again, the public likes to hear good news about cancer research, so I think the media is okay on this one.
  - nonbel 11y ago
    >"For those who are in the field, who were at the talk, I'm sure there was more than enough information presented for them to judge the claims (one way or the other)." I can assure you 100% this is not the case. Perhaps they could get that info by extensively talking to the presenter afterwards (which will not be recorded), but no way you can fit it into a 15-60 min talk or onto a poster, or any way that will be shared later. Not physically possible. Edit: Ok, I imagined a bunch of androids like Data (star trek character), and believe it may be physically possible but would require superhuman capabilities to transmit and process information.