5 ms·

PLOS ONE issues editor's note over controversial chronic fatigue research

37 points by tokenadult 11y ago

davidgerard 11y ago
My jaw dropped at KCL's response to being asked for the data. How to trash your research reputation.
- mcguire 11y ago
  The letter from KCL is worth reading: "Further to your recent request for information held by King’s College London, I am writing to confirm that the requested information is held by the university. The university is withholding the information in accordance with section 14(1) of the Act – Vexatious Request. ... "You initially requested the information in accordance with the data sharing policies of the Public Library of Science (PLOS). The university has decided to treat this as a request under section 1(1) of the Act, as the information is held by the university. We received your information request on 13 November 2015." I'll further note that the professor making the request is a critic of the article. KCL appears to be unilaterally declaring the request to be under the FOIA instead of PLOS rules so they can deny it.
  - RobAley 11y ago
    To be fair, regardless of their actual reason, public bodies like Universities are compelled by law to treat request that amount to, or could be made as, FOIA requests as if they were. I.e. there is no magic incarnation, set of words or similar that need to be uttered to make it so. It's similar to requests under the DPA. As always there are grey areas, and I'm not making a judgement as to whether, even if it was considered an FOIA request, it would be considered Vexatious (I don't know enough detail). But it is incumbent upon them to consider it as such.
  - cpncrunch 11y ago
    This doesn't exactly help Coyne's case: https://twitter.com/CoyneoftheRealm/status/661238808814333952 https://twitter.com/CoyneoftheRealm/status/66123880881433395...
CFSThrowaway 11y ago
Chronic fatigue syndrome research is already too mired in politics. It is disappointing to see how poorly KCL has handled both the trial itself (post-hoc protocol changes, perhaps to make conclusions more significant) and their refusal to share data so others could verify their analysis. It is also disappointing to watch how vigorously a vocal subset of CFS researchers and sufferers oppose and attack any research that touches on the possibility of psychiatric treatments possibly benefiting chronic fatigue syndrome sufferers. KCL's poor behavior in refusing to release their data is only fanning the flames here, but the vigorous objections to CFS research that suggests cognitive behavior therapy (CBT) is beneficial goes back much further than this study. For some background, chronic fatigue syndrome has several different definitions, but many patients end up with a CFS diagnosis by exclusion of other therapies. This distorts the CFS community greatly, with many CFS community members not actually meeting the criteria for CFS. The PACE trial is a good example of this, with approximately 80% of the candidates excluded for various reasons (although obviously not all 80% were excluded strictly for failing to meet CFS criteria, but it's still significant). The problem is that many CFS sufferers, all of whom are suffering from very real physical fatigue and other physical symptoms, are very resistant to any suggestion that the disease has a psychiatric component or can be modified through mental health strategies. This objection runs so deep that researchers receive death threats for participating in studies exploring CBT for CFS, and online CFS forums are filled with vitriolic hate for any suggestion that CBT or other mental health strategies could possibly be effective. Meanwhile, studies (beyond just this PACE trial) have shown CBT to be one of the most effective options we have for CFS. It's not a cure, but it does seem to help on average. My point with this whole tangent is that chronic fatigue syndrome is so caught up in politics and resistance to certain ideas that most researchers don't even want to touch the topic any more. I certainly wouldn't want to research something if advocating the current best treatment option would result in death threats from sufferers. This is all very unfortunate, because instead of progress we're left with situations like the PACE trial where both KCL and a lot of the critical researchers are refusing to play nice with each other, and instead we get nowhere. And of course, there is now even more reason for researchers to not touch CFS research.
- Klinky 11y ago
  The problem is the rush some doctors are in to label a patient's condition as psychosomatic, and then prescribe drugs instead of something like CBT. A psychosomatic diagnosis is often code for "I am lazy" or "I don't know". There is also the other angle that people dealing with other stressfull diseases would likely benefit from CBT too, so does showing a benefit from CBT really mean it's psychosomatic?
hanklazard 11y ago
Why don't they just deidentify the data set? This not-new concept would allow the researchers to provide the data without compromising patient confidentiality. As for the argument re: follow-up study, seems that the scientific community should agree on a reasonable amount of time during which a group can keep and reanalyze their own published data after which the data must be made public (30 days? 60 days?).
- Paul-ish 11y ago
  It isn't easy to deidentidy a dataset. It isn't as simple as removing the names. You don't know in advanced what features in a dataset, when combined with a second dataset, will allow an adversary to reidentify data points. It isn't as simple as adding noise either, because reidentification can be treated like a machine learning problem.
- rflrob 11y ago
  Getting the scientific community to agree to almost anything makes herding cats look like a military parade. Even within a field, there are wildly varying opinions, and some fields tend to have higher or lower means. And thus far, I'm only talking about non-human where the primary concern is the ability to benefit from your own hard work without being "scooped on the analysis". Patient data adds a whole new dimension---with just genome sequence data, it's possible to infer someone's last name [1]. So deidentification is not as trivial as you make it sound. You can remove the obvious factors, but there's likely to be other things in there that you can't or don't know to remove that will inadvertently identify people. None of this is to excuse the original authors or KCL, but it is worth remembering that science is huge, and no single policy will work for everyone. What I think. PLoS could do is, as an editorial position, require anyone publishing with them to have some "close to raw" data publicly available BEFORE the manuscript can be published. Would at be controversial? Sure, but I think it's also in line with PLOS's mission of making science accessible to the public. [1] https://www.sciencemag.org/content/339/6117/321 https://www.sciencemag.org/content/339/6117/321
jimrandomh 11y ago
Chronic fatigue syndrome has the very unfortunate property of being a symptom, rather than a disease per se, so that it ends up being code for "undiagnosed". This makes any research which attempts to study it extremely difficult to interpret and sensitive to details of the study population which may not be reported. This makes raw data especially important.

37 points by tokenadult 11y ago

davidgerard 11y ago

My jaw dropped at KCL's response to being asked for the data. How to trash your research reputation.

mcguire 11y ago
The letter from KCL is worth reading: "Further to your recent request for information held by King’s College London, I am writing to confirm that the requested information is held by the university. The university is withholding the information in accordance with section 14(1) of the Act – Vexatious Request. ... "You initially requested the information in accordance with the data sharing policies of the Public Library of Science (PLOS). The university has decided to treat this as a request under section 1(1) of the Act, as the information is held by the university. We received your information request on 13 November 2015." I'll further note that the professor making the request is a critic of the article. KCL appears to be unilaterally declaring the request to be under the FOIA instead of PLOS rules so they can deny it.
- RobAley 11y ago
  To be fair, regardless of their actual reason, public bodies like Universities are compelled by law to treat request that amount to, or could be made as, FOIA requests as if they were. I.e. there is no magic incarnation, set of words or similar that need to be uttered to make it so. It's similar to requests under the DPA. As always there are grey areas, and I'm not making a judgement as to whether, even if it was considered an FOIA request, it would be considered Vexatious (I don't know enough detail). But it is incumbent upon them to consider it as such.
- cpncrunch 11y ago
  This doesn't exactly help Coyne's case: https://twitter.com/CoyneoftheRealm/status/661238808814333952 https://twitter.com/CoyneoftheRealm/status/66123880881433395...

CFSThrowaway 11y ago

Chronic fatigue syndrome research is already too mired in politics. It is disappointing to see how poorly KCL has handled both the trial itself (post-hoc protocol changes, perhaps to make conclusions more significant) and their refusal to share data so others could verify their analysis. It is also disappointing to watch how vigorously a vocal subset of CFS researchers and sufferers oppose and attack any research that touches on the possibility of psychiatric treatments possibly benefiting chronic fatigue syndrome sufferers. KCL's poor behavior in refusing to release their data is only fanning the flames here, but the vigorous objections to CFS research that suggests cognitive behavior therapy (CBT) is beneficial goes back much further than this study. For some background, chronic fatigue syndrome has several different definitions, but many patients end up with a CFS diagnosis by exclusion of other therapies. This distorts the CFS community greatly, with many CFS community members not actually meeting the criteria for CFS. The PACE trial is a good example of this, with approximately 80% of the candidates excluded for various reasons (although obviously not all 80% were excluded strictly for failing to meet CFS criteria, but it's still significant). The problem is that many CFS sufferers, all of whom are suffering from very real physical fatigue and other physical symptoms, are very resistant to any suggestion that the disease has a psychiatric component or can be modified through mental health strategies. This objection runs so deep that researchers receive death threats for participating in studies exploring CBT for CFS, and online CFS forums are filled with vitriolic hate for any suggestion that CBT or other mental health strategies could possibly be effective. Meanwhile, studies (beyond just this PACE trial) have shown CBT to be one of the most effective options we have for CFS. It's not a cure, but it does seem to help on average. My point with this whole tangent is that chronic fatigue syndrome is so caught up in politics and resistance to certain ideas that most researchers don't even want to touch the topic any more. I certainly wouldn't want to research something if advocating the current best treatment option would result in death threats from sufferers. This is all very unfortunate, because instead of progress we're left with situations like the PACE trial where both KCL and a lot of the critical researchers are refusing to play nice with each other, and instead we get nowhere. And of course, there is now even more reason for researchers to not touch CFS research.

Klinky 11y ago
The problem is the rush some doctors are in to label a patient's condition as psychosomatic, and then prescribe drugs instead of something like CBT. A psychosomatic diagnosis is often code for "I am lazy" or "I don't know". There is also the other angle that people dealing with other stressfull diseases would likely benefit from CBT too, so does showing a benefit from CBT really mean it's psychosomatic?

hanklazard 11y ago

Why don't they just deidentify the data set? This not-new concept would allow the researchers to provide the data without compromising patient confidentiality. As for the argument re: follow-up study, seems that the scientific community should agree on a reasonable amount of time during which a group can keep and reanalyze their own published data after which the data must be made public (30 days? 60 days?).

Paul-ish 11y ago
It isn't easy to deidentidy a dataset. It isn't as simple as removing the names. You don't know in advanced what features in a dataset, when combined with a second dataset, will allow an adversary to reidentify data points. It isn't as simple as adding noise either, because reidentification can be treated like a machine learning problem.
rflrob 11y ago
Getting the scientific community to agree to almost anything makes herding cats look like a military parade. Even within a field, there are wildly varying opinions, and some fields tend to have higher or lower means. And thus far, I'm only talking about non-human where the primary concern is the ability to benefit from your own hard work without being "scooped on the analysis". Patient data adds a whole new dimension---with just genome sequence data, it's possible to infer someone's last name [1]. So deidentification is not as trivial as you make it sound. You can remove the obvious factors, but there's likely to be other things in there that you can't or don't know to remove that will inadvertently identify people. None of this is to excuse the original authors or KCL, but it is worth remembering that science is huge, and no single policy will work for everyone. What I think. PLoS could do is, as an editorial position, require anyone publishing with them to have some "close to raw" data publicly available BEFORE the manuscript can be published. Would at be controversial? Sure, but I think it's also in line with PLOS's mission of making science accessible to the public. [1] https://www.sciencemag.org/content/339/6117/321 https://www.sciencemag.org/content/339/6117/321

jimrandomh 11y ago

Chronic fatigue syndrome has the very unfortunate property of being a symptom, rather than a disease per se, so that it ends up being code for "undiagnosed". This makes any research which attempts to study it extremely difficult to interpret and sensitive to details of the study population which may not be reported. This makes raw data especially important.