6 ms·

Researcher experiences of funder suppression of health trial findings

128 points by aww_dang 5y ago

aww_dang 5y ago
Full title: “He who pays the piper calls the tune”: Researcher experiences of funder suppression of health behaviour intervention trial findings
dootbin 5y ago
For those of you that also hate Reddit’s new interface… https://old.reddit.com/r/privacy/comments/pd4y5m/apples_new_private_relay_is_leaking_your_original/ https://old.reddit.com/r/privacy/comments/pd4y5m/apples_new_...
flenserboy 5y ago
This has been obvious for decades. A very interesting deep dive would be one showing just how firewalls have not been established which would mark particularly trustworthy studies.
phren0logy 5y ago
This is the problem trial registries are supposed to solve. There is an ethical problem with accepting subjects into a clinical trial, and potentially putting them at risk, then suppressing the results.
- hannob 5y ago
  The problem with trials registries is that they're only one piece of the solution, but many more things need to work for them to be effective. Many trials are registered, required by law to be published, but still not published, because there is no enforcement: https://fdaaa.trialstracker.net/ https://fdaaa.trialstracker.net/ https://eu.trialstracker.net/ https://eu.trialstracker.net/ Or trials alre registered, but then published with a different main result than the one that was registered as the main result (so-called outcome switching). This should be noted in peer review, but often it's not. And if you point it out to the journal later they should publish a correction, but they don't.
  - kingkawn 5y ago
    Because all of that has always been what science promised to justify its hold on funding and power while not delivering any of the accountability it claimed.
bjornsing 5y ago
Very interesting. But it’s worth keeping in mind that the most common suppression tactic was to ask the researcher not to publish (9%), and we have no idea in how many cases it worked (since the unpublished trials didn’t make it into this study)…
mmaunder 5y ago
Combine this with the public desire to suppress questioning the efficacy and safety of certain medications, and you have a recipe for disaster.
- jjoonathan 5y ago
  If it takes 1 unit of effort to ask a question, it takes 10 to answer if you know the answer and the audience knows the prereqs. It takes 100 units to answer if the audience needs a quick refresher and 1000 units to answer if the audience is missing fundamentals. Likewise, it takes 1000 units to do the research if you don't know the answer and 1,000,000 units if nobody knows the answer and it needs to be discovered. Your epistemology must take these ratios into account or it is trivial for a bad actor to fabricate truths by "just asking questions," creating unreasonable expectations of a quick and easy answer, and then inviting you to infer the truth of their malicious narrative from the absence of a quick and easy answer to the question they just posed.
  - awesomeusername 5y ago
    Wow interesting I've never heard it put like that. What a good framework for thinking about this
  - SantalBlush 5y ago
    Absolutely. For as long as the internet has been around, there has been this notion that someone asking questions is not only entitled to an explanation, but that it must fit into a brief response, the asker must be able to understand the explanation, and they must be thoroughly convinced by it. If none of these criteria are met, the asker concludes that the explanation does not exist or is wrong. It is a fallacy that is quite obvious with just a little thought. Example: "Someone prove to me in a written reply here that Fermat's Last Theorem is true. If you can't, then I submit that it's false." Complete nonsense.
  - Natsu 5y ago
    Or you can write the answer once and copy it infinitely. I understand the feeling because I frequently tell the vaccine-hesitant why they should vaccinate, but it's not nearly that bad if you actually try to inform people. Also, you're not accounting for the reverse, where something is popular but wrong and asserts that it should never ever have to explain why it's right. Finally, a more useful heuristic is to listen to explanations from people who can and do tell you how things work. The BS artists generally either have models that directly contradict what we know of the world or have no coherent model at all. Incidentally, the above post is an example of this, is something I've explained many times, and took me nowhere near 10 times as long as your post to write unless you can type significantly faster than 120 WPM.
DoreenMichele 5y ago
For me, if anything is surprising here, it's that it wasn't worse. Other issues in behavioral health studies: Self reporting can be unreliable. The types of assumptions you make shape the questions you ask which can bias results. (I.E. "Which color is it? Black? Or white?" "Neither. It's yellow." "I don't have a category for that. I will record that as white.") Honesty of participants for sensitive topics like sexual behavior and drug use that could have serious consequences for them.
- jpeloquin 5y ago
  > Honesty of participants for sensitive topics like sexual behavior and drug use that could have serious consequences for them. One way to help get around that is to ask the respondents to flip a coin—heads they answer truthfully, tails they answer falsely. This gives the respondents plausible deniability and the true underlying rates can still be estimated statistically from the raw results.
  - jonas21 5y ago
    I don't understand. Either you record the result of the coin flip, in which case you can easily determine each participant's actual answer -- or you don't, in which case the data collected is useless. An easy way to see this is if you suppose there's a question for which the truthful answer is "Yes" for 100% of participants. After the coin flip, you will get 50% "Yes" and 50% "No", which is the exact same result you would get if the truthful answer were "No" for 100% of participants.
    - jpeloquin 5y ago
      Here's where I saw it. And yes, I admit sloppiness with my word choice. "Differently from the usual Direct Response technique, in RR, respondents toss coins to determine whether they will respond to the question or just mark “yes”. This still allows admission rates to be calculated, yet it guarantees full anonymity to respondents because no one can tell whether an individual respondent answered “yes” to the question or because of chance." https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2685008/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2685008/ https://en.wikipedia.org/wiki/Randomized_response https://en.wikipedia.org/wiki/Randomized_response

128 points by aww_dang 5y ago

Full title: “He who pays the piper calls the tune”: Researcher experiences of funder suppression of health behaviour intervention trial findings

dootbin 5y ago

For those of you that also hate Reddit’s new interface… https://old.reddit.com/r/privacy/comments/pd4y5m/apples_new_private_relay_is_leaking_your_original/ https://old.reddit.com/r/privacy/comments/pd4y5m/apples_new_...

flenserboy 5y ago

This has been obvious for decades. A very interesting deep dive would be one showing just how firewalls have not been established which would mark particularly trustworthy studies.

phren0logy 5y ago

This is the problem trial registries are supposed to solve. There is an ethical problem with accepting subjects into a clinical trial, and potentially putting them at risk, then suppressing the results.

hannob 5y ago
The problem with trials registries is that they're only one piece of the solution, but many more things need to work for them to be effective. Many trials are registered, required by law to be published, but still not published, because there is no enforcement: https://fdaaa.trialstracker.net/ https://fdaaa.trialstracker.net/ https://eu.trialstracker.net/ https://eu.trialstracker.net/ Or trials alre registered, but then published with a different main result than the one that was registered as the main result (so-called outcome switching). This should be noted in peer review, but often it's not. And if you point it out to the journal later they should publish a correction, but they don't.
- kingkawn 5y ago
  Because all of that has always been what science promised to justify its hold on funding and power while not delivering any of the accountability it claimed.

bjornsing 5y ago

Very interesting. But it’s worth keeping in mind that the most common suppression tactic was to ask the researcher not to publish (9%), and we have no idea in how many cases it worked (since the unpublished trials didn’t make it into this study)…

mmaunder 5y ago

Combine this with the public desire to suppress questioning the efficacy and safety of certain medications, and you have a recipe for disaster.

jjoonathan 5y ago
If it takes 1 unit of effort to ask a question, it takes 10 to answer if you know the answer and the audience knows the prereqs. It takes 100 units to answer if the audience needs a quick refresher and 1000 units to answer if the audience is missing fundamentals. Likewise, it takes 1000 units to do the research if you don't know the answer and 1,000,000 units if nobody knows the answer and it needs to be discovered. Your epistemology must take these ratios into account or it is trivial for a bad actor to fabricate truths by "just asking questions," creating unreasonable expectations of a quick and easy answer, and then inviting you to infer the truth of their malicious narrative from the absence of a quick and easy answer to the question they just posed.
- awesomeusername 5y ago
  Wow interesting I've never heard it put like that. What a good framework for thinking about this
- SantalBlush 5y ago
  Absolutely. For as long as the internet has been around, there has been this notion that someone asking questions is not only entitled to an explanation, but that it must fit into a brief response, the asker must be able to understand the explanation, and they must be thoroughly convinced by it. If none of these criteria are met, the asker concludes that the explanation does not exist or is wrong. It is a fallacy that is quite obvious with just a little thought. Example: "Someone prove to me in a written reply here that Fermat's Last Theorem is true. If you can't, then I submit that it's false." Complete nonsense.
- Natsu 5y ago
  Or you can write the answer once and copy it infinitely. I understand the feeling because I frequently tell the vaccine-hesitant why they should vaccinate, but it's not nearly that bad if you actually try to inform people. Also, you're not accounting for the reverse, where something is popular but wrong and asserts that it should never ever have to explain why it's right. Finally, a more useful heuristic is to listen to explanations from people who can and do tell you how things work. The BS artists generally either have models that directly contradict what we know of the world or have no coherent model at all. Incidentally, the above post is an example of this, is something I've explained many times, and took me nowhere near 10 times as long as your post to write unless you can type significantly faster than 120 WPM.

DoreenMichele 5y ago

For me, if anything is surprising here, it's that it wasn't worse. Other issues in behavioral health studies: Self reporting can be unreliable. The types of assumptions you make shape the questions you ask which can bias results. (I.E. "Which color is it? Black? Or white?" "Neither. It's yellow." "I don't have a category for that. I will record that as white.") Honesty of participants for sensitive topics like sexual behavior and drug use that could have serious consequences for them.

jpeloquin 5y ago
> Honesty of participants for sensitive topics like sexual behavior and drug use that could have serious consequences for them. One way to help get around that is to ask the respondents to flip a coin—heads they answer truthfully, tails they answer falsely. This gives the respondents plausible deniability and the true underlying rates can still be estimated statistically from the raw results.
- jonas21 5y ago
  I don't understand. Either you record the result of the coin flip, in which case you can easily determine each participant's actual answer -- or you don't, in which case the data collected is useless. An easy way to see this is if you suppose there's a question for which the truthful answer is "Yes" for 100% of participants. After the coin flip, you will get 50% "Yes" and 50% "No", which is the exact same result you would get if the truthful answer were "No" for 100% of participants.
  - jpeloquin 5y ago
    Here's where I saw it. And yes, I admit sloppiness with my word choice. "Differently from the usual Direct Response technique, in RR, respondents toss coins to determine whether they will respond to the question or just mark “yes”. This still allows admission rates to be calculated, yet it guarantees full anonymity to respondents because no one can tell whether an individual respondent answered “yes” to the question or because of chance." https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2685008/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2685008/ https://en.wikipedia.org/wiki/Randomized_response https://en.wikipedia.org/wiki/Randomized_response