4 ms·

A study out of thin air

61 points by valinsky 6y ago

ListeningPie 6y ago
Summary: Because of one report in the Lancet By a shady corporation called Surgisphere the WHO stopped all further research into hydroxychloroquine. The article makes a strong case the Surgisphere is shady and the study is badly made but other than timing (paper published 22, ban started 25th) the article does not provide any further information that the Surgisphere study was the source of the WHO decision. I quoted the unsupported accusation below. Instead of performing its own due diligence, the WHO immediately relied on an observational study cloaked in the reputation of the nearly 200-year old medical journal The Lancet.
- scotty79 6y ago
  And people scoff at reporters and journalists that they don't check their sources... Apparently almost nobody does. We have copy/paste information ecosphere.
  - kiba 6y ago
    I did some checking of a lecture by an expert in virology, and I found an error. There are a few facts that I couldn't figure out if they were supported by other sources. It wasn't anything particularly intensive, just making sure the other sources said the same thing. But it was still a lot of work. For most people, learning is a lot of work in and itself.
    - smitty1e 6y ago
      The whole point of peer-reviewed journals is to be a noise filter. The scientific community purportedly polices itself.
      - acqq 6y ago
        The errors are indeed found, eventually, like this time. It's the expectancy of anything being completely "error free" from the start is simply false. Compare with https://news.ycombinator.com/item?id=23412482 https://news.ycombinator.com/item?id=23412482 Look at the all the infrastructure involved there, all the automatic tests that exist in the originating infrastructure, and still only 1) an independent fuzzing test by the company with the most computers in the world 2) the work of the leader of another project; bring "medial" attention to the existence of the problem. Nothing can be always perfect in the first try. In science specifically, the "intentional frauds" aren't the starting assumption of those who do review, that's why the frauds manage to pass the checks that do work most of the time. "Falsehood flies, and truth comes limping after it." Johnatan Swift (1667-1745)
      - 0d9eooo 6y ago
        I think this is kind of the tip of the iceberg, a general reflection on biomedical research and academics. Not that this is how most researchers act, but in the sense that it's not unusual for the field as a whole to encounter this. I mean, if this happens in the Lancet and with the WHO with something with worldwide significant consequences, what do people think is happening with stuff with less scrutiny? It also is a twist on all the calls for the importance of rigorous peer-reviewed studies, as if the preprint flood is just an example of why we need traditional academic structures. Here's a case where we needed the traditional academic structure and it failed miserably, even when prestigious institutions. The irony is that this is being caught, but outside the confines of the traditional peer review. That is, there is peer review, but in the general public of scientists. So which is better? It seems like open publication and transparency is key, more so than the traditional structures per se.
        matthewdgreen 6y ago
        Peer review is like a spam filter. But just because something gets past your spam filter does not mean you should assume it's accurate, particularly when malicious and fraudulent behavior is involved. What appears to have happened here is that a rapid peer-review gave the benefit of the doubt to a purported data source, and then the community rapidly discovered that the data source was not behaving honestly. All of this occurred in a matter of a few weeks, during a time when peer-review is almost certainly moving even faster than normal due to the urgency of the pandemic.
  - ListeningPie 6y ago
    To be fair, finding out all the information about Surgisphere probably took a lot time. Working on free projects myself there just comes a point where I've spent all the energy I was willing to put into a piece of work. Edit: Nevermind. The author of the article is the bit-coin investor how first tweeted about Chloroquine being a potential cure, https://www.wired.com/story/an-old-malaria-drug-may-fight-covid-19-and-silicon-valleys-into-it/ https://www.wired.com/story/an-old-malaria-drug-may-fight-co...
    - m0zg 6y ago
      He's not the only one pointing this out, though: https://www.theguardian.com/world/2020/may/29/covid-19-surgisphere-hydroxychloroquine-study-lancet-coronavirus-who-questioned-by-researchers-medical-professionals https://www.theguardian.com/world/2020/may/29/covid-19-surgi..., and the "study" is very easily verifiable as horseshit.
- beaner 6y ago
  > the article does not provide any further information that the Surgisphere study was the source of the WHO decision Genuine question: What else is it assumed WHO may have made the decision based on?
  - seesawtron 6y ago
    When such a large study is published in a big Journal, claiming that a drug might have a negative impact on lives of people in the trials, the authorities will be considered stupid not to pay attention to it at all and continue the trials. That is exactly what they did: They paused their trials temporarily to look at the evidence of this study, examined it and found that it was indeed controversial and lacked solid data. Now they have resumed the trials again. The media sells it as a clickbait story "All trials STOPPED because of a flawed study" implying they were stopped PERMANENTLY to excite the general audience. For them it is entirely something that brings them huge revenue from ads when they write such "emotinally driven" stories, they do not care about the effects it has on the public in the long term by reducing faith in world health organizations.
    - Fjolsvith 6y ago
      > they do not care about the effects it has on the public in the long term by reducing faith in world health organizations. Or even the direct effects on the public in terms of healthcare.
  - cycomanic 6y ago
    Well quite a few countries had stopped chloroquine use even before this study because they found little to no benifit, but significantly increased risks (not based on studies because they are difficult to do, but based on evidence by the treating doctors) , Sweden stopped IIRC mid april. IMO Germany and France also stopped sometime in May. Several smaller studies have also previously showed that chloroquine has increased risks and very little benefit (a number of them actually interrupted the studies because of the high risks). As much as I was hopeful when I first heard about it, I'm now very sceptical. It also doesn't help that Didier Raoult has a bit of a spotty past with ethical research conduct.
- JorgeGT 6y ago
  WHO director-general Tedros Adhanom Ghebreyesus directly pointed out the Lancet paper in a press conference as the reason to pause the trials and start a safety review. You can see the video here: https://time.com/5842264/hydroxychloroquine-testing-who/ https://time.com/5842264/hydroxychloroquine-testing-who/
  - seesawtron 6y ago
    The trials are beginning again as per WHO decision now.
- deleted 6y ago
  [deleted]
seesawtron 6y ago
The trials are beginning again as per WHO decision now.
thaumaturgy 6y ago
This site's front page has got a lot of red flags on it. There's another treatment of this over on sciencemag: https://www.sciencemag.org/news/2020/06/mysterious-company-s-coronavirus-papers-top-medical-journals-may-be-unraveling https://www.sciencemag.org/news/2020/06/mysterious-company-s... This would be a pretty serious black eye for both The Lancet and the WHO, which has already been under scrutiny from multiple countries. However, it's worth pointing out that this wasn't the only study on HCQ. There have been others: https://www.cidrap.umn.edu/news-perspective/2020/05/studies-find-further-lack-covid-benefit-hydroxychloroquine https://www.cidrap.umn.edu/news-perspective/2020/05/studies-... If the Surgisphere observational study turns out to be erroneous, it could knock another leg out from under the WHO, and if it turns out to be fraudulent, everyone involved with it deserves jail time. It'll be terrible in either case that it resulted in a bunch of other trials and studies being canceled.
- sampo 6y ago
  Lots of hydroxychloroquine studies or preprints have been published, and they mostly observe that the effect is close to zero (in good or in bad), either because there is no effect, or the study size was too small to detect other than large effects. But this now infamous study was the only one to suggest a clear and large negative effect.
  - Fjolsvith 6y ago
    So, if hydroxychloroquine is not effective, what explains an extremely low COVID death rate such as what Bangladesh (a country with endemic Malaria [1]) has? Bangladesh population: 161 million [2] Bangladesh COVID deaths: 811 [3] Bangladesh COVID death rate: 811/161m = 0.000503% 1. https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(13)70176-1/fulltext#:~:text=Malaria%20is%20endemic%20in%2013,Program%20(NMCP)%20in%202007 https://www.thelancet.com/journals/langlo/article/PIIS2214-1.... 2. https://www.google.com/search?q=bangladesh+population&rlz=1C1CHBD_enUS798US798&oq=bangladesh+population&aqs=chrome..69i57.4075j0j7&sourceid=chrome&ie=UTF-8 https://www.google.com/search?q=bangladesh+population&rlz=1C... 3. https://www.google.com/search?q=bangladesh+number+of+covid+deaths&rlz=1C1CHBD_enUS798US798&oq=bangladesh+number+of+covid+deaths&aqs=chrome..69i57.8348j0j7&sourceid=chrome&ie=UTF-8 https://www.google.com/search?q=bangladesh+number+of+covid+d...
lgats 6y ago
Trademark filings tell a similar story https://uspto.report/Search/Surgisphere https://uspto.report/Search/Surgisphere

61 points by valinsky 6y ago

Summary: Because of one report in the Lancet By a shady corporation called Surgisphere the WHO stopped all further research into hydroxychloroquine. The article makes a strong case the Surgisphere is shady and the study is badly made but other than timing (paper published 22, ban started 25th) the article does not provide any further information that the Surgisphere study was the source of the WHO decision. I quoted the unsupported accusation below. Instead of performing its own due diligence, the WHO immediately relied on an observational study cloaked in the reputation of the nearly 200-year old medical journal The Lancet.

scotty79 6y ago
And people scoff at reporters and journalists that they don't check their sources... Apparently almost nobody does. We have copy/paste information ecosphere.
- kiba 6y ago
  I did some checking of a lecture by an expert in virology, and I found an error. There are a few facts that I couldn't figure out if they were supported by other sources. It wasn't anything particularly intensive, just making sure the other sources said the same thing. But it was still a lot of work. For most people, learning is a lot of work in and itself.
  - smitty1e 6y ago
    The whole point of peer-reviewed journals is to be a noise filter. The scientific community purportedly polices itself.
    - acqq 6y ago
      The errors are indeed found, eventually, like this time. It's the expectancy of anything being completely "error free" from the start is simply false. Compare with https://news.ycombinator.com/item?id=23412482 https://news.ycombinator.com/item?id=23412482 Look at the all the infrastructure involved there, all the automatic tests that exist in the originating infrastructure, and still only 1) an independent fuzzing test by the company with the most computers in the world 2) the work of the leader of another project; bring "medial" attention to the existence of the problem. Nothing can be always perfect in the first try. In science specifically, the "intentional frauds" aren't the starting assumption of those who do review, that's why the frauds manage to pass the checks that do work most of the time. "Falsehood flies, and truth comes limping after it." Johnatan Swift (1667-1745)
    - 0d9eooo 6y ago
      I think this is kind of the tip of the iceberg, a general reflection on biomedical research and academics. Not that this is how most researchers act, but in the sense that it's not unusual for the field as a whole to encounter this. I mean, if this happens in the Lancet and with the WHO with something with worldwide significant consequences, what do people think is happening with stuff with less scrutiny? It also is a twist on all the calls for the importance of rigorous peer-reviewed studies, as if the preprint flood is just an example of why we need traditional academic structures. Here's a case where we needed the traditional academic structure and it failed miserably, even when prestigious institutions. The irony is that this is being caught, but outside the confines of the traditional peer review. That is, there is peer review, but in the general public of scientists. So which is better? It seems like open publication and transparency is key, more so than the traditional structures per se.
      - matthewdgreen 6y ago
        Peer review is like a spam filter. But just because something gets past your spam filter does not mean you should assume it's accurate, particularly when malicious and fraudulent behavior is involved. What appears to have happened here is that a rapid peer-review gave the benefit of the doubt to a purported data source, and then the community rapidly discovered that the data source was not behaving honestly. All of this occurred in a matter of a few weeks, during a time when peer-review is almost certainly moving even faster than normal due to the urgency of the pandemic.
- ListeningPie 6y ago
  To be fair, finding out all the information about Surgisphere probably took a lot time. Working on free projects myself there just comes a point where I've spent all the energy I was willing to put into a piece of work. Edit: Nevermind. The author of the article is the bit-coin investor how first tweeted about Chloroquine being a potential cure, https://www.wired.com/story/an-old-malaria-drug-may-fight-covid-19-and-silicon-valleys-into-it/ https://www.wired.com/story/an-old-malaria-drug-may-fight-co...
  - m0zg 6y ago
    He's not the only one pointing this out, though: https://www.theguardian.com/world/2020/may/29/covid-19-surgisphere-hydroxychloroquine-study-lancet-coronavirus-who-questioned-by-researchers-medical-professionals https://www.theguardian.com/world/2020/may/29/covid-19-surgi..., and the "study" is very easily verifiable as horseshit.
beaner 6y ago
> the article does not provide any further information that the Surgisphere study was the source of the WHO decision Genuine question: What else is it assumed WHO may have made the decision based on?
- seesawtron 6y ago
  When such a large study is published in a big Journal, claiming that a drug might have a negative impact on lives of people in the trials, the authorities will be considered stupid not to pay attention to it at all and continue the trials. That is exactly what they did: They paused their trials temporarily to look at the evidence of this study, examined it and found that it was indeed controversial and lacked solid data. Now they have resumed the trials again. The media sells it as a clickbait story "All trials STOPPED because of a flawed study" implying they were stopped PERMANENTLY to excite the general audience. For them it is entirely something that brings them huge revenue from ads when they write such "emotinally driven" stories, they do not care about the effects it has on the public in the long term by reducing faith in world health organizations.
  - Fjolsvith 6y ago
    > they do not care about the effects it has on the public in the long term by reducing faith in world health organizations. Or even the direct effects on the public in terms of healthcare.
- cycomanic 6y ago
  Well quite a few countries had stopped chloroquine use even before this study because they found little to no benifit, but significantly increased risks (not based on studies because they are difficult to do, but based on evidence by the treating doctors) , Sweden stopped IIRC mid april. IMO Germany and France also stopped sometime in May. Several smaller studies have also previously showed that chloroquine has increased risks and very little benefit (a number of them actually interrupted the studies because of the high risks). As much as I was hopeful when I first heard about it, I'm now very sceptical. It also doesn't help that Didier Raoult has a bit of a spotty past with ethical research conduct.
JorgeGT 6y ago
WHO director-general Tedros Adhanom Ghebreyesus directly pointed out the Lancet paper in a press conference as the reason to pause the trials and start a safety review. You can see the video here: https://time.com/5842264/hydroxychloroquine-testing-who/ https://time.com/5842264/hydroxychloroquine-testing-who/
- seesawtron 6y ago
  The trials are beginning again as per WHO decision now.
deleted 6y ago
[deleted]

seesawtron 6y ago

The trials are beginning again as per WHO decision now.

thaumaturgy 6y ago

This site's front page has got a lot of red flags on it. There's another treatment of this over on sciencemag: https://www.sciencemag.org/news/2020/06/mysterious-company-s-coronavirus-papers-top-medical-journals-may-be-unraveling https://www.sciencemag.org/news/2020/06/mysterious-company-s... This would be a pretty serious black eye for both The Lancet and the WHO, which has already been under scrutiny from multiple countries. However, it's worth pointing out that this wasn't the only study on HCQ. There have been others: https://www.cidrap.umn.edu/news-perspective/2020/05/studies-find-further-lack-covid-benefit-hydroxychloroquine https://www.cidrap.umn.edu/news-perspective/2020/05/studies-... If the Surgisphere observational study turns out to be erroneous, it could knock another leg out from under the WHO, and if it turns out to be fraudulent, everyone involved with it deserves jail time. It'll be terrible in either case that it resulted in a bunch of other trials and studies being canceled.

sampo 6y ago
Lots of hydroxychloroquine studies or preprints have been published, and they mostly observe that the effect is close to zero (in good or in bad), either because there is no effect, or the study size was too small to detect other than large effects. But this now infamous study was the only one to suggest a clear and large negative effect.
- Fjolsvith 6y ago
  So, if hydroxychloroquine is not effective, what explains an extremely low COVID death rate such as what Bangladesh (a country with endemic Malaria [1]) has? Bangladesh population: 161 million [2] Bangladesh COVID deaths: 811 [3] Bangladesh COVID death rate: 811/161m = 0.000503% 1. https://www.thelancet.com/journals/langlo/article/PIIS2214-109X(13)70176-1/fulltext#:~:text=Malaria%20is%20endemic%20in%2013,Program%20(NMCP)%20in%202007 https://www.thelancet.com/journals/langlo/article/PIIS2214-1.... 2. https://www.google.com/search?q=bangladesh+population&rlz=1C1CHBD_enUS798US798&oq=bangladesh+population&aqs=chrome..69i57.4075j0j7&sourceid=chrome&ie=UTF-8 https://www.google.com/search?q=bangladesh+population&rlz=1C... 3. https://www.google.com/search?q=bangladesh+number+of+covid+deaths&rlz=1C1CHBD_enUS798US798&oq=bangladesh+number+of+covid+deaths&aqs=chrome..69i57.8348j0j7&sourceid=chrome&ie=UTF-8 https://www.google.com/search?q=bangladesh+number+of+covid+d...

lgats 6y ago

Trademark filings tell a similar story https://uspto.report/Search/Surgisphere https://uspto.report/Search/Surgisphere