7 ms·

I'm going to make what is probably a controversial statement, which is that, in my experience, most meta-analyses are trash. A meta-analysis is usually grounde

I'm going to make what is probably a controversial statement, which is that, in my experience, most meta-analyses are trash. A meta-analysis is usually grounded on the assumption that an aggregation of multiple, well-conducted studies can provide significant information that may not be evident by taking each study individually. Essential to this assumption is the premise that those studies included in the meta-analysis be scientifically rigorous in themselves. Unfortunately, this is almost never the case. The great utility of the meta-analysis in the clinical literature is such that there have emerged a series of standards for how to conduct such analyses, such as controlling for bias, controlling for variability, etc. These seldom include actual assessment of the scientific quality of the studies, as this is harder to standardize. Consequently, there is a preponderance of meta-analyzes that come to conclusions that are simply not justified, because the studies themselves were not scientifically rigorous. I believe this applies to the situation of ivermectin, and it just so happens that this is the thrust of the point raised in the Nature article cited here.

tantalor 5y ago
FiveThirtyEight weights its averages by pollster accuracy/methodology. Next, we determine how much weight to give each poll in our average. First, polls conducted by pollsters with higher FiveThirtyEight pollster ratings — a letter grade measuring how accurate and methodologically sound pollsters are — are given more weight https://fivethirtyeight.com/features/how-were-tracking-joe-bidens-approval-rating/ https://fivethirtyeight.com/features/how-were-tracking-joe-b...
- nradov 5y ago
  There is no equivalent feedback mechanism in medical research. Other than looking for retracted papers, we don't have a reliable way to quantify whether a particular author is "correct" more often than another.
  - antman 5y ago
    The author’s Hindex could be an initial weighting factor
    - civilized 5y ago
      "Initial"? You haven't proposed any way to improve the weighting, so it would just be H index forever. We'd be baking in a cultural presumption that researchers who are more cited are more likely to be correct. A really dystopian prospect.
- ekianjo 5y ago
  FiveThirtyEight forecasts are very questionable themselves. Been completely wrong on multiple counts.
  - jakeinspace 5y ago
    Sure, but generally because the aggregated polls were systematically off. You may be thinking of other examples, but the 2016 US presidential election really wasn't that bad of a look for 538, they had Trump at reasonable odds going into election night. It's not like he was at 5%, 538 pegged him at 1/3 odds as polls opened if I remember correctly.
  - rapind 5y ago
    So is the weather forecast though. 10% chance of rain tomorrow means, yeah it could rain. Doesn't mean it isn't useful.
    - pid-1 5y ago
      You can check rain forecast results thousands of times and estimate how often they are correct. Assigning probabilities to events that happen infrequently (US pres. Elections, wars, end of the world...) is bullshit IMO.
    - ekianjo 5y ago
      It's on the aggregate of predictions that you can see the accuracy. If my weather app tells me hundred times there is 10% change of rain but it rains 50% of the time after 100 days, it's completely useless. Same for FiveThirtyEight. They have been massively wrong on multiple counts.
  - NikolaeVarius 5y ago
    538 is pretty explicit about what forecasts went wrong and how they will try and fix it in the future
- gjsman-1000 5y ago
  FiveThirtyEight is literally owned by Disney - and should be treated as such. How is FTE determining their "pollster rating"?
  - tantalor 5y ago
    > should be treated as such Ok > How is FTE determining their "pollster rating" https://projects.fivethirtyeight.com/pollster-ratings/ https://projects.fivethirtyeight.com/pollster-ratings/ https://fivethirtyeight.com/features/the-death-of-polling-is-greatly-exaggerated/ https://fivethirtyeight.com/features/the-death-of-polling-is...
Spellman 5y ago
Meta-analysis have a good place. But they also have known flaws/assumptions. Often it is impractical to perform large studies. Partially by logistics and often by funding. But if there are lots of smaller studies you can aggregate the data to check for overall significant results. But mainly they are helpful to give a "survey" of the current research instead of having to link to 10 different studies and hope someone else sorts out what the data says. But as you point out, there are two major flaws in the assumptions. First, that the scientific procedure is sound. Secondly, that the data is handled properly, and thus you can take the summary and back out the underlying data. Unfortunately trying to fix the first is really, really hard. The second is somewhat mitigatable. As the Nature article suggests, you could publish the underlying data (anonymized of course). This would help in two ways. First, the meta-analysis could check for confounding variables to control across all of the data. The second major one is it would help people spot fraudulent data. However, as anyone who handles datasets knows, publishing and wrangling data into a useable state from multiple sources is a serious pain in the neck. Plus a lot of concerns about how de-anonymized the data would be. As we've known, with enough metadata it can be used to identify individuals. And publicly publishing dais data would definitely allow for some serious sleuthing work to be done.
- trts 5y ago
  The problem I saw with the Ivermectin metastudies (which I did a fair amount of research on given that people close to me had decided to employ it as a prophylactic against Covid) was that the people who did the metanalyses were trained to read the results rather than grapple with the methodology and statistics. None of the study results were robust, but many of the study outcomes were positive. So they were counting up positive outcomes and saying that more often than not, it was evidence that Ivermectin was effective. This is a good means of generating a hypothesis, but a bad means of generating a conclusion. Most of these studies were not in any way controlled or comparable in their conditions, and some had a very low number of observations. It didn't seem plausible that the data could be aggregated together. It will be very interesting to see the results of the larger RCTs underway. Regardless of the outcome there _has_ to be a lesson for at least some people who arrived at fervent conclusions about what a miracle/scam this particular drug is. Unfortunately it seems unlikely that it will be a lesson that endures.
calibas 5y ago
It's so easy to abuse meta-analysis, you simply modify the inclusion criteria until you get the results you're looking for.
gnicholas 5y ago
Do researchers ever pre-register their studies when they do meta analyses? Pre-registration makes sense for the timing of doing actual studies, but not as much in the case of meta analyses (which presumably happen on a much faster timeline). But given what we've seen with p-hacking, it seems like it could be good if people pre-registered their meta analyses before crunching the numbers.
- mohammad_ali85 5y ago
  For systematic reviews there are one of two ways that I'm aware of: PROSPERO [1] and Cochrane[2]. Cochrane is slightly more rigorous for conducting a review of this type. Although you do declare what meta-analyses you intend to do, you can't be overly explicit as it is largely dependent on the number of studies that are returned in your search strategy. For example two studies is too small for a meta-analysis so you would do a narrative review (discuss what you found). [1]https://www.crd.york.ac.uk/prospero/ https://www.crd.york.ac.uk/prospero/ [2]https://www.cochranelibrary.com https://www.cochranelibrary.com
- nradov 5y ago
  Sure some researchers do pre-register their meta analysis studies on clinicaltrials.gov. It's generally optional. Of course there's nothing stopping a dishonest researcher from doing a little secret p-hacking before pre-registering.
- rscho 5y ago
  Good practices dictate that preregistration should always be done nowadays, including for meta-analysis. In practice though, this has very little impact. There is absolutely no guarantee that what's written in the paper is a faithful report of what really happened. There's nothing holding back dishonest (or, at that point I guess I should write 'standard') researcher from doing absolutely whatever they want.
- ekianjo 5y ago
  how would you know when they actually started the analysis?
derbOac 5y ago
So I'm someone who has published several meta-analyses of different forms, and written about meta-analysis as a topic. I won't disagree with you that there are many poorly conducted meta-analyses. However, I think there's many well-done meta-analyses as well, and most importantly maybe meta-analyses aren't really different from anything else in life: some are good, some are bad, and many are in between. One thing I've always argued is that meta-analyses have as a benefit a way of honing discussion around concrete specifics. The linked paper, for example, exists in part because there was a meta-analysis drawing attention to the literature at large. There's a decent chance that these studies would never be discussed if there wasn't a spotlight being pointed at the area. With reviews, what happens is people pick and choose studies anyway, or don't, and then come to some subjective conclusion that's based on some unclear process. Meta-analysis makes all of this clear, and forces everyone to be absolutely explicit (or as explicit as can be) about how they're coming to their conclusions. If there's something wrong with it, then you can point to the specifics of that instead of going back and forth. The problem with relying on definitive studies alone is that sometimes there will be more than one of them, or there won't be any definitive study, but many decently-done studies. Or the "definitive" study will have some controversial feature that doesn't clearly rule it out, but clouds the waters in a way that several smaller studies might draw attention to. Alternatively, there might be important heterogeneity across designs that illuminates moderating variables (like dose, or environmental context, or gender, or age, or whatever). This paper is about meta-analysis of summary statistics, which to me is kind of bringing up a red herring. Statistically speaking if you can calculate the right summary statistics, the results should be the same as having the raw data. Issues about irregularities in results apply to raw as well as summary statistics; it also seems unrealistic to expect raw data in every case, and journals don't apply that standard either (that is, journals don't expect reviewers to reanalyze the data from scratch). What's really needed is open data sharing, and scrutiny about studies that increases as the stakes of the results increase. I can speak to cases where I've been surprised at the state of the raw data, even in situations where the whole point of the study was to skeptically replicate a finding. Maybe for something like invermectin raw data analyses are appropriate. But it seems absurd to expect to throw out studies in the literature just because you don't have access to the raw data in every case.
- 5y ago
dekhn 5y ago
It's best to understand that the vast majority of literature is just faulty. Either it's technically wrong (experimental mistakes), or missing information (left out a step in the methods), or the conclusions are not supported the (valid) experiment. Learning how to filter literature is non-trivial and I don't think very many people do it successfully. I've found the best thing is to take a group of aggressive smart grad students and have them tear every paper apart until I'm convinced there are no basic technical errors, and the conclusion is supported by the evidence. Then and only then, would I care that the methods were incomplete because I'd be ready to run a replication experiment, and many of my replications failed because the methods section was not sufficient (missing steps, misleading instructions, challenging step).
- KennyBlanken 5y ago
  > I've found the best thing is to take a group of aggressive smart grad students and have them tear every paper apart That's a pretty passable simplification of meta-analysis studies.
whiddershins 5y ago
I agree with your general point but in all fairness part of doing a meta-analysis is weighting the studies included in it based on many of the factors you mention.
aabaker99 5y ago
I am also skeptical of meta-analyses. I will re-post some thoughts I have previously shared from John P.A. Ioannidis who is a professor of medicine and thoughtful critic of medical research. He often raises good points about trends in research and research ethics. His view is that meta-analyses are mass produced, redundant, misleading, and conflicted [1]! One criticism of meta-analyses in [1], using anti-depressants as a case study: "the results of several meta‐analytic evaluations that addressed the effectiveness of and/or tolerability for diverse antidepressants showed that their ranking of antidepressants was markedly different. These studies had been conducted by some of the best meta‐analysts in the world, all of them researchers with major contributions in the methods of meta‐analysis and extremely experienced in its conduct. However, among 12 considered drugs, paroxetine ranked anywhere from first to tenth best and sertraline ranked anywhere from second to tenth best." I like this quote because it highlights the conflict of interest and misleading-ness(or at least reproducibility problems) with meta-analyses. Antidepressants have a huge amount of primary research dedicated to them. They also have the attention of researchers experienced in meta-analysis. Yet, meta-analyses do not agree with each other (and in fact they strongly disagree with each other). [1] https://pubmed.ncbi.nlm.nih.gov/27620683/ https://pubmed.ncbi.nlm.nih.gov/27620683/
- KennyBlanken 5y ago
  > I will re-post some thoughts I have previously shared from John P.A. Ioannidis who is a professor of medicine and thoughtful critic of medical research https://en.wikipedia.org/wiki/John_Ioannidis#COVID-19 https://en.wikipedia.org/wiki/John_Ioannidis#COVID-19 > In an editorial on STAT published March 17, 2020, Ioannidis called the global response to the COVID-19 pandemic a "once-in-a-century evidence fiasco" and wrote that lockdowns were likely an overreaction to unreliable data.[14] He estimated that the coronavirus could cause 10,000 U.S. deaths if it infected 1% of the U.S. population, and argued that more data was needed to determine if the virus would spread more.[28][5][14] The virus in fact eventually infected far more people, and would cause more than 600,000 deaths in the U.S.[29][28][5] Marc Lipsitch, Director of the Center for Communicable Disease Dynamics at the Harvard T.H. Chan School of Public Health, objected to Ioannidis's characterization of the global response in a reply that was published on STAT the next day after Ioannidis's.[30] > Ioannidis widely promoted a study of which he had been co-author, "COVID-19 Antibody Seroprevalence in Santa Clara County, California", released as a preprint on April 17, 2020. It asserted that Santa Clara County's number of infections was between 50 and 85 times higher than the official count, putting the virus's fatality rate as low as 0.1% to 0.2%.[n 1][32][29] Ioannidis concluded from the study that the coronavirus is "not the apocalyptic problem we thought".[33] The message found favor with right-wing media outlets, but the paper drew criticism from a number of epidemiologists who said its testing was inaccurate and its methods were sloppy. Okay then. Nothing like spending a career picking apart people's research and then generating absolutely garbage research outside your field of expertise, that is widely criticized by people who are actually the experts in that field...as being inaccurate and sloppy. COVID hit, dude went all Don Quixote seeing conspiracies everywhere, and then generated a paper that suited his personal biases...
deeviant 5y ago
More over, they have a bit of a raid 0 problem. There more studies you include, the greater the chance you'll get a bad one. If you look at where these meta-analyses often fail, you'll often see just a few "Bad" studies that end up corrupting their results. You'd think the statistical tools they bring to bear in these would be resistant to this, but it appears to not be the case in several recent cases.

tantalor 5y ago

FiveThirtyEight weights its averages by pollster accuracy/methodology. Next, we determine how much weight to give each poll in our average. First, polls conducted by pollsters with higher FiveThirtyEight pollster ratings — a letter grade measuring how accurate and methodologically sound pollsters are — are given more weight https://fivethirtyeight.com/features/how-were-tracking-joe-bidens-approval-rating/ https://fivethirtyeight.com/features/how-were-tracking-joe-b...

nradov 5y ago
There is no equivalent feedback mechanism in medical research. Other than looking for retracted papers, we don't have a reliable way to quantify whether a particular author is "correct" more often than another.
- antman 5y ago
  The author’s Hindex could be an initial weighting factor
  - civilized 5y ago
    "Initial"? You haven't proposed any way to improve the weighting, so it would just be H index forever. We'd be baking in a cultural presumption that researchers who are more cited are more likely to be correct. A really dystopian prospect.
ekianjo 5y ago
FiveThirtyEight forecasts are very questionable themselves. Been completely wrong on multiple counts.
- jakeinspace 5y ago
  Sure, but generally because the aggregated polls were systematically off. You may be thinking of other examples, but the 2016 US presidential election really wasn't that bad of a look for 538, they had Trump at reasonable odds going into election night. It's not like he was at 5%, 538 pegged him at 1/3 odds as polls opened if I remember correctly.
- rapind 5y ago
  So is the weather forecast though. 10% chance of rain tomorrow means, yeah it could rain. Doesn't mean it isn't useful.
  - pid-1 5y ago
    You can check rain forecast results thousands of times and estimate how often they are correct. Assigning probabilities to events that happen infrequently (US pres. Elections, wars, end of the world...) is bullshit IMO.
  - ekianjo 5y ago
    It's on the aggregate of predictions that you can see the accuracy. If my weather app tells me hundred times there is 10% change of rain but it rains 50% of the time after 100 days, it's completely useless. Same for FiveThirtyEight. They have been massively wrong on multiple counts.
- NikolaeVarius 5y ago
  538 is pretty explicit about what forecasts went wrong and how they will try and fix it in the future
gjsman-1000 5y ago
FiveThirtyEight is literally owned by Disney - and should be treated as such. How is FTE determining their "pollster rating"?
- tantalor 5y ago
  > should be treated as such Ok > How is FTE determining their "pollster rating" https://projects.fivethirtyeight.com/pollster-ratings/ https://projects.fivethirtyeight.com/pollster-ratings/ https://fivethirtyeight.com/features/the-death-of-polling-is-greatly-exaggerated/ https://fivethirtyeight.com/features/the-death-of-polling-is...

Spellman 5y ago

Meta-analysis have a good place. But they also have known flaws/assumptions. Often it is impractical to perform large studies. Partially by logistics and often by funding. But if there are lots of smaller studies you can aggregate the data to check for overall significant results. But mainly they are helpful to give a "survey" of the current research instead of having to link to 10 different studies and hope someone else sorts out what the data says. But as you point out, there are two major flaws in the assumptions. First, that the scientific procedure is sound. Secondly, that the data is handled properly, and thus you can take the summary and back out the underlying data. Unfortunately trying to fix the first is really, really hard. The second is somewhat mitigatable. As the Nature article suggests, you could publish the underlying data (anonymized of course). This would help in two ways. First, the meta-analysis could check for confounding variables to control across all of the data. The second major one is it would help people spot fraudulent data. However, as anyone who handles datasets knows, publishing and wrangling data into a useable state from multiple sources is a serious pain in the neck. Plus a lot of concerns about how de-anonymized the data would be. As we've known, with enough metadata it can be used to identify individuals. And publicly publishing dais data would definitely allow for some serious sleuthing work to be done.

trts 5y ago
The problem I saw with the Ivermectin metastudies (which I did a fair amount of research on given that people close to me had decided to employ it as a prophylactic against Covid) was that the people who did the metanalyses were trained to read the results rather than grapple with the methodology and statistics. None of the study results were robust, but many of the study outcomes were positive. So they were counting up positive outcomes and saying that more often than not, it was evidence that Ivermectin was effective. This is a good means of generating a hypothesis, but a bad means of generating a conclusion. Most of these studies were not in any way controlled or comparable in their conditions, and some had a very low number of observations. It didn't seem plausible that the data could be aggregated together. It will be very interesting to see the results of the larger RCTs underway. Regardless of the outcome there _has_ to be a lesson for at least some people who arrived at fervent conclusions about what a miracle/scam this particular drug is. Unfortunately it seems unlikely that it will be a lesson that endures.

calibas 5y ago

It's so easy to abuse meta-analysis, you simply modify the inclusion criteria until you get the results you're looking for.

gnicholas 5y ago

Do researchers ever pre-register their studies when they do meta analyses? Pre-registration makes sense for the timing of doing actual studies, but not as much in the case of meta analyses (which presumably happen on a much faster timeline). But given what we've seen with p-hacking, it seems like it could be good if people pre-registered their meta analyses before crunching the numbers.

mohammad_ali85 5y ago
For systematic reviews there are one of two ways that I'm aware of: PROSPERO [1] and Cochrane[2]. Cochrane is slightly more rigorous for conducting a review of this type. Although you do declare what meta-analyses you intend to do, you can't be overly explicit as it is largely dependent on the number of studies that are returned in your search strategy. For example two studies is too small for a meta-analysis so you would do a narrative review (discuss what you found). [1]https://www.crd.york.ac.uk/prospero/ https://www.crd.york.ac.uk/prospero/ [2]https://www.cochranelibrary.com https://www.cochranelibrary.com
nradov 5y ago
Sure some researchers do pre-register their meta analysis studies on clinicaltrials.gov. It's generally optional. Of course there's nothing stopping a dishonest researcher from doing a little secret p-hacking before pre-registering.
rscho 5y ago
Good practices dictate that preregistration should always be done nowadays, including for meta-analysis. In practice though, this has very little impact. There is absolutely no guarantee that what's written in the paper is a faithful report of what really happened. There's nothing holding back dishonest (or, at that point I guess I should write 'standard') researcher from doing absolutely whatever they want.
ekianjo 5y ago
how would you know when they actually started the analysis?

derbOac 5y ago

So I'm someone who has published several meta-analyses of different forms, and written about meta-analysis as a topic. I won't disagree with you that there are many poorly conducted meta-analyses. However, I think there's many well-done meta-analyses as well, and most importantly maybe meta-analyses aren't really different from anything else in life: some are good, some are bad, and many are in between. One thing I've always argued is that meta-analyses have as a benefit a way of honing discussion around concrete specifics. The linked paper, for example, exists in part because there was a meta-analysis drawing attention to the literature at large. There's a decent chance that these studies would never be discussed if there wasn't a spotlight being pointed at the area. With reviews, what happens is people pick and choose studies anyway, or don't, and then come to some subjective conclusion that's based on some unclear process. Meta-analysis makes all of this clear, and forces everyone to be absolutely explicit (or as explicit as can be) about how they're coming to their conclusions. If there's something wrong with it, then you can point to the specifics of that instead of going back and forth. The problem with relying on definitive studies alone is that sometimes there will be more than one of them, or there won't be any definitive study, but many decently-done studies. Or the "definitive" study will have some controversial feature that doesn't clearly rule it out, but clouds the waters in a way that several smaller studies might draw attention to. Alternatively, there might be important heterogeneity across designs that illuminates moderating variables (like dose, or environmental context, or gender, or age, or whatever). This paper is about meta-analysis of summary statistics, which to me is kind of bringing up a red herring. Statistically speaking if you can calculate the right summary statistics, the results should be the same as having the raw data. Issues about irregularities in results apply to raw as well as summary statistics; it also seems unrealistic to expect raw data in every case, and journals don't apply that standard either (that is, journals don't expect reviewers to reanalyze the data from scratch). What's really needed is open data sharing, and scrutiny about studies that increases as the stakes of the results increase. I can speak to cases where I've been surprised at the state of the raw data, even in situations where the whole point of the study was to skeptically replicate a finding. Maybe for something like invermectin raw data analyses are appropriate. But it seems absurd to expect to throw out studies in the literature just because you don't have access to the raw data in every case.

5y ago

dekhn 5y ago

It's best to understand that the vast majority of literature is just faulty. Either it's technically wrong (experimental mistakes), or missing information (left out a step in the methods), or the conclusions are not supported the (valid) experiment. Learning how to filter literature is non-trivial and I don't think very many people do it successfully. I've found the best thing is to take a group of aggressive smart grad students and have them tear every paper apart until I'm convinced there are no basic technical errors, and the conclusion is supported by the evidence. Then and only then, would I care that the methods were incomplete because I'd be ready to run a replication experiment, and many of my replications failed because the methods section was not sufficient (missing steps, misleading instructions, challenging step).

KennyBlanken 5y ago
> I've found the best thing is to take a group of aggressive smart grad students and have them tear every paper apart That's a pretty passable simplification of meta-analysis studies.

whiddershins 5y ago

I agree with your general point but in all fairness part of doing a meta-analysis is weighting the studies included in it based on many of the factors you mention.

aabaker99 5y ago

I am also skeptical of meta-analyses. I will re-post some thoughts I have previously shared from John P.A. Ioannidis who is a professor of medicine and thoughtful critic of medical research. He often raises good points about trends in research and research ethics. His view is that meta-analyses are mass produced, redundant, misleading, and conflicted [1]! One criticism of meta-analyses in [1], using anti-depressants as a case study: "the results of several meta‐analytic evaluations that addressed the effectiveness of and/or tolerability for diverse antidepressants showed that their ranking of antidepressants was markedly different. These studies had been conducted by some of the best meta‐analysts in the world, all of them researchers with major contributions in the methods of meta‐analysis and extremely experienced in its conduct. However, among 12 considered drugs, paroxetine ranked anywhere from first to tenth best and sertraline ranked anywhere from second to tenth best." I like this quote because it highlights the conflict of interest and misleading-ness(or at least reproducibility problems) with meta-analyses. Antidepressants have a huge amount of primary research dedicated to them. They also have the attention of researchers experienced in meta-analysis. Yet, meta-analyses do not agree with each other (and in fact they strongly disagree with each other). [1] https://pubmed.ncbi.nlm.nih.gov/27620683/ https://pubmed.ncbi.nlm.nih.gov/27620683/

KennyBlanken 5y ago
> I will re-post some thoughts I have previously shared from John P.A. Ioannidis who is a professor of medicine and thoughtful critic of medical research https://en.wikipedia.org/wiki/John_Ioannidis#COVID-19 https://en.wikipedia.org/wiki/John_Ioannidis#COVID-19 > In an editorial on STAT published March 17, 2020, Ioannidis called the global response to the COVID-19 pandemic a "once-in-a-century evidence fiasco" and wrote that lockdowns were likely an overreaction to unreliable data.[14] He estimated that the coronavirus could cause 10,000 U.S. deaths if it infected 1% of the U.S. population, and argued that more data was needed to determine if the virus would spread more.[28][5][14] The virus in fact eventually infected far more people, and would cause more than 600,000 deaths in the U.S.[29][28][5] Marc Lipsitch, Director of the Center for Communicable Disease Dynamics at the Harvard T.H. Chan School of Public Health, objected to Ioannidis's characterization of the global response in a reply that was published on STAT the next day after Ioannidis's.[30] > Ioannidis widely promoted a study of which he had been co-author, "COVID-19 Antibody Seroprevalence in Santa Clara County, California", released as a preprint on April 17, 2020. It asserted that Santa Clara County's number of infections was between 50 and 85 times higher than the official count, putting the virus's fatality rate as low as 0.1% to 0.2%.[n 1][32][29] Ioannidis concluded from the study that the coronavirus is "not the apocalyptic problem we thought".[33] The message found favor with right-wing media outlets, but the paper drew criticism from a number of epidemiologists who said its testing was inaccurate and its methods were sloppy. Okay then. Nothing like spending a career picking apart people's research and then generating absolutely garbage research outside your field of expertise, that is widely criticized by people who are actually the experts in that field...as being inaccurate and sloppy. COVID hit, dude went all Don Quixote seeing conspiracies everywhere, and then generated a paper that suited his personal biases...

deeviant 5y ago

More over, they have a bit of a raid 0 problem. There more studies you include, the greater the chance you'll get a bad one. If you look at where these meta-analyses often fail, you'll often see just a few "Bad" studies that end up corrupting their results. You'd think the statistical tools they bring to bear in these would be resistant to this, but it appears to not be the case in several recent cases.