5 ms·

This may seem like bad news, but as a grad student studying evidence synthesis I can say that there is a silver lining here. The quality of studies in systemati

This may seem like bad news, but as a grad student studying evidence synthesis I can say that there is a silver lining here. The quality of studies in systematic inputs is low and even Cochrane's reviews have been critiqued for missing things. It's a big factory that hasn't fundamentally innovated how research is synthesized. There are newer forms of synthesis like realist reviews, meta-modeling, and model-driven meta-analyses (those are just a few examples). Hopefully this results in some innovation at the conceptual and financial levels. For those who lost their income, they have transferrable skills that would be welcomed in healthtech companies, other governmental groups, and academic positions. It's really not as bad as it seems.

haldujai 3y ago
> The quality of studies in systematic inputs is low and even Cochrane's reviews have been critiqued for missing things. Criticism is good, expecting perfection is not. Quality issues are a problem of the underlying research rather than the review process. High quality reviews, like Cochrane, assess risk of bias which can limit the strength of conclusion they make where the underlying studies are poor quality. > There are newer forms of synthesis like realist reviews, meta-modeling, and model-driven meta-analyses (those are just a few examples). All of these answer different questions/have different use cases than a SR and are much harder to do while maintaining comparable quality and risk of bias. What's your gripe with high quality SRs following standards like PRISMA, STARD and QUADAS-2?
cyrillite 3y ago
This is an insane ask, but I would love to hear more about various reviewing and synthesis styles, their strengths and weaknesses, which are particularly promising and why, etc. you’re welcome to link me to material, but I’d also appreciate a brief overview. I have nothing to offer other than thinking that this is a super cool area of study and I’d love to hear more
- thedailymail 3y ago
  This site provides a concise description of many review types: https://guides.mclibrary.duke.edu/sysreview/types https://guides.mclibrary.duke.edu/sysreview/types
aaron695 3y ago
[dead]
ttpphd 3y ago
Totally agree. Cochrane is a dinosaur and can do more harm than good.
TheBlight 3y ago
I don't remember hearing anything about Cochrane in mainstream news until they put out an unfavorable review on the effectiveness of mask usage during covid 19.
- ggm 3y ago
  A very unfortunate, badly run meta analysis as I understand it which fundamentally undermined trust in their governance because it was so easily critiqued. I haven't seen a robust defence of that one.
  - csours 3y ago
    My opinion (feel free to have another one) is that it points to the need for training and more comfortable masks. The studies weren't WRONG per se, but they are very hard to interpret. My whacky opinion: human facing sciences (medicine, psychology, diet, sociology, economics, etc.) have a huge problem with solutions that require motivation to succeed. In other words, if you are motivated, you can make masks work. If you are motivated, you can make intermittent fasting work. Same for EVs, Solar Panels, etc.
  - TheBlight 3y ago
    I haven't seen compelling critiques of it[0]. Curious to read them. Their findings: On medical/surgical vs. nothing: "Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)" On N95/P2 respirators vs. medical/surgical: "The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence)." [0] https://www.cochranelibrary.com/web/cochrane/content?templateType=full&urlTitle=%2Fcdsr%2Fdoi%2F10.1002%2F14651858.CD006207.pub6&doi=10.1002%2F14651858.CD006207.pub6&type=cdsr&contentLanguage= https://www.cochranelibrary.com/web/cochrane/content?templat...
    - 3y ago

haldujai 3y ago

> The quality of studies in systematic inputs is low and even Cochrane's reviews have been critiqued for missing things. Criticism is good, expecting perfection is not. Quality issues are a problem of the underlying research rather than the review process. High quality reviews, like Cochrane, assess risk of bias which can limit the strength of conclusion they make where the underlying studies are poor quality. > There are newer forms of synthesis like realist reviews, meta-modeling, and model-driven meta-analyses (those are just a few examples). All of these answer different questions/have different use cases than a SR and are much harder to do while maintaining comparable quality and risk of bias. What's your gripe with high quality SRs following standards like PRISMA, STARD and QUADAS-2?

cyrillite 3y ago

This is an insane ask, but I would love to hear more about various reviewing and synthesis styles, their strengths and weaknesses, which are particularly promising and why, etc. you’re welcome to link me to material, but I’d also appreciate a brief overview. I have nothing to offer other than thinking that this is a super cool area of study and I’d love to hear more

thedailymail 3y ago
This site provides a concise description of many review types: https://guides.mclibrary.duke.edu/sysreview/types https://guides.mclibrary.duke.edu/sysreview/types

aaron695 3y ago

[dead]

ttpphd 3y ago

Totally agree. Cochrane is a dinosaur and can do more harm than good.

TheBlight 3y ago

I don't remember hearing anything about Cochrane in mainstream news until they put out an unfavorable review on the effectiveness of mask usage during covid 19.

ggm 3y ago
A very unfortunate, badly run meta analysis as I understand it which fundamentally undermined trust in their governance because it was so easily critiqued. I haven't seen a robust defence of that one.
- csours 3y ago
  My opinion (feel free to have another one) is that it points to the need for training and more comfortable masks. The studies weren't WRONG per se, but they are very hard to interpret. My whacky opinion: human facing sciences (medicine, psychology, diet, sociology, economics, etc.) have a huge problem with solutions that require motivation to succeed. In other words, if you are motivated, you can make masks work. If you are motivated, you can make intermittent fasting work. Same for EVs, Solar Panels, etc.
- TheBlight 3y ago
  I haven't seen compelling critiques of it[0]. Curious to read them. Their findings: On medical/surgical vs. nothing: "Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)" On N95/P2 respirators vs. medical/surgical: "The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence)." [0] https://www.cochranelibrary.com/web/cochrane/content?templateType=full&urlTitle=%2Fcdsr%2Fdoi%2F10.1002%2F14651858.CD006207.pub6&doi=10.1002%2F14651858.CD006207.pub6&type=cdsr&contentLanguage= https://www.cochranelibrary.com/web/cochrane/content?templat...
  - 3y ago