3 ms·

I had a look, I'm not an expert in the area by any means, but at a reasonable glance it looks like a good study to me. Of course, it could be a giant fiddle (al

I had a look, I'm not an expert in the area by any means, but at a reasonable glance it looks like a good study to me. Of course, it could be a giant fiddle (almost any experiment can be), but I try hard not to automatically assume studies which say what I like are good, and ones that say what I don't like are bad. I'm sure there will be more research in this area, now that there is not the threat of jail if people report on their usage.

dekhn 9y ago
Tell ya what. As a homework assignment, can you do enough stats research to do a reasonable critique of the methodology?
- josinalvo 9y ago
  I am very interested in the answer key for this homework... In all seriousness, I'd love a breakdown, and references for further study (on how to read medical papers, or scientific papers in general)
  - dekhn 9y ago
    I suggest starting with "Primer on Biostatistics" by Glantz https://www.amazon.com/Primer-Biostatistics-Seventh-Glantz-Paperback/dp/0071781501 https://www.amazon.com/Primer-Biostatistics-Seventh-Glantz-P... which contains the basics you need to know to do analysis of stats analysis. I haven't read it, but you'll probably want a book like https://www.amazon.com/Design-Studies-Medical-Research/dp/0470844957 https://www.amazon.com/Design-Studies-Medical-Research/dp/04... or https://www.amazon.com/Study-Design-Statistical-Analysis-Clinicians/dp/0521534070 https://www.amazon.com/Study-Design-Statistical-Analysis-Cli... realistically you need at least a master's degree in a quantitative field to get the time and knowledge required to evaluate studies like this. I see a lot of armchair "scientists" trying to evaluate studies, it's very common to see basic mistakes in evaluating papers.
btilly 9y ago
What did you have a look at? The article doesn't link to the study, so I'm going to just go off of questions that the article left me with. First of all, what period did they look at, how many they expected to die, and how many of those did die. 1200 people sounds like a lot, but if their model said that 2.5 people should have died but 9 actually did, your results are extreme but the p-value isn't very high. Further, do we trust the model that they had for how many should have died? They say what they did. But there are a lot of ways to tweak such a model, and p-hacking is naturally going to lead them to tweak it in ways that make the results look more significant. Next, even if all of this is true, what is cause versus effect. Are people using marijuana to deal with life stress? If so, is the stress the real risk factor here? There are a lot of things that can go wrong in a study, and most published studies are wrong. Until I see the methodology checked and the result replicated with a high reported p-value, I'm not trusting this study. Not because I don't like the outcome, but because that's simply how unreliable this type of science is.

dekhn 9y ago

Tell ya what. As a homework assignment, can you do enough stats research to do a reasonable critique of the methodology?

josinalvo 9y ago
I am very interested in the answer key for this homework... In all seriousness, I'd love a breakdown, and references for further study (on how to read medical papers, or scientific papers in general)
- dekhn 9y ago
  I suggest starting with "Primer on Biostatistics" by Glantz https://www.amazon.com/Primer-Biostatistics-Seventh-Glantz-Paperback/dp/0071781501 https://www.amazon.com/Primer-Biostatistics-Seventh-Glantz-P... which contains the basics you need to know to do analysis of stats analysis. I haven't read it, but you'll probably want a book like https://www.amazon.com/Design-Studies-Medical-Research/dp/0470844957 https://www.amazon.com/Design-Studies-Medical-Research/dp/04... or https://www.amazon.com/Study-Design-Statistical-Analysis-Clinicians/dp/0521534070 https://www.amazon.com/Study-Design-Statistical-Analysis-Cli... realistically you need at least a master's degree in a quantitative field to get the time and knowledge required to evaluate studies like this. I see a lot of armchair "scientists" trying to evaluate studies, it's very common to see basic mistakes in evaluating papers.

btilly 9y ago

What did you have a look at? The article doesn't link to the study, so I'm going to just go off of questions that the article left me with. First of all, what period did they look at, how many they expected to die, and how many of those did die. 1200 people sounds like a lot, but if their model said that 2.5 people should have died but 9 actually did, your results are extreme but the p-value isn't very high. Further, do we trust the model that they had for how many should have died? They say what they did. But there are a lot of ways to tweak such a model, and p-hacking is naturally going to lead them to tweak it in ways that make the results look more significant. Next, even if all of this is true, what is cause versus effect. Are people using marijuana to deal with life stress? If so, is the stress the real risk factor here? There are a lot of things that can go wrong in a study, and most published studies are wrong. Until I see the methodology checked and the result replicated with a high reported p-value, I'm not trusting this study. Not because I don't like the outcome, but because that's simply how unreliable this type of science is.