3 ms·

That's not a shallow dismissal. Fundamentally, what I read was a cohort size of 11. 1 died. No control group. No double blind (as no control group). There's

by hpcjoe 7y ago

That's not a shallow dismissal. Fundamentally, what I read was a cohort size of 11. 1 died. No control group. No double blind (as no control group). There's no aspect of this that would be statistically distinguishable from noise. Then again, the original study suffered from this as well. n=11 may be succinct, but it is not shallow. There is a world of statistical DoE, analytics, etc. behind this. My concern is the rush to headlines about studies without any real resolving power. The bad part of this is that we don't have time to do proper studies (this takes months and many people to make work). The n=11 comment actually infers this issue. Sort of like -1 = exp(sqrt(-1)*pi), you don't need many letters to convey a tremendous amount of meaning.

elcomet 7y ago
It is shallow because it doesn't teach anything to people that are not already familiar with this kind of work. Something not shallow would be like "only eleven people participated in this study, this is too little to get a clear result that's not due to luck."
- dang 7y ago
  That would be just as shallow. The issue isn't brevity, it's genericness. Are we really making the assumption that every study with a small sample is worthless, and that our reflexes know better than the people who worked on it? Obviously we need to do better than that. Generic discussion is shallow discussion (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20generic%20discussion&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...). Here's a comparable comment that at least begins to engage with the specifics: https://news.ycombinator.com/item?id=22798031 https://news.ycombinator.com/item?id=22798031. I don't know if it's a good point, but it's at least actually about this particular study. We wouldn't moderate such a comment.
  - klmadfejno 7y ago
    I'm not sure I agree with this. Would you feel the same way if the sample size was 1? 5? 11 is very small, and useful only for evaluating the presence of very very prominent effect sizes, regardless of domain. A comment saying n=11 is not a novel insight, but I believe it is useful, similar to people who add "in mice" to articles whose headlines might imply the test was on humans. I appreciate those comments. In this case I think it is fair to conclude that the headline "Severe Covid-19 Cases Don't Respond to..." is an accurate takeaway purely because the sample size is so small. With this sample size you can, at best, conclude that these combinations of drugs don't reliably cure the disease in severe cases, which is a very different idea.
    - dang 7y ago
      Why not? It depends on the study. I agree with you about misleading headlines and the mice thing. It's an interesting point to compare. While I have you: could you please stop creating accounts for every few comments you post? We ban accounts that do that, and in fact this account was banned. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. You needn't use your real name, of course, but for HN to be a community, users need some identity for others to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix&page=0&query=by:dang%20community%20identity https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
      - klmadfejno 7y ago
        Yes
        dang 7y ago
        Appreciated! (I only just saw this.)
dang 7y ago
Somehow the hivemind decided a few years ago (it would be interesting to trace how and when) that every study with a small sample size is worthless and merely mentioning sample size is enough to cancel it, and whoever is first to bring this up in a thread wins. It's a gotcha response. Obviously there are statistical concerns with small samples. Equally obviously, not every such study is worthless and the authors are not all idiots who knows less than any internet commenter. So to get substantive discussion, we need to engage with the specifics of a particular study, not post a generic dismissal. "n = 11?" looks specific but it isn't. It's a template instantiation, as stock as "Correlation is not causation". The answer to "the rush to headlines about studies" is not the opposite reflex, it's to slow down and reflect, and then maybe post if one has something thoughtful to say. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20reflective%20reflex&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... I don't mean to pick on the GP commenter. Maybe they had something else in mind. The issue is how such a comment plugs into a known shallow-discussion dynamic. The problem is at the group level, not the individual, and we all suffer from this reflexiveness, so none of us gets to feel smug.

by hpcjoe 7y ago

elcomet 7y ago

It is shallow because it doesn't teach anything to people that are not already familiar with this kind of work. Something not shallow would be like "only eleven people participated in this study, this is too little to get a clear result that's not due to luck."

dang 7y ago
That would be just as shallow. The issue isn't brevity, it's genericness. Are we really making the assumption that every study with a small sample is worthless, and that our reflexes know better than the people who worked on it? Obviously we need to do better than that. Generic discussion is shallow discussion (https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=by%3Adang%20generic%20discussion&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...). Here's a comparable comment that at least begins to engage with the specifics: https://news.ycombinator.com/item?id=22798031 https://news.ycombinator.com/item?id=22798031. I don't know if it's a good point, but it's at least actually about this particular study. We wouldn't moderate such a comment.
- klmadfejno 7y ago
  I'm not sure I agree with this. Would you feel the same way if the sample size was 1? 5? 11 is very small, and useful only for evaluating the presence of very very prominent effect sizes, regardless of domain. A comment saying n=11 is not a novel insight, but I believe it is useful, similar to people who add "in mice" to articles whose headlines might imply the test was on humans. I appreciate those comments. In this case I think it is fair to conclude that the headline "Severe Covid-19 Cases Don't Respond to..." is an accurate takeaway purely because the sample size is so small. With this sample size you can, at best, conclude that these combinations of drugs don't reliably cure the disease in severe cases, which is a very different idea.
  - dang 7y ago
    Why not? It depends on the study. I agree with you about misleading headlines and the mice thing. It's an interesting point to compare. While I have you: could you please stop creating accounts for every few comments you post? We ban accounts that do that, and in fact this account was banned. This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. You needn't use your real name, of course, but for HN to be a community, users need some identity for others to relate to. Otherwise we may as well have no usernames and no community, and that would be a different kind of forum. https://hn.algolia.com/?sort=byDate&dateRange=all&type=comment&storyText=false&prefix&page=0&query=by:dang%20community%20identity https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
    - klmadfejno 7y ago
      Yes
      - dang 7y ago
        Appreciated! (I only just saw this.)

dang 7y ago

Somehow the hivemind decided a few years ago (it would be interesting to trace how and when) that every study with a small sample size is worthless and merely mentioning sample size is enough to cancel it, and whoever is first to bring this up in a thread wins. It's a gotcha response. Obviously there are statistical concerns with small samples. Equally obviously, not every such study is worthless and the authors are not all idiots who knows less than any internet commenter. So to get substantive discussion, we need to engage with the specifics of a particular study, not post a generic dismissal. "n = 11?" looks specific but it isn't. It's a template instantiation, as stock as "Correlation is not causation". The answer to "the rush to headlines about studies" is not the opposite reflex, it's to slow down and reflect, and then maybe post if one has something thoughtful to say. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=by%3Adang%20reflective%20reflex&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... I don't mean to pick on the GP commenter. Maybe they had something else in mind. The issue is how such a comment plugs into a known shallow-discussion dynamic. The problem is at the group level, not the individual, and we all suffer from this reflexiveness, so none of us gets to feel smug.