5 ms·

This whole text is such a prime example of what's wrong with science reporting. In the very first sentence a causal relationship is claimed ("Frequent sauna ba

by hannob 10y ago

This whole text is such a prime example of what's wrong with science reporting. In the very first sentence a causal relationship is claimed ("Frequent sauna bathing can reduce the risk of dementia"). Yet the whole article doesn't live up to that claim, because it is very clear that the study purely relies on observational data and thus may very well just have found a confounder. However it doesn't even bother to mention or discuss that limitation.

kosei 10y ago
I felt the same way. If the authors did more to control for additional variables, then it was not clear in this article and that is fairly critical in terms of assessing the validity of the article's claim.
coldtea 10y ago
And this comment is a prime example of what's wrong with internet comments on scientific research. Usually all the basic concerns have been accounted for in the actual study. Whether an article in the mainstream press conveys that or not, is not important, since the article is just supposed to give a summary of the findings. It's seldom an issue of "those stupid PhDs and experts in their field, they didn't think of a confounder a random internet person can come up with in 5 seconds". Whoever wants to delve in can read the actual reports. And whoever wants to start the same practice (e.g. sauna here) would be foolish to do it based on one article anyway, they should consult with their doctor, and/or wait until a fuller consensus has been reached after several meta-studies (which can take decades).
- sambe 10y ago
  The parent comment is a valid criticism of the quality of reporting, especially in the context of the overall standard right now. Claiming a causal relationship is an incorrect summary if the study did not show this. This is a very serious problem. If the particular study did in fact show good evidence of causality, that's worth specific highlighting because it's not that common. The parent is not saying they've thought of some obvious confounder that the study authors didn't. They've emphasized the importance of unknown unknowns and correlation vs causation. Raising the standard of science journalism is a noble goal, if you ask me. In this case, the authors are not making as strong a claim as the summary does.
- htns 10y ago
  That's far too optimistic. I can't find the full text of this dementia study, but the one where they concluded sauna improves cardiovascular health barely gives any thought to the idea that perhaps men with a worse subjective assessment of their health are more likely to find it unwise to go to sauna too frequently. (In the closing paragraph they do a disclaimer "It is possible that underlying diagnosed or undiagnosed diseases may affect saunabathing habits, but our subgroup analyses according to various clinical characteristics were consistent.")
- hannob 10y ago
  > Usually all the basic concerns have been accounted for in the actual study. No. It's an observational study. If they'd account for the basic concerns they'd do an RCT. Yes, that's more work and takes longer. > It's seldom an issue of "those stupid PhDs and experts in their field, they didn't think of a confounder a random internet person can come up with in 5 seconds". No, you don't understand the problem. Confounders everyone can come up with can be controlled for (although it gets complicated the more you consider). But the major problem are unknown confounders. You can't control for things you don't know.
  - timr 10y ago
    "Observational study" does not mean "bad science". There are good and bad observational studies, and without reading the paper, you can't know which one this is. Moreover, your assertion that only random trials can be controlled is just completely wrong - there are plenty of ways to do controlled studies on observational data. You're engaged in the scientific version of the middlebrow dismissal. In health sciences, a lot of good work starts as well-controlled observational science. In fact, for a question like "does sauna use lead to a reduced risk of dementia over a human lifetime?", pretty much the only way you're ever getting the funding to do an "RCT" is to start with lots of observational evidence.
    - ptaipale 10y ago
      Yes, observational study does not mean bad science. But claiming or inferring that causation was found when it was not, that's bad science reporting.
      - timr 10y ago
        Never said anything about the reporting. The headline is certainly bad. But a bad article doesn't mean bad science.
- ptaipale 10y ago
  THe summary is actually wrong or at least severely premature, if it doesn't tell you whether this was about correlation without any real proof or analysis of causation.
- nolemurs 10y ago
  > Usually all the basic concerns have been accounted for in the actual study. Often. Yes. But I often go try to find the original publications when a study interests me, and I have yet to, once, find one where the causal claims made in the reporting article are actually supported by the research. Often the original paper explicitly mentions the fact that the claimed causal relationship can't be inferred. The claim here isn't that the science is bad. No one's saying the PhDs are stupid (there may be an argument there, but that's not what anyone here is saying). The complaint is that the science reporting is terrible. And it is. > And whoever wants to start the same practice (e.g. sauna here) would be foolish to do it based on one article anyway, they should consult with their doctor You are suffering under a severe delusion if you think the average doctor has the interest or competence to provide useful advice on this sort of issue. The average doctor will not have read any relevant studies, and has no interest in being informed about them, and I can say with some confidence that having a medical degree does not give him a magical ability to judge the usefulness of a practice without consulting any evidence. Science is great, but science reporting is terrible, and for that matter unreasoned faith in the academic establishment is anathema to real science.
- edblarney 10y ago
  "Usually all the basic concerns have been accounted for in the actual study." Fair enough ... but there could be simple and nuanced elements that they didn't or couldn't measure. For example: maybe Sauna dwellers are far more socially active than others? Social interaction is maybe they driver of longevity, the sauna may simply be a part of that process. Anyhow, just a crude example ... but I'd suggest it's very hard for researchers to rule out most such things.
- mangodrunk 10y ago
  >Usually all the basic concerns have been accounted for in the actual study. No, I don't think so: http://www.economist.com/news/leaders/21588069-scientific-research-has-changed-world-now-it-needs-change-itself-how-science-goes-wrong http://www.economist.com/news/leaders/21588069-scientific-re...
wwkeyboard 10y ago
At least they reference the original article, including the DOI. That being noteworthy is another indication of how bad most science reporting is.

by hannob 10y ago

kosei 10y ago

I felt the same way. If the authors did more to control for additional variables, then it was not clear in this article and that is fairly critical in terms of assessing the validity of the article's claim.

coldtea 10y ago

And this comment is a prime example of what's wrong with internet comments on scientific research. Usually all the basic concerns have been accounted for in the actual study. Whether an article in the mainstream press conveys that or not, is not important, since the article is just supposed to give a summary of the findings. It's seldom an issue of "those stupid PhDs and experts in their field, they didn't think of a confounder a random internet person can come up with in 5 seconds". Whoever wants to delve in can read the actual reports. And whoever wants to start the same practice (e.g. sauna here) would be foolish to do it based on one article anyway, they should consult with their doctor, and/or wait until a fuller consensus has been reached after several meta-studies (which can take decades).

sambe 10y ago
The parent comment is a valid criticism of the quality of reporting, especially in the context of the overall standard right now. Claiming a causal relationship is an incorrect summary if the study did not show this. This is a very serious problem. If the particular study did in fact show good evidence of causality, that's worth specific highlighting because it's not that common. The parent is not saying they've thought of some obvious confounder that the study authors didn't. They've emphasized the importance of unknown unknowns and correlation vs causation. Raising the standard of science journalism is a noble goal, if you ask me. In this case, the authors are not making as strong a claim as the summary does.
htns 10y ago
That's far too optimistic. I can't find the full text of this dementia study, but the one where they concluded sauna improves cardiovascular health barely gives any thought to the idea that perhaps men with a worse subjective assessment of their health are more likely to find it unwise to go to sauna too frequently. (In the closing paragraph they do a disclaimer "It is possible that underlying diagnosed or undiagnosed diseases may affect saunabathing habits, but our subgroup analyses according to various clinical characteristics were consistent.")
hannob 10y ago
> Usually all the basic concerns have been accounted for in the actual study. No. It's an observational study. If they'd account for the basic concerns they'd do an RCT. Yes, that's more work and takes longer. > It's seldom an issue of "those stupid PhDs and experts in their field, they didn't think of a confounder a random internet person can come up with in 5 seconds". No, you don't understand the problem. Confounders everyone can come up with can be controlled for (although it gets complicated the more you consider). But the major problem are unknown confounders. You can't control for things you don't know.
- timr 10y ago
  "Observational study" does not mean "bad science". There are good and bad observational studies, and without reading the paper, you can't know which one this is. Moreover, your assertion that only random trials can be controlled is just completely wrong - there are plenty of ways to do controlled studies on observational data. You're engaged in the scientific version of the middlebrow dismissal. In health sciences, a lot of good work starts as well-controlled observational science. In fact, for a question like "does sauna use lead to a reduced risk of dementia over a human lifetime?", pretty much the only way you're ever getting the funding to do an "RCT" is to start with lots of observational evidence.
  - ptaipale 10y ago
    Yes, observational study does not mean bad science. But claiming or inferring that causation was found when it was not, that's bad science reporting.
    - timr 10y ago
      Never said anything about the reporting. The headline is certainly bad. But a bad article doesn't mean bad science.
ptaipale 10y ago
THe summary is actually wrong or at least severely premature, if it doesn't tell you whether this was about correlation without any real proof or analysis of causation.
nolemurs 10y ago
> Usually all the basic concerns have been accounted for in the actual study. Often. Yes. But I often go try to find the original publications when a study interests me, and I have yet to, once, find one where the causal claims made in the reporting article are actually supported by the research. Often the original paper explicitly mentions the fact that the claimed causal relationship can't be inferred. The claim here isn't that the science is bad. No one's saying the PhDs are stupid (there may be an argument there, but that's not what anyone here is saying). The complaint is that the science reporting is terrible. And it is. > And whoever wants to start the same practice (e.g. sauna here) would be foolish to do it based on one article anyway, they should consult with their doctor You are suffering under a severe delusion if you think the average doctor has the interest or competence to provide useful advice on this sort of issue. The average doctor will not have read any relevant studies, and has no interest in being informed about them, and I can say with some confidence that having a medical degree does not give him a magical ability to judge the usefulness of a practice without consulting any evidence. Science is great, but science reporting is terrible, and for that matter unreasoned faith in the academic establishment is anathema to real science.
edblarney 10y ago
"Usually all the basic concerns have been accounted for in the actual study." Fair enough ... but there could be simple and nuanced elements that they didn't or couldn't measure. For example: maybe Sauna dwellers are far more socially active than others? Social interaction is maybe they driver of longevity, the sauna may simply be a part of that process. Anyhow, just a crude example ... but I'd suggest it's very hard for researchers to rule out most such things.
mangodrunk 10y ago
>Usually all the basic concerns have been accounted for in the actual study. No, I don't think so: http://www.economist.com/news/leaders/21588069-scientific-research-has-changed-world-now-it-needs-change-itself-how-science-goes-wrong http://www.economist.com/news/leaders/21588069-scientific-re...

wwkeyboard 10y ago

At least they reference the original article, including the DOI. That being noteworthy is another indication of how bad most science reporting is.