3 ms·

Rule of thumb: The longer and more complicated the article, the more reliable the content. NY Post article citing 'scientists' or 'a study' about how doing han

Rule of thumb: The longer and more complicated the article, the more reliable the content. NY Post article citing 'scientists' or 'a study' about how doing handstands increase your sexual capabilities in 250 words or less? Unreliable. Scientific paper linking biweekly handstands to increased cardiovascular health in 40-50 year-old Asian men in the Baltimore area, with raw data attached, in 12500 words? Reliable.

mgh2 5y ago
Not that simple Articles: Not really, a lot of SEO practitioners and expert storytellers use these tactics to their advantage. Scientific papers: You have to be technically trained a bit to see the nuances - ex: journal impact factor (citations), methodology (sample size, variables controlled, stat. significance...) affects results, analysis & interpretation; conflicts of interest with industry sponsors (often stated, but sometimes hidden), etc.
folbec 5y ago
With, however the caveat that scientific papers can be embarrassingly wrong, and the error can not be spotted for 17 years, and the paper can go on being widely believed and quoted many years after the full debunk (and the case I link is apparently an honest mistake, not fraud which may be obfuscated). https://economicsfromthetopdown.com/2022/04/08/the-dunning-kruger-effect-is-autocorrelation/ https://economicsfromthetopdown.com/2022/04/08/the-dunning-k...
- mgh2 5y ago
  That is the nuance of the scientific method in order to advance knowledge: it is not perfect, but the peer reviewed process is rigorous
  - folbec 5y ago
    In a perfect world, yes. Statistics is a HARD subject, specially when you get a juicy believable story in front of you. And a lack of credit to do confirmation research because for administrators "it is not new". Richard Feynman : We've learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature's phenomena will agree or they'll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven't tried to be very careful in this kind of work. And it's this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science. He is overly optimistic, it only works if the "other experimenters" get the money and time to check. A big part of modern science is very expensive, and controls aren't made.
nonameiguess 5y ago
You should actually be careful if you're really seeing the third line there. That's a form of p-hacking. Don't find a statistically significant effect in your entire study population? Fine, just segment it into 20 subsets, and in an average study, you can now expect to randomly find at least one population subset in which the effect is statistically significant.

mgh2 5y ago

Not that simple Articles: Not really, a lot of SEO practitioners and expert storytellers use these tactics to their advantage. Scientific papers: You have to be technically trained a bit to see the nuances - ex: journal impact factor (citations), methodology (sample size, variables controlled, stat. significance...) affects results, analysis & interpretation; conflicts of interest with industry sponsors (often stated, but sometimes hidden), etc.

folbec 5y ago

With, however the caveat that scientific papers can be embarrassingly wrong, and the error can not be spotted for 17 years, and the paper can go on being widely believed and quoted many years after the full debunk (and the case I link is apparently an honest mistake, not fraud which may be obfuscated). https://economicsfromthetopdown.com/2022/04/08/the-dunning-kruger-effect-is-autocorrelation/ https://economicsfromthetopdown.com/2022/04/08/the-dunning-k...

mgh2 5y ago
That is the nuance of the scientific method in order to advance knowledge: it is not perfect, but the peer reviewed process is rigorous
- folbec 5y ago
  In a perfect world, yes. Statistics is a HARD subject, specially when you get a juicy believable story in front of you. And a lack of credit to do confirmation research because for administrators "it is not new". Richard Feynman : We've learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature's phenomena will agree or they'll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven't tried to be very careful in this kind of work. And it's this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science. He is overly optimistic, it only works if the "other experimenters" get the money and time to check. A big part of modern science is very expensive, and controls aren't made.

nonameiguess 5y ago

You should actually be careful if you're really seeing the third line there. That's a form of p-hacking. Don't find a statistically significant effect in your entire study population? Fine, just segment it into 20 subsets, and in an average study, you can now expect to randomly find at least one population subset in which the effect is statistically significant.