5 ms·

One thing that comes up again and again is understanding significances, errors and such. This stuff is second nature to a good (and honest) physicist coming out

by Maro 15y ago

One thing that comes up again and again is understanding significances, errors and such. This stuff is second nature to a good (and honest) physicist coming out of University, even before he becomes a "pro". In my experience, to a doctor, even a good one, _at first_ it's just an annoying thing they have to do to get published and they don't really understand the significance of significances. (LOL.) I would argue that this type of sensitivity to understanding data should be important if you're running a country. You can't even say that there are underlings for this, because this data is so important, and the possibility of underlings skewing it for whatever reason so sizable, that I certainly would want to have some understanding and feeling for the data before doing a press conference about it or using it to guide my lawmaking.

barry-cotter 15y ago
In my experience, to a doctor, even a good one, _at first_ it's just an annoying thing they have to do to get published and they don't really understand the significance of significances. There is an entire medical discipline that knowingly uses shitty statistics[0]. Then there are the ones who unknowingly use shitty statistics[1]. Absent someone with a qualification that is impossible to get without learning real statistics as a co-author the default when reading medical research should be that the result is either overblown or wrong. Biologists aren't much better than medical doctors, to my knowledge. [0]http://lesswrong.com/lw/72f/why_epidemiology_will_not_correct_itself/ http://lesswrong.com/lw/72f/why_epidemiology_will_not_correc... [1]http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/ http://www.theatlantic.com/magazine/archive/2010/11/lies-dam...
- Maro 15y ago
  Just recently Science published, and NASA had a PR hoopla about a bogus paper about arsenic life, after it went through the usual Science peer "review" process. The paper was dicredited by numerous Comment papers for outrageously bad statistics and bad methodology, which Science actually published. For reasons I don't really understand, Science did not redact the paper and called the ensuing outrage normal scientific discourse, even though I think they would have benefited from a mae culpa moment regarding their peer review.
dhimes 15y ago
Also important, and related, is my wife claims it took a year or so of fellowship training before she learned how to ask a scientific question. She's an MD who did a research fellowship at NIH; I'm a physicist.
gcp 15y ago
You have my vote. Lots of policy decisions are made based on statistics. Yet they are easy to fool someone with if he or she does not understand them properly enough. In my experience, to a doctor, even a good one, _at first_ it's just an annoying thing they have to do to get published and they don't really understand the significance of significances. Slightly disappointing. I'd have expected doctors to have an understanding of placebo effects, and the field of medicine is one where you regularly hear (even outside medicine) about studies being discredited afterwards for wrong/improper controls, positive results being unreproducible etc. I don't remember the last time I heard a comp sci paper being withdrawn for bad statistics. (They tend to have none, even when they should)
- Maro 15y ago
  I'm not saying doctors doing research never learn this stuff, just that they come out of University without this knowledge. (I certainly don't know enough about medical research literature to make a statement like this.) Curiously, at one biomed lab (can't remember if it's the wife's) they have a dedicated person (shared by several labs) to do the statistics for the presentations and the papers. Kind of weird if you ask me, you'd think understanding whether your results are meaningful is not something you calculate at the end!
- khafra 15y ago
  Any time some psychologist or economist does a test for Base Rate Neglect or other bias on doctors, they perform horribly.

by Maro 15y ago

barry-cotter 15y ago

In my experience, to a doctor, even a good one, _at first_ it's just an annoying thing they have to do to get published and they don't really understand the significance of significances. There is an entire medical discipline that knowingly uses shitty statistics[0]. Then there are the ones who unknowingly use shitty statistics[1]. Absent someone with a qualification that is impossible to get without learning real statistics as a co-author the default when reading medical research should be that the result is either overblown or wrong. Biologists aren't much better than medical doctors, to my knowledge. [0]http://lesswrong.com/lw/72f/why_epidemiology_will_not_correct_itself/ http://lesswrong.com/lw/72f/why_epidemiology_will_not_correc... [1]http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/ http://www.theatlantic.com/magazine/archive/2010/11/lies-dam...

Maro 15y ago
Just recently Science published, and NASA had a PR hoopla about a bogus paper about arsenic life, after it went through the usual Science peer "review" process. The paper was dicredited by numerous Comment papers for outrageously bad statistics and bad methodology, which Science actually published. For reasons I don't really understand, Science did not redact the paper and called the ensuing outrage normal scientific discourse, even though I think they would have benefited from a mae culpa moment regarding their peer review.

dhimes 15y ago

Also important, and related, is my wife claims it took a year or so of fellowship training before she learned how to ask a scientific question. She's an MD who did a research fellowship at NIH; I'm a physicist.

gcp 15y ago

You have my vote. Lots of policy decisions are made based on statistics. Yet they are easy to fool someone with if he or she does not understand them properly enough. In my experience, to a doctor, even a good one, _at first_ it's just an annoying thing they have to do to get published and they don't really understand the significance of significances. Slightly disappointing. I'd have expected doctors to have an understanding of placebo effects, and the field of medicine is one where you regularly hear (even outside medicine) about studies being discredited afterwards for wrong/improper controls, positive results being unreproducible etc. I don't remember the last time I heard a comp sci paper being withdrawn for bad statistics. (They tend to have none, even when they should)

Maro 15y ago
I'm not saying doctors doing research never learn this stuff, just that they come out of University without this knowledge. (I certainly don't know enough about medical research literature to make a statement like this.) Curiously, at one biomed lab (can't remember if it's the wife's) they have a dedicated person (shared by several labs) to do the statistics for the presentations and the papers. Kind of weird if you ask me, you'd think understanding whether your results are meaningful is not something you calculate at the end!
khafra 15y ago
Any time some psychologist or economist does a test for Base Rate Neglect or other bias on doctors, they perform horribly.