4 ms·

This blog is by Larry Wasserman, so i think his advice should be taken seriously. I agree that there are uses of both philosophies, and that statisticians shoul

This blog is by Larry Wasserman, so i think his advice should be taken seriously. I agree that there are uses of both philosophies, and that statisticians should be pragmatic rather than dogmatic. My issue is that his advice is most useful for statisticians working in the abstract, but it doesn’t really help people working with real data. Scientists and data analysts just want to know how to analyze their data, and this does not help them. I know that stats isn’t a cookbook, but we could put some guard rails down that help practitioners with their problems.

biomcgary 4y ago
>Scientists and data analysts just want to know how to analyze their data, and this does not help them. I'm a computational biologist that uses Bayesian and Frequentist approaches depending on what I'm trying to achieve. This article was very helpful for making explicit a distinction I had not recognized before. With most of science (in my field) being done by people with Doctorates of Philosophy, I think it is reasonable to expect them to understand the underlying concepts of the math they are using. But, I'm anachronistic in wanting science to have a bit more natural philosophy rather than just "shut up and calculate" (https://en.wikiquote.org/wiki/Shut_up_and_calculate https://en.wikiquote.org/wiki/Shut_up_and_calculate).
okennedy 4y ago
Guardrails for stats are something that I've put a lot of thought towards. The fundamental challenge is that statistics is operating in a world of incomplete information. Statistical measures are almost never monotone with respect to new information, and so any new piece of context might completely invalidate an analysis. Going beyond the abstract requires an intimate knowledge of the domain being analyzed, and the limitations of statistical methods as applied to that domain. "Guardrails for stats" have to be domain- and even dataset-specific.
kqr 4y ago
The problem I see with guard rails is that it's very hard to know if you're doing statistics right, due to its nature. Inference is sometimes hard enough on its own (and I sometimes use computational methods in addition to theory just to double-check my results, but that's just the innermost layer. Outside of that you have to define appropriate and efficient samples, which is more difficult. You have to know what population you're actually interested in, which is less obvious than it sounds like, and on top of that you have to pick an experimental/observational method that minimises error and ideally lets you quantify it -- extremely hard in most practical cases. Add to that the fact that the outcome of statistical analysis might often be, "well, we still don't know anything meaningful!" But if you say that, someone else will sound more confident and guess who people will listen to? ---- The way out is not guard rails, it's much better training from earlier ages. This stuff is hard and we are not born with intuition for it. We need lots of practise. I still don't get why there's so much analysis and calculus in our curricula -- those are problems we can solve with numerical (sometimes statistical) methods. We ought to replace at least half off that with more probability and statistical inference and experimental design.
- tpoacher 4y ago
  Guardrails is actually an apt term. Just because you train people not to peer over the edge doesn't mean there's no need for safeguards. (Death Star laser operating platform, I'm looking at you!). And just because there's safeguards doesn't mean you stop warning people not to peer over the edge. The implicit assumption in your text is that if there's a guardrail, people will be leaning all day. Whereas my experience is that if there isn't, people will try that stuff regardless, and keep falling off the platform. Obligatory: https://www.youtube.com/watch?v=9bSZXucTH4A https://www.youtube.com/watch?v=9bSZXucTH4A

biomcgary 4y ago

>Scientists and data analysts just want to know how to analyze their data, and this does not help them. I'm a computational biologist that uses Bayesian and Frequentist approaches depending on what I'm trying to achieve. This article was very helpful for making explicit a distinction I had not recognized before. With most of science (in my field) being done by people with Doctorates of Philosophy, I think it is reasonable to expect them to understand the underlying concepts of the math they are using. But, I'm anachronistic in wanting science to have a bit more natural philosophy rather than just "shut up and calculate" (https://en.wikiquote.org/wiki/Shut_up_and_calculate https://en.wikiquote.org/wiki/Shut_up_and_calculate).

okennedy 4y ago

Guardrails for stats are something that I've put a lot of thought towards. The fundamental challenge is that statistics is operating in a world of incomplete information. Statistical measures are almost never monotone with respect to new information, and so any new piece of context might completely invalidate an analysis. Going beyond the abstract requires an intimate knowledge of the domain being analyzed, and the limitations of statistical methods as applied to that domain. "Guardrails for stats" have to be domain- and even dataset-specific.

kqr 4y ago

The problem I see with guard rails is that it's very hard to know if you're doing statistics right, due to its nature. Inference is sometimes hard enough on its own (and I sometimes use computational methods in addition to theory just to double-check my results, but that's just the innermost layer. Outside of that you have to define appropriate and efficient samples, which is more difficult. You have to know what population you're actually interested in, which is less obvious than it sounds like, and on top of that you have to pick an experimental/observational method that minimises error and ideally lets you quantify it -- extremely hard in most practical cases. Add to that the fact that the outcome of statistical analysis might often be, "well, we still don't know anything meaningful!" But if you say that, someone else will sound more confident and guess who people will listen to? ---- The way out is not guard rails, it's much better training from earlier ages. This stuff is hard and we are not born with intuition for it. We need lots of practise. I still don't get why there's so much analysis and calculus in our curricula -- those are problems we can solve with numerical (sometimes statistical) methods. We ought to replace at least half off that with more probability and statistical inference and experimental design.

tpoacher 4y ago
Guardrails is actually an apt term. Just because you train people not to peer over the edge doesn't mean there's no need for safeguards. (Death Star laser operating platform, I'm looking at you!). And just because there's safeguards doesn't mean you stop warning people not to peer over the edge. The implicit assumption in your text is that if there's a guardrail, people will be leaning all day. Whereas my experience is that if there isn't, people will try that stuff regardless, and keep falling off the platform. Obligatory: https://www.youtube.com/watch?v=9bSZXucTH4A https://www.youtube.com/watch?v=9bSZXucTH4A