9 ms·

On the one hand, I have (in the semiconductor manufacturing industry) encountered statisticians who were greatly averse to making any prior assumptions about th

On the one hand, I have (in the semiconductor manufacturing industry) encountered statisticians who were greatly averse to making any prior assumptions about the likelihood of something. Also, any article with the phrase "nonsense on stilts" in it is not entirely unworthy. It also does a credible job of explaining Bayes' theorem. On the other hand, I think it somewhat exaggerates the extent to which Bayes' theorem and its associated work are rejected by statisticians, and also the extent to which a magical p > 0.05 limit was advocated. I believe it was non-statisticians who wanted easy tests that they could apply without understanding much about statistics, who were to blame in that case.

eli_gottlieb 8y ago
Statistics is formalized philosophy of science, so of course scientists outside the stats departments wanted "easy tests". Arguably, a lot of the new applications of computational Bayesian methods in the natural science are the same statistical witch-doctoring with a bit more rigor. (Ok, I do computational Bayes for a living with natural scientists, so I'm inherently biased to view our work as shakier than it really is.)
- Karrot_Kream 8y ago
  Aside: What do you do for a living?
  - AlexCoventry 8y ago
    LinkedIn in his HN user page. > Variational Bayesian methods and deep learning for affective neuroscience, using probabilistic programming systems ProbTorch and Pyro. It does sound like he's in for a rough time, epistemologically speaking.
    - eli_gottlieb 8y ago
      You'd be surprised how easily the witch-doctory passes for real science ;-)! Kidding, kidding, our empirical side actually has to be at least as rigorous as the other statistical methods used in neuroscience, usually more so. That's what they keep us around for!
      - AlexCoventry 8y ago
        Twenty years ago I would have been surprised. :-)
- sewercake 8y ago
  'Statistics is formalized philosophy of science' -- care to expand? Or provide supplementary literature?
  - pash 8y ago
    He’s likely referring to the fact that the rules of probabilistic reasoning can be shown to be the only rules of reasoning that yield correct inferences from data. Bayesian statistics is in that sense the “logic of science”. The first chapter of E.T. Jaynes’s book with that subtitle [0] is a good introduction to these ideas if you haven’t seen them before. 0. E.T. Jaynes (2003), “Probability Theory: the Logic of Science,” http://bayes.wustl.edu/etj/prob/book.pdf http://bayes.wustl.edu/etj/prob/book.pdf [PDF]
  - eli_gottlieb 8y ago
    As a matter of fact, Jaynes is a good reference even though I didn't have him in mind at the time. I was more referring to the general impression I get reading a lot of statistics textbooks and papers, which is that they're trying to numerically formalize methodologies appropriate to experimental sciences. You end up with a fair amount of papers co-written between phil-of-science scholars and statisticians.
analog31 8y ago
A statistician could talk me out of this, but I've always been puzzled by the use of "prior." Nothing in Bayes's Theorem says that the prior has to be established before anything else. It makes Bayesian statistics seem like a matter of letting your expectations influence your results. Instead, an interpretation that seems more favorable to me, is that you consider all of the information at your disposal, that can be brought to bear on a problem, and this could include known constraints on likelihoods. Bayes's Theorem becomes a tool for working through problems where a single statistic can't be readily used to analyze an entire data set, e.g., when data come from disparate sources and can't be readily combined.
- lgas 8y ago
  I'm not a statistician but as I understand it, in Bayesian terms, when you get some new piece of information you must update your beliefs to incorporate the new information. So you take your belief prior to the new information and incorporate the new information to get your updated belief.
  - analog31 8y ago
    That's cool. It sounds like how we all learned to think since time immoral, so I can hardly dispute it.
- hooloovoo_zoo 8y ago
  It sounds like you're talking about just using the maximum likelihood estimator (with a special likelihood) which is similar to using a flat (improper) prior. One disadvantage of doing so is that the results aren't invariant to reparameterization of the problem. The latter problem can be fixed by using Jeffreys' priors. However, the MLE is very effective in its own right.
- sjg007 8y ago
  A prior could be a population level statistic which is then updated by a likelihood to reveal a posterior probability. Take for example a medical test. We know that on average that 1/10 people have some disease X. We have a test that tests for X (with some false positive rate).. Using that we can calculate the posterior probability that you actually have disease X. If you relied just on the test you don't discount the population level statistics which might be relevant. You can also use the tests FP rate in the same way as your prior... e.g. the people who test positive who don't have the disease etc...
gweinberg 8y ago
There's probably some truth to this. But if statisticians weren't able to convince the academic world that arbitrarily setting a "statistical significance" threshold is a gross misuse of statistics, then what good are they? It would be like MDs failing to convince the rest of the medical community that bloodletting isn't a universal cure for whatever the hell ails you.
- smallnamespace 8y ago
  Responsibility in communication is a two-way street, and also depends on incentives and relative power. - Are security researchers grossly incompetent for failing to convince the IT industry to adopt best practices? - Are voters in democracies grossly incompetent for failing to police their politicians and keep them honest? - Are media outlets grossly incompetent for not squeezing out 'fake news', whatever your definition? - Are epidemiologists and public health officials grossly incompetent for not convincing anti-vaxxers of their foolishness? Anytime there is a breakdown in communications, both the listener and the speaker should probably shoulder some of it.
- Bromskloss 8y ago
  > statisticians weren't able to convince the academic world that arbitrarily setting a "statistical significance" threshold is a gross misuse of statistics What would the proper use of statistics be? I believe it is to produce a Bayesian posterior probability distribution, but that wouldn't be the opinion of a frequentist.
wirrbel 8y ago
> I believe it was non-statisticians who wanted easy tests that they could apply without understanding much about statistics, who were to blame in that case. Yes and no. I think most journal articles feature a copy-and-paste approach to statistics, reusing what other papers do, or worse, tinkering with test functions in excel. Yet, this seems to be ok with statisticians it seems. From my personal experience applying statistics in the software world (and trying to build alerting models for monitoring applications, etc.) is, that since we have a lot of systems that continuously record date (instead of the way scientific studies usually work, collecting a set of data points, N=x), we get cross-validation of our results, which has drastically made me aware of the pitfalls of frequentist statistics.

eli_gottlieb 8y ago

Statistics is formalized philosophy of science, so of course scientists outside the stats departments wanted "easy tests". Arguably, a lot of the new applications of computational Bayesian methods in the natural science are the same statistical witch-doctoring with a bit more rigor. (Ok, I do computational Bayes for a living with natural scientists, so I'm inherently biased to view our work as shakier than it really is.)

Karrot_Kream 8y ago
Aside: What do you do for a living?
- AlexCoventry 8y ago
  LinkedIn in his HN user page. > Variational Bayesian methods and deep learning for affective neuroscience, using probabilistic programming systems ProbTorch and Pyro. It does sound like he's in for a rough time, epistemologically speaking.
  - eli_gottlieb 8y ago
    You'd be surprised how easily the witch-doctory passes for real science ;-)! Kidding, kidding, our empirical side actually has to be at least as rigorous as the other statistical methods used in neuroscience, usually more so. That's what they keep us around for!
    - AlexCoventry 8y ago
      Twenty years ago I would have been surprised. :-)
sewercake 8y ago
'Statistics is formalized philosophy of science' -- care to expand? Or provide supplementary literature?
- pash 8y ago
  He’s likely referring to the fact that the rules of probabilistic reasoning can be shown to be the only rules of reasoning that yield correct inferences from data. Bayesian statistics is in that sense the “logic of science”. The first chapter of E.T. Jaynes’s book with that subtitle [0] is a good introduction to these ideas if you haven’t seen them before. 0. E.T. Jaynes (2003), “Probability Theory: the Logic of Science,” http://bayes.wustl.edu/etj/prob/book.pdf http://bayes.wustl.edu/etj/prob/book.pdf [PDF]
- eli_gottlieb 8y ago
  As a matter of fact, Jaynes is a good reference even though I didn't have him in mind at the time. I was more referring to the general impression I get reading a lot of statistics textbooks and papers, which is that they're trying to numerically formalize methodologies appropriate to experimental sciences. You end up with a fair amount of papers co-written between phil-of-science scholars and statisticians.

analog31 8y ago

A statistician could talk me out of this, but I've always been puzzled by the use of "prior." Nothing in Bayes's Theorem says that the prior has to be established before anything else. It makes Bayesian statistics seem like a matter of letting your expectations influence your results. Instead, an interpretation that seems more favorable to me, is that you consider all of the information at your disposal, that can be brought to bear on a problem, and this could include known constraints on likelihoods. Bayes's Theorem becomes a tool for working through problems where a single statistic can't be readily used to analyze an entire data set, e.g., when data come from disparate sources and can't be readily combined.

lgas 8y ago
I'm not a statistician but as I understand it, in Bayesian terms, when you get some new piece of information you must update your beliefs to incorporate the new information. So you take your belief prior to the new information and incorporate the new information to get your updated belief.
- analog31 8y ago
  That's cool. It sounds like how we all learned to think since time immoral, so I can hardly dispute it.
hooloovoo_zoo 8y ago
It sounds like you're talking about just using the maximum likelihood estimator (with a special likelihood) which is similar to using a flat (improper) prior. One disadvantage of doing so is that the results aren't invariant to reparameterization of the problem. The latter problem can be fixed by using Jeffreys' priors. However, the MLE is very effective in its own right.
sjg007 8y ago
A prior could be a population level statistic which is then updated by a likelihood to reveal a posterior probability. Take for example a medical test. We know that on average that 1/10 people have some disease X. We have a test that tests for X (with some false positive rate).. Using that we can calculate the posterior probability that you actually have disease X. If you relied just on the test you don't discount the population level statistics which might be relevant. You can also use the tests FP rate in the same way as your prior... e.g. the people who test positive who don't have the disease etc...

gweinberg 8y ago

There's probably some truth to this. But if statisticians weren't able to convince the academic world that arbitrarily setting a "statistical significance" threshold is a gross misuse of statistics, then what good are they? It would be like MDs failing to convince the rest of the medical community that bloodletting isn't a universal cure for whatever the hell ails you.

smallnamespace 8y ago
Responsibility in communication is a two-way street, and also depends on incentives and relative power. - Are security researchers grossly incompetent for failing to convince the IT industry to adopt best practices? - Are voters in democracies grossly incompetent for failing to police their politicians and keep them honest? - Are media outlets grossly incompetent for not squeezing out 'fake news', whatever your definition? - Are epidemiologists and public health officials grossly incompetent for not convincing anti-vaxxers of their foolishness? Anytime there is a breakdown in communications, both the listener and the speaker should probably shoulder some of it.
Bromskloss 8y ago
> statisticians weren't able to convince the academic world that arbitrarily setting a "statistical significance" threshold is a gross misuse of statistics What would the proper use of statistics be? I believe it is to produce a Bayesian posterior probability distribution, but that wouldn't be the opinion of a frequentist.

wirrbel 8y ago

> I believe it was non-statisticians who wanted easy tests that they could apply without understanding much about statistics, who were to blame in that case. Yes and no. I think most journal articles feature a copy-and-paste approach to statistics, reusing what other papers do, or worse, tinkering with test functions in excel. Yet, this seems to be ok with statisticians it seems. From my personal experience applying statistics in the software world (and trying to build alerting models for monitoring applications, etc.) is, that since we have a lot of systems that continuously record date (instead of the way scientific studies usually work, collecting a set of data points, N=x), we get cross-validation of our results, which has drastically made me aware of the pitfalls of frequentist statistics.