14 ms·

The Great Statistical Schism

49 points by Schiphol 11y ago

msandford 11y ago
What's amusing is that Bayesian reasoning is what human beings do all the time. Most of the -isms that people can do (racism, sexism, ageism, etc) have to do with taking the prior (what already I know about other similar people) and applying that knowledge to the new person. It's "too hard to understand" and yet all kinds of people are doing it every day.
- squidfood 11y ago
  Practically speaking, modern science applies priors via funding decisions. Only variables that fall within the range what funders consider "worth testing" will be tested.
  - danieltillett 11y ago
    In theory this is the case, but in practice since nobody actually checks on what you spend the money on you can spend a fairly large chunk of resources on whatever idea you want. You just need to ensure you put enough effort into whatever area you can get money for to allow renewal.
- deleted 11y ago
  [deleted]
- streptomycin 11y ago
  Frequentist: Here's a couple formulas that kind of make sense. Plug in your data and go. Bayesian: Here's Bayes' theorem, a single formula that makes perfect sense! See, it's simpler, right? What, you want to use it? Something something beta distribution something something Markov chain Monte Carlo something something - okay, got it? Good luck teaching that to undergrads. In the real world, intro stats classes are taken mostly by people with little math background, no programming background, little interest in statistics, and little innate talent for any of those things. With the frequentist approach, at least they have a fucking chance.
  - asgard1024 11y ago
    I don't know about you, but I found Bayesian approach much more intuitive than frequentist one. The only reason to use frequentist approach is to have a procedure that you can do with pencil and paper on real problems (and unlike Bayesian methods only under some often weird assumptions). But with computers and statistical packages, that need is going away.
  - Turing_Machine 11y ago
    I agree that frequentist methods are more amenable to teaching students to apply rote formulas that they do not understand. I do not agree that this is a feature. Quite the contrary.
    - jerven 11y ago
      I think it is easier to do frequest statistics "ok", then to asses whether your priors are going to be any good. Especially when looking at systems we don't really understand very well.
    - deleted 11y ago
      [deleted]
    - streptomycin 11y ago
      Whether it's a feature or not is subjective. And I think I mostly agree with you - it's probably bad that we're encouraging the application of bad statistics. But the alternative isn't just replacing bad with good, it'd be more like telling ~98% of the science majors in the country, "hey, you're too dumb to do real statistics, so please just stop trying". I can't see that realistically happening.
- jonahx 11y ago
  Bayesian thinking is so pervasive, and in most cases so unconsicous, that you don't even have to look that far. "Where did my keys go?" They vanished into thin air is not even considered, because its prior is zero. I must have put them somewhere else is the instant frontrunner, because I do it so often. After much unsuccessful searching, hypotheses such as "someone is playing a joke on me" or "my wife accidentally grabbed them" might be considered -- low priors that require additional evidence. Probably only after noticing something like a broken window would "someone broke in and stole them" be considered -- extraordinary claims require extraordinary evidence. And so on...
- dalke 11y ago
  "what already I know about other similar people" But it doesn't work that way. Some properties get associated to members of a class, while others do not. Let's take discrimination on the basis of blood type personality theory in parts of Asia. Many people in Japan believe that one's personality is strongly affected by one's blood type. Quoting from https://en.wikipedia.org/wiki/Blood_type_personality_theory https://en.wikipedia.org/wiki/Blood_type_personality_theory : > Many people have been discriminated against because of their blood type. Employers ask blood types during interviews despite the warnings they have been given. Children at schools have been split up according to their blood type. The national softball team has customized training to fit each player's blood type. Companies have given work assignments according to their employee's blood type. Yet all evidence is that there is no basis for this form of discrimination. It's pseudoscience akin to discrimination based on one's astrological sign. So it's clear that discrimination can rise even without a meaningful prior for a given attribute. We can also look at left- vs. right-handedness for an example of varying levels of discrimination even when the underlying genetics and thus putative justification for discrimination hasn't changed. To copy a quote from https://en.wikipedia.org/wiki/Bias_against_left-handed_people#Forced_use_of_right_hand https://en.wikipedia.org/wiki/Bias_against_left-handed_peopl... : > "I was educated in the USA in Catholic school in the 60s. My left hand was beaten until it was swollen, so I would use my right right [sic] hand" ... "I had a teacher who would smack my left hand with a yardstick every time she caught me writing with my left hand" ... "My fourth grade teacher [...] would force me to use my right hand to perform all of my school work. If she caught me using my left hand, I was hit in the head with a dictionary. It turned out that she believed left handers were connected with Satan." Since we know that discrimination can arise without a prior, and the level of discrimination can change even the prior does not change, it's not clear that Bayesian reasoning is a good way to understand the issue. In addition, it doesn't help understand why some factors are used to define 'similar people' while others are not. Men are far more likely to be involved in crime in the US, including domestic terrorism to white collar crime, but aren't universally discriminated against for being a member of that class. Or on the topic of white collar crime, editorials like 'Wall Street Criminals Are Still a Protected Class in America' at http://www.vice.com/read/wall-street-criminals-are-still-a-protected-class-in-america-808 http://www.vice.com/read/wall-street-criminals-are-still-a-p... highlight how the Bayesian prior is outright ignored. Thus, I'm not convinced that humans really use Bayesian reasoning as the basis for -isms.
nonbel 11y ago
>"For example, consider the ‘null’ hypothesis that the new drug has exactly the same effectiveness as the old drug." I'd encourage people to ask: Why should I consider that null hypothesis when it sounds like a strawman? Wasn't the use of strawman arguments debunked over 1000 years ago?
- andrewflnr 11y ago
  A strawman argument is a distorted version of someone else's position, formulated to be easy to knock down for the purpose of appearing to win debates. A null hypothesis is just a starting point for reasoning.
  - nonbel 11y ago
    And whose position is it that a drug/treatment/whatever has exactly zero effect? Also that there are exactly zero differences between two or more groups at baseline and this continues to be true for the duration of the study (except if due to the treatment)? Really, who is advocating that hypothesis? Also, a null hypothesis was originally the hypothesis to be nullified. There may be confusion due to the use of "null" here. Null != nil
    - evmar 11y ago
      From your comment history you appear to have some background in statistics, but what you're written here is curiously missing the fundamentals. It's not exactly clear what you're arguing, but I am definitely curious because I must be missing it! To answer your last question directly: nobody argues that there are zero differences between groups in an experiment, but rather that the central limit theorem applies, which is a statement about the distribution of means in particular circumstances. https://en.wikipedia.org/wiki/Central_limit_theorem https://en.wikipedia.org/wiki/Central_limit_theorem
      - nonbel 11y ago
        >"nobody argues that there are zero differences between groups in an experiment" The null hypothesis (usually) assumes that the two groups are samples from the same distribution. Do you agree?
imh 11y ago
I really never understood the divide. They are all just tools in our toolbox. It feels like arguing screwdrivers versus hammers. They may not do the same thing, but both can be useful!
- jonahx 11y ago
  The author specifically argues against this view as something that obscures the superiority, in most cases, of the Bayesian approach: One downside of this ecumenicalism is a reluctance to ask fundamental questions: having a strong opinion on this matter has gone out of fashion. Who’s to say one statistical philosophy is better than another? Aren’t all statistical philosophies equally valid paths to good data analysis? Frequentism is “true for me”. As in religion, so in statistics. If you criticise a colleague for using p-values when posterior probabilities are clearly more appropriate, it will lead to accusations of being a ‘zealot’ [2] who should stop ‘crusading’ and then later: If Einstein’s theory were easier (as well as being more accurate), teaching Newton’s would be silly. Yet that’s the way most statistics curricula are structured. The only reason statisticians think frequentist ideas are easier is that they are used to them
  - imh 11y ago
    I guess I hate the idea of a "statistical philosophy." It's math for god's sake. Given these axioms, these other things are true. The point about teaching relativistic versus newtonian mechanics is totally different because they're incompatible. One is more correct than the other, which is not the case here. In the stats case, both tools give correct answers to different questions. Maybe I just get frustrated feeling like people want a data analysis silver bullet to teach.
    - kgwgk 11y ago
      I'd say the analogy is more valid than you think. Classical (frequentist) methods are valid as long as the results correspond to the relativistic (Bayesian) ones.
      - 11y ago
keithwinstein 11y ago
I think these kinds of articles generally overstate the "schism." I wrote about this here: http://blog.keithw.org/2013/02/q-what-is-difference-between-bayesian.html http://blog.keithw.org/2013/02/q-what-is-difference-between-... and here: http://qr.ae/70H3k7 http://qr.ae/70H3k7 Three points: (1) I couldn't follow the author's Bayesian analysis or what "standard assumptions" were used. Does anybody know what is meant? Here's a plausible Bayesian analysis using what are arguably standard assumptions: "assume, a priori, that the success rate of the new drug was drawn uniformly between 0 and 1. Given the outcome of 83/100 i.i.d. successes, the probability that the new drug has a success rate worse than the old drug is Integrate[PDF[BetaDistribution[84,18],x],{x,0,0.7}]. In other words, p(worse|outcome,uniform prior) = 0.0018." Here's a plausible classical analysis: "Before doing the experiment, let's specify how to calculate the p-value at the end. For whatever outcome we get, we'll calculate the one-sided probability that the old drug produces an outcome that good or better. Now perform the experiment: our outcome is 83/100. Per the method that we pre-specified, the probability that the old drug would have produced an outcome that good or better is Sum[Binomial[100,i] (0.7)^i (1-0.7)^(100-i), {i, 83, 100}]. In other words, p<0.0022." Here's a plausible takeaway: "What do you know, the p-value from the classical analysis and the posterior probability from the Bayesian analysis are almost the same. There isn't much difference in this case, and contrary to a point emphasized in the article, the posterior probability was actually slightly smaller. The claim that p-values overstate the certainty of findings compared with Bayesian methods is not supported and probably not true -- often the accusation is the reverse, that p-values are too conservative!" (2) The Bayesian analysis and the classical one are trying to achieve different things. Speaking very generally, classical methods are about designing an experiment that has a cap on the rate of false positives, even in the worst-case input (and then running that experiment). Depending on who you are, this may or may not be what you really want to do. Bayesian methods are (again very generally) about calculating the conditional probability of some event, given a particular observation and well-stated prior assumptions. Again, this may or may not be what you really want to do. The difference is sort of like the difference between saying that the running time of QuickSort is O(n^2) on adversarial input, compared with saying that the running time is O(n lg n) in expectation, assuming the input order is uniformly distributed. Both of these statements can be useful. You don't have to pick a side and declare yourself a worstcaster or an expecterian, any more than you have to call yourself a Bayesian or a Frequentist. These are families of mathematically-sound techniques, not religions. (3) Ultimately, statistics doesn't really matter until somebody starts making a decision based on the results. And once you start putting a cost on bad decisions and designing a decision rule to maximize utility (the domain of decision theory), methods based on p-values and posterior probabilities end up reaching THE SAME DECISIONS. This makes sense since there can really only be one utility-maximizing decision theory. This was understood in the early 1940s when they had to decide how to balance the cost of mistakenly shooting down an Allied aircraft versus the cost of mistakenly letting a Nazi aircraft off the hook. It was understood in the late 1940s when Shannon and others worked out the mathematical theory of communication. These fields do not have squabbles about a Bayesian vs. frequentist schism.
cormullion 11y ago
There's a good book on the subject, "The theory that would not die", which covers the history and sheds some light on the people (Fisher, Laplace, Bayes, Pearson) behind the names. Reviewed: http://www.ams.org/notices/201205/rtx120500657p.pdf http://www.ams.org/notices/201205/rtx120500657p.pdf
pdkl95 11y ago
http://jakevdp.github.io/blog/2014/03/11/frequentism-and-bayesianism-a-practical-intro/ http://jakevdp.github.io/blog/2014/03/11/frequentism-and-bay... I recommend this series of posts on frequentism and bayesianism. Of particular interest is part 3, which has a very good explanation of why frequetism shouldn't be used in science. Let me say it directly: you should be suspicious of the use of frequentist confidence intervals and p-values in science. In a scientific setting, confidence intervals, and closely-related p-values, provide the correct answer to the wrong question.
- david_ar 11y ago
  Also read chapter 37 "Bayesian Inference and Sampling Theory" of http://www.inference.phy.cam.ac.uk/itila/book.html http://www.inference.phy.cam.ac.uk/itila/book.html
kriro 11y ago
I think the author is treating the frequentists somewhat unfairly. A good design involves both NHST and equivalence tests + discussion about power/effect size (imo). Random grab from my methods collection in my literature tool: "A Comparison Of Equivalence Testing In Combination With Hypothesis Testing And Effect Sizes": http://digitalcommons.wayne.edu/jmasm/vol2/iss2/6/ http://digitalcommons.wayne.edu/jmasm/vol2/iss2/6/ I think the biggest problem is that many fields just churn out people who can click buttons in SPSS but couldn't really tell you what a p-value actually represents. "Unfortunately" it's fairly easy to do the testing with tools. From students, I often get the feeling that tests are conducted for the sake of conduction a test (talking about BA theses etc.). I'm not sure how wide spread Bayesian thinking is in other fields. Coming from CS it is somewhat natural since it's pretty pervasive in AI/ML. I think our econ students didn't get exposed to it at all (iirc). Would love some feedback from other fields :)

49 points by Schiphol 11y ago

What's amusing is that Bayesian reasoning is what human beings do all the time. Most of the -isms that people can do (racism, sexism, ageism, etc) have to do with taking the prior (what already I know about other similar people) and applying that knowledge to the new person. It's "too hard to understand" and yet all kinds of people are doing it every day.

squidfood 11y ago
Practically speaking, modern science applies priors via funding decisions. Only variables that fall within the range what funders consider "worth testing" will be tested.
- danieltillett 11y ago
  In theory this is the case, but in practice since nobody actually checks on what you spend the money on you can spend a fairly large chunk of resources on whatever idea you want. You just need to ensure you put enough effort into whatever area you can get money for to allow renewal.
deleted 11y ago
[deleted]
streptomycin 11y ago
Frequentist: Here's a couple formulas that kind of make sense. Plug in your data and go. Bayesian: Here's Bayes' theorem, a single formula that makes perfect sense! See, it's simpler, right? What, you want to use it? Something something beta distribution something something Markov chain Monte Carlo something something - okay, got it? Good luck teaching that to undergrads. In the real world, intro stats classes are taken mostly by people with little math background, no programming background, little interest in statistics, and little innate talent for any of those things. With the frequentist approach, at least they have a fucking chance.
- asgard1024 11y ago
  I don't know about you, but I found Bayesian approach much more intuitive than frequentist one. The only reason to use frequentist approach is to have a procedure that you can do with pencil and paper on real problems (and unlike Bayesian methods only under some often weird assumptions). But with computers and statistical packages, that need is going away.
- Turing_Machine 11y ago
  I agree that frequentist methods are more amenable to teaching students to apply rote formulas that they do not understand. I do not agree that this is a feature. Quite the contrary.
  - jerven 11y ago
    I think it is easier to do frequest statistics "ok", then to asses whether your priors are going to be any good. Especially when looking at systems we don't really understand very well.
  - deleted 11y ago
    [deleted]
  - streptomycin 11y ago
    Whether it's a feature or not is subjective. And I think I mostly agree with you - it's probably bad that we're encouraging the application of bad statistics. But the alternative isn't just replacing bad with good, it'd be more like telling ~98% of the science majors in the country, "hey, you're too dumb to do real statistics, so please just stop trying". I can't see that realistically happening.
jonahx 11y ago
Bayesian thinking is so pervasive, and in most cases so unconsicous, that you don't even have to look that far. "Where did my keys go?" They vanished into thin air is not even considered, because its prior is zero. I must have put them somewhere else is the instant frontrunner, because I do it so often. After much unsuccessful searching, hypotheses such as "someone is playing a joke on me" or "my wife accidentally grabbed them" might be considered -- low priors that require additional evidence. Probably only after noticing something like a broken window would "someone broke in and stole them" be considered -- extraordinary claims require extraordinary evidence. And so on...
dalke 11y ago
"what already I know about other similar people" But it doesn't work that way. Some properties get associated to members of a class, while others do not. Let's take discrimination on the basis of blood type personality theory in parts of Asia. Many people in Japan believe that one's personality is strongly affected by one's blood type. Quoting from https://en.wikipedia.org/wiki/Blood_type_personality_theory https://en.wikipedia.org/wiki/Blood_type_personality_theory : > Many people have been discriminated against because of their blood type. Employers ask blood types during interviews despite the warnings they have been given. Children at schools have been split up according to their blood type. The national softball team has customized training to fit each player's blood type. Companies have given work assignments according to their employee's blood type. Yet all evidence is that there is no basis for this form of discrimination. It's pseudoscience akin to discrimination based on one's astrological sign. So it's clear that discrimination can rise even without a meaningful prior for a given attribute. We can also look at left- vs. right-handedness for an example of varying levels of discrimination even when the underlying genetics and thus putative justification for discrimination hasn't changed. To copy a quote from https://en.wikipedia.org/wiki/Bias_against_left-handed_people#Forced_use_of_right_hand https://en.wikipedia.org/wiki/Bias_against_left-handed_peopl... : > "I was educated in the USA in Catholic school in the 60s. My left hand was beaten until it was swollen, so I would use my right right [sic] hand" ... "I had a teacher who would smack my left hand with a yardstick every time she caught me writing with my left hand" ... "My fourth grade teacher [...] would force me to use my right hand to perform all of my school work. If she caught me using my left hand, I was hit in the head with a dictionary. It turned out that she believed left handers were connected with Satan." Since we know that discrimination can arise without a prior, and the level of discrimination can change even the prior does not change, it's not clear that Bayesian reasoning is a good way to understand the issue. In addition, it doesn't help understand why some factors are used to define 'similar people' while others are not. Men are far more likely to be involved in crime in the US, including domestic terrorism to white collar crime, but aren't universally discriminated against for being a member of that class. Or on the topic of white collar crime, editorials like 'Wall Street Criminals Are Still a Protected Class in America' at http://www.vice.com/read/wall-street-criminals-are-still-a-protected-class-in-america-808 http://www.vice.com/read/wall-street-criminals-are-still-a-p... highlight how the Bayesian prior is outright ignored. Thus, I'm not convinced that humans really use Bayesian reasoning as the basis for -isms.

nonbel 11y ago

>"For example, consider the ‘null’ hypothesis that the new drug has exactly the same effectiveness as the old drug." I'd encourage people to ask: Why should I consider that null hypothesis when it sounds like a strawman? Wasn't the use of strawman arguments debunked over 1000 years ago?

andrewflnr 11y ago
A strawman argument is a distorted version of someone else's position, formulated to be easy to knock down for the purpose of appearing to win debates. A null hypothesis is just a starting point for reasoning.
- nonbel 11y ago
  And whose position is it that a drug/treatment/whatever has exactly zero effect? Also that there are exactly zero differences between two or more groups at baseline and this continues to be true for the duration of the study (except if due to the treatment)? Really, who is advocating that hypothesis? Also, a null hypothesis was originally the hypothesis to be nullified. There may be confusion due to the use of "null" here. Null != nil
  - evmar 11y ago
    From your comment history you appear to have some background in statistics, but what you're written here is curiously missing the fundamentals. It's not exactly clear what you're arguing, but I am definitely curious because I must be missing it! To answer your last question directly: nobody argues that there are zero differences between groups in an experiment, but rather that the central limit theorem applies, which is a statement about the distribution of means in particular circumstances. https://en.wikipedia.org/wiki/Central_limit_theorem https://en.wikipedia.org/wiki/Central_limit_theorem
    - nonbel 11y ago
      >"nobody argues that there are zero differences between groups in an experiment" The null hypothesis (usually) assumes that the two groups are samples from the same distribution. Do you agree?

imh 11y ago

I really never understood the divide. They are all just tools in our toolbox. It feels like arguing screwdrivers versus hammers. They may not do the same thing, but both can be useful!

jonahx 11y ago
The author specifically argues against this view as something that obscures the superiority, in most cases, of the Bayesian approach: One downside of this ecumenicalism is a reluctance to ask fundamental questions: having a strong opinion on this matter has gone out of fashion. Who’s to say one statistical philosophy is better than another? Aren’t all statistical philosophies equally valid paths to good data analysis? Frequentism is “true for me”. As in religion, so in statistics. If you criticise a colleague for using p-values when posterior probabilities are clearly more appropriate, it will lead to accusations of being a ‘zealot’ [2] who should stop ‘crusading’ and then later: If Einstein’s theory were easier (as well as being more accurate), teaching Newton’s would be silly. Yet that’s the way most statistics curricula are structured. The only reason statisticians think frequentist ideas are easier is that they are used to them
- imh 11y ago
  I guess I hate the idea of a "statistical philosophy." It's math for god's sake. Given these axioms, these other things are true. The point about teaching relativistic versus newtonian mechanics is totally different because they're incompatible. One is more correct than the other, which is not the case here. In the stats case, both tools give correct answers to different questions. Maybe I just get frustrated feeling like people want a data analysis silver bullet to teach.
  - kgwgk 11y ago
    I'd say the analogy is more valid than you think. Classical (frequentist) methods are valid as long as the results correspond to the relativistic (Bayesian) ones.
    - 11y ago

keithwinstein 11y ago

I think these kinds of articles generally overstate the "schism." I wrote about this here: http://blog.keithw.org/2013/02/q-what-is-difference-between-bayesian.html http://blog.keithw.org/2013/02/q-what-is-difference-between-... and here: http://qr.ae/70H3k7 http://qr.ae/70H3k7 Three points: (1) I couldn't follow the author's Bayesian analysis or what "standard assumptions" were used. Does anybody know what is meant? Here's a plausible Bayesian analysis using what are arguably standard assumptions: "assume, a priori, that the success rate of the new drug was drawn uniformly between 0 and 1. Given the outcome of 83/100 i.i.d. successes, the probability that the new drug has a success rate worse than the old drug is Integrate[PDF[BetaDistribution[84,18],x],{x,0,0.7}]. In other words, p(worse|outcome,uniform prior) = 0.0018." Here's a plausible classical analysis: "Before doing the experiment, let's specify how to calculate the p-value at the end. For whatever outcome we get, we'll calculate the one-sided probability that the old drug produces an outcome that good or better. Now perform the experiment: our outcome is 83/100. Per the method that we pre-specified, the probability that the old drug would have produced an outcome that good or better is Sum[Binomial[100,i] (0.7)^i (1-0.7)^(100-i), {i, 83, 100}]. In other words, p<0.0022." Here's a plausible takeaway: "What do you know, the p-value from the classical analysis and the posterior probability from the Bayesian analysis are almost the same. There isn't much difference in this case, and contrary to a point emphasized in the article, the posterior probability was actually slightly smaller. The claim that p-values overstate the certainty of findings compared with Bayesian methods is not supported and probably not true -- often the accusation is the reverse, that p-values are too conservative!" (2) The Bayesian analysis and the classical one are trying to achieve different things. Speaking very generally, classical methods are about designing an experiment that has a cap on the rate of false positives, even in the worst-case input (and then running that experiment). Depending on who you are, this may or may not be what you really want to do. Bayesian methods are (again very generally) about calculating the conditional probability of some event, given a particular observation and well-stated prior assumptions. Again, this may or may not be what you really want to do. The difference is sort of like the difference between saying that the running time of QuickSort is O(n^2) on adversarial input, compared with saying that the running time is O(n lg n) in expectation, assuming the input order is uniformly distributed. Both of these statements can be useful. You don't have to pick a side and declare yourself a worstcaster or an expecterian, any more than you have to call yourself a Bayesian or a Frequentist. These are families of mathematically-sound techniques, not religions. (3) Ultimately, statistics doesn't really matter until somebody starts making a decision based on the results. And once you start putting a cost on bad decisions and designing a decision rule to maximize utility (the domain of decision theory), methods based on p-values and posterior probabilities end up reaching THE SAME DECISIONS. This makes sense since there can really only be one utility-maximizing decision theory. This was understood in the early 1940s when they had to decide how to balance the cost of mistakenly shooting down an Allied aircraft versus the cost of mistakenly letting a Nazi aircraft off the hook. It was understood in the late 1940s when Shannon and others worked out the mathematical theory of communication. These fields do not have squabbles about a Bayesian vs. frequentist schism.

cormullion 11y ago

There's a good book on the subject, "The theory that would not die", which covers the history and sheds some light on the people (Fisher, Laplace, Bayes, Pearson) behind the names. Reviewed: http://www.ams.org/notices/201205/rtx120500657p.pdf http://www.ams.org/notices/201205/rtx120500657p.pdf

pdkl95 11y ago

http://jakevdp.github.io/blog/2014/03/11/frequentism-and-bayesianism-a-practical-intro/ http://jakevdp.github.io/blog/2014/03/11/frequentism-and-bay... I recommend this series of posts on frequentism and bayesianism. Of particular interest is part 3, which has a very good explanation of why frequetism shouldn't be used in science. Let me say it directly: you should be suspicious of the use of frequentist confidence intervals and p-values in science. In a scientific setting, confidence intervals, and closely-related p-values, provide the correct answer to the wrong question.

david_ar 11y ago
Also read chapter 37 "Bayesian Inference and Sampling Theory" of http://www.inference.phy.cam.ac.uk/itila/book.html http://www.inference.phy.cam.ac.uk/itila/book.html

kriro 11y ago

I think the author is treating the frequentists somewhat unfairly. A good design involves both NHST and equivalence tests + discussion about power/effect size (imo). Random grab from my methods collection in my literature tool: "A Comparison Of Equivalence Testing In Combination With Hypothesis Testing And Effect Sizes": http://digitalcommons.wayne.edu/jmasm/vol2/iss2/6/ http://digitalcommons.wayne.edu/jmasm/vol2/iss2/6/ I think the biggest problem is that many fields just churn out people who can click buttons in SPSS but couldn't really tell you what a p-value actually represents. "Unfortunately" it's fairly easy to do the testing with tools. From students, I often get the feeling that tests are conducted for the sake of conduction a test (talking about BA theses etc.). I'm not sure how wide spread Bayesian thinking is in other fields. Coming from CS it is somewhat natural since it's pretty pervasive in AI/ML. I think our econ students didn't get exposed to it at all (iirc). Would love some feedback from other fields :)