4 ms·

It feels like the fact that this is a bayesian-favoured study is some kind of sign that people (i.e. people that are upvoting, not necessarily the researchers)

by tw1010 9y ago

It feels like the fact that this is a bayesian-favoured study is some kind of sign that people (i.e. people that are upvoting, not necessarily the researchers) are actually taking that silly bayesian vs frequentist political war thing seriously. (Which I don't think is advisable.)

nonbel 9y ago
I agree, there is no reason to mention "bayesian" in the title other than using it as a hype term like "blockchain" or "deep learning". Also, the real difference is whether they test a "null hypothesis" vs testing an "actual hypothesis". Like almost all psych, this falls in the first category.
- KirinDave 9y ago
  You say this but then you go read a paper with a cruddy t-test and then injure your eyes with the spasmodic rolling that it induces, like I did. It's kinda remarkable how many people don't actually use modern bayesian analysis techniques, which is why it's such a policy war. It's the difference between "older statistical methods uninformed by the glut of modern processing power" and "things which require a computer to do well, and preserve uncertainty."
  - closed 9y ago
    I don't think it's an issue of old versus new methods. Daniel Lakens has an great string of blog posts / publications on how a lot of the same things can go wrong with Bayesian anaylses [1], and how p-values can be viewed as a "poor man's Bayesian updating function" [2]. [1]: http://daniellakens.blogspot.com/2017/03/no-p-values-are-not-to-blame-part-53.html http://daniellakens.blogspot.com/2017/03/no-p-values-are-not... [2]: https://daniellakens.blogspot.com/2015/11/the-relation-between-p-values-and.html https://daniellakens.blogspot.com/2015/11/the-relation-betwe...
    - KirinDave 9y ago
      When I do not have a processor easily capable of bayesian mkmv sampling I'll consider using that method. But I do and I guess this means I am rich? Sure.
  - nonbel 9y ago
    Here is what they say they did: >"A Bayesian analysis was performed to test the relative likelihood of observing our results given the null hypothesis (i.e., the absence of any depletion effect) versus the likelihood of observing our results given the alternative hypothesis (the presence of a nonzero effect)." I mean that I have no interest in the result of this "test" at all. First of all, they are not directly testing for the existence of a "depletion effect". From the paper: >"Participants in both conditions were provided with a lengthy passage of text. Those in the Control condition simply crossed out with a pencil all occurrences of the letter ‘e’ in a passage of text, whereas those in the Depletion condition were given the following instructions: ‘cross off the letter “e” every time it appears with the following exceptions: 1. Do not cross out the “e” if it is adjacent to another vowel (e.g., friend); 2. Do not cross out the “e” if it is one letter away from another vowel (e.g., vowel); 3. Do not cross out the “e” if the word has 6 letters (e.g., “there”); 4.Do not cross out the “e” if it is the third to last letter (e.g., customers); 5.Do not cross out the “e” if there are double letters in the word (e.g., “hello”)’. Participants were asked to continue the writing task for six minutes, after which the experimenter instructed participants to discontinue. [...] Participants were given a sheet on which 30 anagrams were listed. Participants were asked to complete as many anagrams as they could within five minutes. After the anagram task, participants were asked to refer to a Likert scale to rate the difficulty of the initial crossing-out letters task (1 = not at all difficult; 7 = very difficult). The number of correctly completed anagrams was recorded for each participant." So "depletion effect" means "got fewer anagrams correct after inspecting text closely for 6 minutes". How about their eyes just got tired from squinting closer at the page? Or they were primed to think/write slower since they had to cross out fewer e's? I'm sure you can come up with more alternatives. Second of all, who is actually predicting exactly zero effect of crossing out the letter e more or less often on filling out anagrams? It seems totally implausible to me there is no effect, especially if you go for longer than 6 minutes. Because that is what they are comparing: exactly zero effect vs any other size effect. This is not "small effect" vs "large effect". Literally any way they can mess up the experiments will tend toward non-zero effect. It is only a matter of sample size. Bayes factors vs p-values is irrelevant to the actual problems with these studies.
  - BeetleB 9y ago
    >It's kinda remarkable how many people don't actually use modern bayesian analysis techniques, which is why it's such a policy war. It's the difference between "older statistical methods uninformed by the glut of modern processing power" and "things which require a computer to do well, and preserve uncertainty." There's an old joke: There are two types of statisticians. Those who use Bayesian techniques, and those who use both.

by tw1010 9y ago

nonbel 9y ago

I agree, there is no reason to mention "bayesian" in the title other than using it as a hype term like "blockchain" or "deep learning". Also, the real difference is whether they test a "null hypothesis" vs testing an "actual hypothesis". Like almost all psych, this falls in the first category.

KirinDave 9y ago
You say this but then you go read a paper with a cruddy t-test and then injure your eyes with the spasmodic rolling that it induces, like I did. It's kinda remarkable how many people don't actually use modern bayesian analysis techniques, which is why it's such a policy war. It's the difference between "older statistical methods uninformed by the glut of modern processing power" and "things which require a computer to do well, and preserve uncertainty."
- closed 9y ago
  I don't think it's an issue of old versus new methods. Daniel Lakens has an great string of blog posts / publications on how a lot of the same things can go wrong with Bayesian anaylses [1], and how p-values can be viewed as a "poor man's Bayesian updating function" [2]. [1]: http://daniellakens.blogspot.com/2017/03/no-p-values-are-not-to-blame-part-53.html http://daniellakens.blogspot.com/2017/03/no-p-values-are-not... [2]: https://daniellakens.blogspot.com/2015/11/the-relation-between-p-values-and.html https://daniellakens.blogspot.com/2015/11/the-relation-betwe...
  - KirinDave 9y ago
    When I do not have a processor easily capable of bayesian mkmv sampling I'll consider using that method. But I do and I guess this means I am rich? Sure.
- nonbel 9y ago
  Here is what they say they did: >"A Bayesian analysis was performed to test the relative likelihood of observing our results given the null hypothesis (i.e., the absence of any depletion effect) versus the likelihood of observing our results given the alternative hypothesis (the presence of a nonzero effect)." I mean that I have no interest in the result of this "test" at all. First of all, they are not directly testing for the existence of a "depletion effect". From the paper: >"Participants in both conditions were provided with a lengthy passage of text. Those in the Control condition simply crossed out with a pencil all occurrences of the letter ‘e’ in a passage of text, whereas those in the Depletion condition were given the following instructions: ‘cross off the letter “e” every time it appears with the following exceptions: 1. Do not cross out the “e” if it is adjacent to another vowel (e.g., friend); 2. Do not cross out the “e” if it is one letter away from another vowel (e.g., vowel); 3. Do not cross out the “e” if the word has 6 letters (e.g., “there”); 4.Do not cross out the “e” if it is the third to last letter (e.g., customers); 5.Do not cross out the “e” if there are double letters in the word (e.g., “hello”)’. Participants were asked to continue the writing task for six minutes, after which the experimenter instructed participants to discontinue. [...] Participants were given a sheet on which 30 anagrams were listed. Participants were asked to complete as many anagrams as they could within five minutes. After the anagram task, participants were asked to refer to a Likert scale to rate the difficulty of the initial crossing-out letters task (1 = not at all difficult; 7 = very difficult). The number of correctly completed anagrams was recorded for each participant." So "depletion effect" means "got fewer anagrams correct after inspecting text closely for 6 minutes". How about their eyes just got tired from squinting closer at the page? Or they were primed to think/write slower since they had to cross out fewer e's? I'm sure you can come up with more alternatives. Second of all, who is actually predicting exactly zero effect of crossing out the letter e more or less often on filling out anagrams? It seems totally implausible to me there is no effect, especially if you go for longer than 6 minutes. Because that is what they are comparing: exactly zero effect vs any other size effect. This is not "small effect" vs "large effect". Literally any way they can mess up the experiments will tend toward non-zero effect. It is only a matter of sample size. Bayes factors vs p-values is irrelevant to the actual problems with these studies.
- BeetleB 9y ago
  >It's kinda remarkable how many people don't actually use modern bayesian analysis techniques, which is why it's such a policy war. It's the difference between "older statistical methods uninformed by the glut of modern processing power" and "things which require a computer to do well, and preserve uncertainty." There's an old joke: There are two types of statisticians. Those who use Bayesian techniques, and those who use both.