4 ms·
Everything about this study says it is not worth taking seriously. Why? Because... 1) The size of the effect is not reported. Without any indication of what th
by tjradcliffe 11y ago
Everything about this study says it is not worth taking seriously. Why? Because...
1) The size of the effect is not reported. Without any indication of what the dose-response curve is other than it has a positive slope (at what significance level?) we know nothing about the relevance of this result to clinical practice.
For example, it is true that if you pay very careful attention to the sidewalk you will increase the amount of dropped money you find. It's not going to make anyone rich, though, any more than just any measurable effect is going to make one less anxious to a useful degree. There is zero indication of how big the effect is, other than it is measureable in a large population. The lack of indication of effect size suggests that it is really small.
2) Self-report via recollection is a lousy data collection technique. Was a food log app really too much trouble? So the raw data are incredibly noisy and may contain systematic errors that are related to the phenomenon under test.
3) They've subsetted the data by pulling out neuroticism, thereby dramatically reducing the significance of any result by an unknown amount (how many other characteristics did they investigate?)
This is the greatest sin. Any time you see a paper that says, "The effect was particularly large in sub-population X" the first question to ask it, "How many sub-populations did you look at?" The next question to ask is, "What is your calculated probability that any one of your sub-populations would have had such a large effect by chance?" These numbers should be reported in the abstract because people subset their data all the time and then analyze the subset that shows an effect as if it was the only one. This is either incompetence or dishonesty.
Statisical significance rolls off with astonishing speed as you subset the data, and if you don't have a formal discipline for subsetting as part of the experiment design that is fixed prior to analysis, any result on a subset is necessarily insignificant because you could have kept subsetting until you found a 'significant' result. If you don't have a formal discipline that says, "We will cut the data in these N ways and only these N ways" you are going fishing on an infinite ocean, where you will certainly catch the kind of fish you are looking for.
- batbomb 11y agoOh yes, I agree. More importantly, food is inherently cultural and varies on class and it might be that neuroticism/social anxiety does as well. I posted this because I thought it sounded like a week study with an overly strong conclusion.
- hollerith 11y ago>I posted this because I thought it sounded like a [weak] study Let me see if I understand: you posted (the abstract of) a study you consider not worth taking seriously so that you could read criticisms of the study by people who know more than you; is that correct? If not, then please explain more about your motivation for posting because I'm a little confused on that point.
- nosefrog 11y agobatbomb did not post the paper.