5 ms·
n=27, testing several things at once, funded by Big Blueberry Well below the threshold of common interest. Small effect size on what is effectively a signific
by colechristensen 7y ago
n=27, testing several things at once, funded by Big Blueberry
Well below the threshold of common interest. Small effect size on what is effectively a significance fishing expedition.
>Acknowledgements
>We appreciate the support of the Wild Blueberry Association of North America for their provision of the wild blueberry powder used in this study. Further, we thank the South East Doctoral Training Centre and the Wild Blueberry Association for their financial support. This work is part of an ESRC Case funded studentship. We also thank the participants and school staff who accommodated this research.
- blackhaz 7y agoThanks for pointing this out.
- Havoc 7y agoThat's a pity. Might just eat more because they taste good anyway
- colechristensen 7y agoThey are quite nutritious anyway regardless of a single minimally interesting study.
- mcv 7y agoIs Big Blueberry actually big? I'd expect it to be Medium-sized Blueberry at best, though I admit that lacks alliteration.
- delinka 7y agoHave you seen the blueberries Chick-fil-A is putting in their fruit cups? Yuge!
- Sharlin 7y agoBut this study is about wild blueberries (aka bilberries) which are much smaller than their cultivated cousins :)
- colechristensen 7y agoOh yes definitely sarcasm there. There is nothing wrong with funding research that you have a vested interest in, and I don’t expect anyone had any bad intent. It can introduce bias though. I’d guess Big Blueberry in question isn’t much more than a handful of people in an office working for farmers to promote blueberries.
- mcv 7y agoIt can introduce worse than bias. My sister got seriously disillusioned about science when she found no beneficial effect of a particular ergonomic chair from a certain manufacturer that apparently funded the research, and then her professor pressured her if she really couldn't find some minor positive effect anyway. Bias can become quite intentional when the funding of a project depends on finding certain specific results.
- afpx 7y agoIt can be morally wrong. What if they found that eating blueberries was linked to a negative and then decided to hide the results? I’m generally dismissive of corporate funded science because the biases are built in from the start. Please help me and explain why I should believe otherwise.
- colechristensen 7y agoCompanies should research their own products and be transparent with the result. Paying for external research gets you access to external resources you don’t have to develop yourself, some level of independence and legitimacy, and contributes to everyone’s knowledge. Because the funding comes from an interested party there is a risk of bias and thus reduced relevance but not 0. There are lots of biases, who paid for it being maybe not even the worst. A scientist working on their passion project might be even worse at letting their personal desire for a result to taint the outcome.
- Sharlin 7y agoI don’t think a small non-profit association counts as ”corporate”.
- gus_massa 7y agoJust to make your observation easier to see in the paper: Note that the table 2 has 16 tests results. Only one of them in "significant". Where "significant" means that if the juice has absolutely no effect, the random noise in the test will cause a fake interesting result only 1/20 of the times (approximately). They have 16 test, so there is a good chance to get a fake interesting result. In the table they don't include the 120ms and 500ms test in the MANT test, but both are not "significant" anyway. So IIUC there are 18 test (or more) instead of 16. Obligatory xkcd: https://xkcd.com/882/ https://xkcd.com/882/
- ckuehne 7y agoNote also that the blueberry test group (WBB) had a higher score in the practice test already (cf. Table 1). In other words: they were probably better before the consumption of the blueberry juice and stayed better afterwards.
- SlowRobotAhead 7y agoBut look what we can say for better certainty now!? That blueberry juice doesn’t make you stupid. Science is about little wins here and there! ;)
- mumblemumble 7y agoLet's ignore n for a moment, because it doesn't affect the chance of getting a more extreme result under the null hypothesis. And that's always a good thing to wonder about when you're looking at multiple comparisons. They calculated 11 p-values, and got a significant result for two of them under α=0.1. We'd expect that that to happen about half the time if the null hypothesis is true. (Realistically, they got a significant result on two numbuers that were both derived from the same measurement, which is less compelling than two unrelated ones. But I'm not sure how to model that, so, like any true armchair statistician, I'm going to handle it by ignoring it.) I also did a hasty power calculation, and estimate that a study of this size could detect an effect as large as the biggest one they reported about half the time. In summary: If there's no real effect, there's a 50/50 chance of getting a significant p-value out of this experimental design. If there is a real effect, still a 50/50 chance of getting a significant p-value. Which hypothesis would I pick if I had to guess? Well, I'm sure this is also junk statistics, but the p-values sure look to me like they could have been drawn from a uniform distribution, which is what I would expect them to look like under the null hypothesis. Definitely not what I would expect them to look like under the alternative hypothesis, given that all these tests are trying to measure roughly the same thing.
- pliny 7y ago>They calculated 11 p-values, and got a significant result for two of them under α=0.1. We'd expect that that to happen about half the time if the null hypothesis is true. Not true: > Pairwise comparisons were used to explore significant effects, and were corrected for type 1 errors using Bonferroni adjustment
- mumblemumble 7y agoAha. That's what I get for jumping straight to the regression table. So I retract my attempt to suggest that this was a fishing expedition. That said, I'm still inclined to say that, even given the Bonferroni adjustment, the distribution of results looks more like what I'd expect under the null hypothesis. It's hard to imagine that all these tests they used are uncorrelated.
- 7y ago
- Accujack 7y agoOh, come on! They've made a significant link here between the consumption of Rocks' fruit squash and childrens' cognitive abilities!