5 ms·

I bet they didn't randomise for brand of deodorant and diet which makes this research almost meaningless.

by 2skep 11y ago

knodi123 11y ago
it was a sample of 12 people, so it's already meaningless. this "research" is the justification for a study, not a study in and of itself.
- sliverstorm 11y ago
  How is it simultaneously meaningless and also a justification for further study?
  - opnitro 11y ago
    Meaningless to draw large scale conclusions on. It's a "This is something we should look more closely at" not a "Send this person around the country STAT"
- biomcgary 11y ago
  What makes this result meaningless? The probability of her guessing all 12 correctly at random is: 0.0002 (i.e., approximately 0.5^12). So, it is far more statistically significant than many published results.
- Rexxar 11y ago
  The minimal sample size depends on the effect you try to measure. Big effects can be validated with smaller sample size.
- KingMob 11y ago
  While it's true this is mostly justification for further investigation, correctly categorizing 12/12 people into 2 categories actually has a p-val of .000244141 = (1/(2^12)), which would easily allow you to reject the null hypothesis of random categorization. The stronger the effect, the fewer samples you need. We consider n=12 generally underpowered only because many real-world effects are way weaker than the ability this woman demonstrated.
scott_karana 11y ago
Well, if diet or deodorant causes Parkinson's, that's absolutely meaningful. ;) The methodology should have been mentioned more in the article, and should be scrutinized, but that doesn't mean it's worthless if she truly diagnosed these people after a (single?) blind experiment.
- ars 11y ago
  > Well, if diet or deodorant causes Parkinson's, that's absolutely meaningful. ;) You jest, but there is actually a suggestion that aluminum in sweat blocking deodorant causes neurological problems like Parkinson's and Alzheimer's. (The evidence for this is not strong however.)
  - cortesoft 11y ago
    This wouldn't explain why she noticed a change in her husband's scent. I would assume he was using the same deodorant before and after the change.
    - 2skep 11y ago
      one, that assumption is not tested. two, it is also not tested that the smell she say she noticed had anything to do with parkinson's. I know someone who can predict the future by looking at tea leaves.
azernik 11y ago
Do you have any reason to think brand of deodorant and diet are strongly correlated with Parkinson's? (Especially pre-diagnosis.) Ideal would be to check large numbers of undiagnosed people, and then see how many of those she "alerted" on developed the disease, but given the generally-low incidence of Parkinson's I suspect this approach would be impractical. Larger sample sizes than 12 would always be nice, of course.
- spacehome 11y ago
  Are you kidding? That'd be awesome if we could skip these expensive tests and diagnose Parkinson's by deodorant brand preference!
- 2skep 11y ago
  No, if we discovered that all 6 (+1) happen to use the same deodorant which have the smell she thinks is parkinson's it would make the research look a bit silly.
mfoy_ 11y ago
I can't tell if you're being sarcastic or not... But the "research" is literally just concluding "Hey, there may be a simple way to test for this incredibly hard-to-diagnose disease".
klodolph 11y ago
But in the real world, we do less rigorous studies first and follow up with more rigorous studies if our preliminary investigations show promise.
- mikekchar 11y ago
  I always wonder if this is a good idea. While getting a false positive is not really a problem, because you're going to do a follow up experiment, what happens to the things we miss? If you do an experiment that doesn't really have a large enough sample size, or comes from a biased sample (because it's really an offshoot of a different experiment) and you decided that there is no effect, does it stop others from researching that effect? I suppose since we don't tend to publish negative results maybe it doesn't matter, but it's always something that has niggled at me.
  - KingMob 11y ago
    The trade-off between Type I and Type II error is an inherent problem in research. But false positives are most certainly a problem, too. Just look at the issues psychology and biomedicine have been grappling with in terms of replication. Whole careers were wasted based on what seem now like false positives.
Pyxl101 11y ago
Please avoid gratuitous negativity on Hacker News. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

I bet they didn't randomise for brand of deodorant and diet which makes this research almost meaningless.

by 2skep 11y ago

I bet they didn't randomise for brand of deodorant and diet which makes this research almost meaningless.

it was a sample of 12 people, so it's already meaningless. this "research" is the justification for a study, not a study in and of itself.

sliverstorm 11y ago
How is it simultaneously meaningless and also a justification for further study?
- opnitro 11y ago
  Meaningless to draw large scale conclusions on. It's a "This is something we should look more closely at" not a "Send this person around the country STAT"
biomcgary 11y ago
What makes this result meaningless? The probability of her guessing all 12 correctly at random is: 0.0002 (i.e., approximately 0.5^12). So, it is far more statistically significant than many published results.
Rexxar 11y ago
The minimal sample size depends on the effect you try to measure. Big effects can be validated with smaller sample size.
KingMob 11y ago
While it's true this is mostly justification for further investigation, correctly categorizing 12/12 people into 2 categories actually has a p-val of .000244141 = (1/(2^12)), which would easily allow you to reject the null hypothesis of random categorization. The stronger the effect, the fewer samples you need. We consider n=12 generally underpowered only because many real-world effects are way weaker than the ability this woman demonstrated.

scott_karana 11y ago

Well, if diet or deodorant causes Parkinson's, that's absolutely meaningful. ;) The methodology should have been mentioned more in the article, and should be scrutinized, but that doesn't mean it's worthless if she truly diagnosed these people after a (single?) blind experiment.

ars 11y ago
> Well, if diet or deodorant causes Parkinson's, that's absolutely meaningful. ;) You jest, but there is actually a suggestion that aluminum in sweat blocking deodorant causes neurological problems like Parkinson's and Alzheimer's. (The evidence for this is not strong however.)
- cortesoft 11y ago
  This wouldn't explain why she noticed a change in her husband's scent. I would assume he was using the same deodorant before and after the change.
  - 2skep 11y ago
    one, that assumption is not tested. two, it is also not tested that the smell she say she noticed had anything to do with parkinson's. I know someone who can predict the future by looking at tea leaves.

azernik 11y ago

Do you have any reason to think brand of deodorant and diet are strongly correlated with Parkinson's? (Especially pre-diagnosis.) Ideal would be to check large numbers of undiagnosed people, and then see how many of those she "alerted" on developed the disease, but given the generally-low incidence of Parkinson's I suspect this approach would be impractical. Larger sample sizes than 12 would always be nice, of course.

spacehome 11y ago
Are you kidding? That'd be awesome if we could skip these expensive tests and diagnose Parkinson's by deodorant brand preference!
2skep 11y ago
No, if we discovered that all 6 (+1) happen to use the same deodorant which have the smell she thinks is parkinson's it would make the research look a bit silly.

mfoy_ 11y ago

I can't tell if you're being sarcastic or not... But the "research" is literally just concluding "Hey, there may be a simple way to test for this incredibly hard-to-diagnose disease".

klodolph 11y ago

But in the real world, we do less rigorous studies first and follow up with more rigorous studies if our preliminary investigations show promise.

mikekchar 11y ago
I always wonder if this is a good idea. While getting a false positive is not really a problem, because you're going to do a follow up experiment, what happens to the things we miss? If you do an experiment that doesn't really have a large enough sample size, or comes from a biased sample (because it's really an offshoot of a different experiment) and you decided that there is no effect, does it stop others from researching that effect? I suppose since we don't tend to publish negative results maybe it doesn't matter, but it's always something that has niggled at me.
- KingMob 11y ago
  The trade-off between Type I and Type II error is an inherent problem in research. But false positives are most certainly a problem, too. Just look at the issues psychology and biomedicine have been grappling with in terms of replication. Whole careers were wasted based on what seem now like false positives.

Pyxl101 11y ago

Please avoid gratuitous negativity on Hacker News. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html