5 ms·

Let's apply Bayesian reasoning. :P P(skill): Let's choose a prior that someone can smell parkinson's as one in a million, or 10^-6. (Not very well argued, I ad

Let's apply Bayesian reasoning. :P P(skill): Let's choose a prior that someone can smell parkinson's as one in a million, or 10^-6. (Not very well argued, I admit.) P(data): This exact data's random occurrence probability is 1/2538. P(data|skill): The probability of the result, taking account that she has the skill, is 1 (this assumes she never errs). So we get P(skill|data) = P(data|skill) x P(skill) / P(data) P(skill) = 1 x 10^-6 / (1/2538) = 0.002538 Or 0.25 percent probability, based on this test, that she has the skill. Which is low. Intuitively, I would have expected the calculation to yield a much higher number. The prior was very low though. So I think the grandparent post has some merit. It can be argued that the claim is so extraordinary (the prior) that even twelve "coin tosses" guessed right in a row is more likely.

biomcgary 11y ago
How many people claim they can smell Parkinson's? How many otherwise truthful, sane people have falsely made this claim? Perhaps a better approach to a prior would take into consideration that she claims that she can smell Parkinson's and her reputation as a truthful, sane person. My prior wouldn't be less that 0.1.
- Gravityloss 11y ago
  That's a good point. On the other hand, if it was common for people (one in a thousand) to smell Parkinson's, somebody would have come forward 200 years ago? A doctor or nurse, perhaps. So, on population level it should be extremely rare. Of course, she is not a person picked at random, we should look at the population of people claiming to have done these sort of things already on their own. This population would be expected to contain a large portion of people with mental issues and charlatans trying to gain some financial benefit. If she doesn't have a history of either, then the prior jumps up to a very high level already. It is not very likely for a normal person to claim this sort of thing, unless they already have good evidence by themselves already.
- codeonfire 11y ago
  So, with no scientific basis at all you believe there is a 10% chance that people can detect disease by smell, something that has AFAIK never been proven for any disease? We don't go to the doctor and have him or her smell our armpits. Do you use a psychic? What we have here is an organization that is basing their research funding on math tricks that I, ten years out of undergrad, maybe didn't remember exactly as it was characterized but was able to see through.
  - adenadel 11y ago
    The prior in this case isn't for people in general, it is for this person.
  - KingMob 11y ago
    You may want to revisit your stats, since there's no "math tricks" here. Even if you assume she knew that some sizable portion of the shirts came from AD patients, the p-val is quite low. If you assume every shirt was independent, so there's no knowledge to gained by knowing how many other shirts she classified as AD, the p-val is ludicrously low. And if you go Bayesian, no matter how you slice the prior probabilities, it's still low.
    - codeonfire 11y ago
      First, what p-value are you referring to? You aren't even stating what your test is. Please explain how you got to "p-value is quite low"
      - KingMob 11y ago
        Look at the various analyses in the threads. Mine was a frequentist analysis based on independent categorizations, which has a p-val of ~.0002. Others have posted more sophisticated frequentist and Bayesian analyses based on priors and the subject having advance knowledge of the number of ADs present. But no matter what assumptions were made, no p-val was greater than .001, which is quite low for n=12 with a single test. Our generally accepted threshold is p<.05. She literally had a perfect score. Also, saying "An actual test would need to allow any possible sample including those that had zero Parkinson's patients" indicates you don't understand experimental design. Splitting the data into equal groups maximizes your chances of detecting something when effect sizes are small, since sensitivity is related to minimum group size. (P-values are hurt more by low sample sizes than they gain by large ones, which is why a 3/9 split is less powerful than a 6/6 split.)
StavrosK 11y ago
You're exactly right, if the researchers picked a random person off the street and made them smell shirts. But they didn't.
sov 11y ago
I think your probabilities are way off. We know that some people can smell some disease factors already (fetor hepaticus). Additionally, there's a LOT of genetic smell differences -- cilantro, cucumber, asparagus, cyanide, etc. Knowing these facts drastically affects both the P(smell) and P(skill) (which you've combined into P(skill)). Additionally, the likelihood that a particular person would be so convinced in their ability to smell Parkinson's that they sufficiently convince a few researchers AND THEN predict with perfect accuracy is much smaller than 1/4096. This isn't a random person picked off the street--this is a person specifically claiming to be able to accomplish a feat and then being successful on the first try at 1/4096 odds (1/2^12). Combined, I find this result very interesting and strongly believe the validity of this research. Granted, just because something is likely doesn't necessarily make it true, but it's certainly a strong impetus for future research, or even for people with a similar talent to come forward.
klodolph 11y ago
There's a missing variable here: she claimed that she can tell the difference. We are interested in the following calculation: P(skill|claim&data) = P(claim&data|skill) x P(skill) / P(claim&data) The upper limit of P(claim&data) is P(claim). If P(claim) were above 10^-4, I think the researchers would have met somebody else with the claim, so 10^-4 is a reasonable pessimistic value. So, Bayesian reasoning won't save you here.

biomcgary 11y ago

How many people claim they can smell Parkinson's? How many otherwise truthful, sane people have falsely made this claim? Perhaps a better approach to a prior would take into consideration that she claims that she can smell Parkinson's and her reputation as a truthful, sane person. My prior wouldn't be less that 0.1.

Gravityloss 11y ago
That's a good point. On the other hand, if it was common for people (one in a thousand) to smell Parkinson's, somebody would have come forward 200 years ago? A doctor or nurse, perhaps. So, on population level it should be extremely rare. Of course, she is not a person picked at random, we should look at the population of people claiming to have done these sort of things already on their own. This population would be expected to contain a large portion of people with mental issues and charlatans trying to gain some financial benefit. If she doesn't have a history of either, then the prior jumps up to a very high level already. It is not very likely for a normal person to claim this sort of thing, unless they already have good evidence by themselves already.
codeonfire 11y ago
So, with no scientific basis at all you believe there is a 10% chance that people can detect disease by smell, something that has AFAIK never been proven for any disease? We don't go to the doctor and have him or her smell our armpits. Do you use a psychic? What we have here is an organization that is basing their research funding on math tricks that I, ten years out of undergrad, maybe didn't remember exactly as it was characterized but was able to see through.
- adenadel 11y ago
  The prior in this case isn't for people in general, it is for this person.
- KingMob 11y ago
  You may want to revisit your stats, since there's no "math tricks" here. Even if you assume she knew that some sizable portion of the shirts came from AD patients, the p-val is quite low. If you assume every shirt was independent, so there's no knowledge to gained by knowing how many other shirts she classified as AD, the p-val is ludicrously low. And if you go Bayesian, no matter how you slice the prior probabilities, it's still low.
  - codeonfire 11y ago
    First, what p-value are you referring to? You aren't even stating what your test is. Please explain how you got to "p-value is quite low"
    - KingMob 11y ago
      Look at the various analyses in the threads. Mine was a frequentist analysis based on independent categorizations, which has a p-val of ~.0002. Others have posted more sophisticated frequentist and Bayesian analyses based on priors and the subject having advance knowledge of the number of ADs present. But no matter what assumptions were made, no p-val was greater than .001, which is quite low for n=12 with a single test. Our generally accepted threshold is p<.05. She literally had a perfect score. Also, saying "An actual test would need to allow any possible sample including those that had zero Parkinson's patients" indicates you don't understand experimental design. Splitting the data into equal groups maximizes your chances of detecting something when effect sizes are small, since sensitivity is related to minimum group size. (P-values are hurt more by low sample sizes than they gain by large ones, which is why a 3/9 split is less powerful than a 6/6 split.)

StavrosK 11y ago

You're exactly right, if the researchers picked a random person off the street and made them smell shirts. But they didn't.

sov 11y ago

I think your probabilities are way off. We know that some people can smell some disease factors already (fetor hepaticus). Additionally, there's a LOT of genetic smell differences -- cilantro, cucumber, asparagus, cyanide, etc. Knowing these facts drastically affects both the P(smell) and P(skill) (which you've combined into P(skill)). Additionally, the likelihood that a particular person would be so convinced in their ability to smell Parkinson's that they sufficiently convince a few researchers AND THEN predict with perfect accuracy is much smaller than 1/4096. This isn't a random person picked off the street--this is a person specifically claiming to be able to accomplish a feat and then being successful on the first try at 1/4096 odds (1/2^12). Combined, I find this result very interesting and strongly believe the validity of this research. Granted, just because something is likely doesn't necessarily make it true, but it's certainly a strong impetus for future research, or even for people with a similar talent to come forward.

klodolph 11y ago

There's a missing variable here: she claimed that she can tell the difference. We are interested in the following calculation: P(skill|claim&data) = P(claim&data|skill) x P(skill) / P(claim&data) The upper limit of P(claim&data) is P(claim). If P(claim) were above 10^-4, I think the researchers would have met somebody else with the claim, so 10^-4 is a reasonable pessimistic value. So, Bayesian reasoning won't save you here.