4 ms·

It’s crazy to me how much weight the intelligence agencies seem to put on polygraph tests. Some enterprising huckster should sell them on some high-dollar phr

It’s crazy to me how much weight the intelligence agencies seem to put on polygraph tests. Some enterprising huckster should sell them on some high-dollar phrenology gear.

teddyh 7y ago
Polygraph tests “work” because those being tested believe that they work. But people don’t believe in phrenology, so it wouldn’t work.
- mrlala 7y ago
  This seems like the right answer. If the polygraph is able to essentially determine some sort of nervousness beyond normal, then someone who really believes they work and is not used to lying is going to be really damn nervous trying to lie and get caught up. But if someone knows what it really does, and then just knows you just have to be calm and say whatever you want, then it won't work.
  - stallmanite 7y ago
    These are supposed to be smart people. Why would they believe it works? Wouldn’t they google polygraph tests before showing up?
    - domnomnom 7y ago
      What makes you so sure there isn't some sort of classified biomarker or correlate?
      - stallmanite 7y ago
        If it’s some other “secret” test then we’re no longer discussing the polygraph are we?
- close04 7y ago
  I agree with the first part but the second doesn't really follow. How will your belief that phrenology doesn't work influence the result of such a test (assuming you live in a world where such tests are even considered)? Phrenology doesn't allow the same "luxuries" as the polygraph because people are more or less stuck with the size and shape of their skull. Their guilt is exclusively in a chart not in their beliefs. Only thing that matters is if the people (judicial system) giving the test believe in it.
- stallmanite 7y ago
  Ok but are applicants for intelligence jobs that naive/uninformed? I’m not particularly intelligent or informed but I’ve come across articles and books debunking polygraphy many times over the past 20 years. It’s not very obscure information. Scary to think that our “best and brightest” can’t be bothered to do cursory research. Also if I was facing a polygraph you can be damn sure I’d research it ahead of time. Are candidates not doing this?
  - javagram 7y ago
    Even being aware that something is fake doesn’t necessarily mean that you believe it. There are placebo effectiveness studies where the participants know they are taking a placebo pill, for instance, and yet still feel better than a control group who isn’t offered any treatment. https://www.health.harvard.edu/blog/placebo-can-work-even-know-placebo-201607079926 https://www.health.harvard.edu/blog/placebo-can-work-even-kn...
- ChrisSD 7y ago
  The fact that that polygraph tests don't work was well known certainly by the 90's (if not much much earlier). I struggle to believe anyone at all interested in the field wouldn't have some awareness of that, especially if they knew in advance they were going to be tested. And I'd dispute that they "work" even if the subject is unaware of the flaws. Studies have shown innocent people regularly react to assertions of criminality. It's basically left up to the individual tester to "interpret" whether or not they find the person to be lying. Sure sometimes it may inspire someone to admit to something they otherwise wouldn't. But these people are more likely to be largely honest people who made a mistake then hardened criminals or people actively working against the state.
  - traderjane 7y ago
    How is this different from a clinician who has to diagnose autism without even such a handy instrument.
    - skissane 7y ago
      The main test they use for ASD, the Autism Diagnostic Observation Schedule (ADOS) – a Cochrane review [1] found that across 12 studies of diagnosis in preschoolers, the ADOS-1 [2] had a summary specificity of 0.8 – in other words, an average false positive rate of 20%. But, looking at the studies individually, the specificity ranged from 0.2 to 1.0 – in other words, the false positive rate ranged from 0% to 80% across the individual studies. And, that is in research contexts – there are good reasons to believe that the test will be less accurate in clinical use than in research contexts. In order to use the ADOS in research contexts, one has to demonstrate "research reliability", which is a process of scoring videos of the test being performed and comparing your scores with one's colleagues, repeating the process until you get sufficient agreement, and doing this again once every few months to make sure you stay in agreement. None of that rigour is required in clinical use. Added to that, I think in clinical contexts psychologists sometimes feel the urge to "put their thumb on the scales", they'll think "I know the test says this child doesn't have ASD, but I know they are struggling and an ASD diagnosis will get them funding that will help them, so I'll just fudge the numbers a bit to make it come out positive." Some psychologists who would never do that in research (it's scientific fraud) will do that sort of thing clinically. So, it seems very likely (if difficult to prove rigorously) that tests like the ADOS will have less accuracy in clinical contexts than research contexts, so however accurate they are measured to be in these studies, it is probably worse than that in real life. Sometimes, they suggest combining the ADOS with the ADI-R (Autism Diagnostic Interview-Revised), which is a structured interview of parents/caregivers. The combination is even called the "gold standard", with the suggestion that using the two in combination is more accurate than either alone. But, does the research actually support that belief? Well, that Cochrane review says they could only find one study (in preschoolers, they excluded studies of older individuals) which compared the accuracy of ADOS-1+ADI-R to ADOS-1 alone. It found a modest improvement in specificity (equivalent to false positive rate), at the cost of a modest worsening in sensitivity (equivalent to false negative rate), but the result wasn't statistically significant. So, this belief that using the combination improves accuracy doesn't appear to really be supported by the evidence. But, do you think clinicians tell parents any of this? In my personal experience, no. Either they are ignorant of the research, or they are keeping that knowledge close to their chests. How much faith would parents put in the results of the ADOS if they were told that in research contexts it had a 20% false positive rate on average? And that there is good reason to believe that in clinical contexts (their current context) the test probably performs worse than that? And that 20% is just an average over multiple studies, and in some individual studies its false positive rate was substantially higher? [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6513463/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6513463/ [2] the ADOS-1 has been replaced by a new version, the ADOS-2. At the time of that Cochrane review, there hadn't yet been any studies published on using the ADOS-2 in preschoolers, so it was excluded from their analysis. But, the fundamentals of the test haven't changed, so I doubt the ADOS-2 performs significantly better than the ADOS-1 did.
aaronharnly 7y ago
One of my undergrad neuroscience professors used to like to say that phrenology was based on three hypotheses: 1. Cognitive capacities or traits are often centered in particular regions of the brain. 2. The degree of expression or strength of those capacities is correlated with the size of the region. 3. Those size differences can be detected from the outer surface of the skull. Only the third is false. (The first two are true to varying degrees for particular capacities or traits, of course...)
- whatshisface 7y ago
  There's a fourth assumption missing, which is also wrong: that the localized traits are things like "honesty" or "good character" instead of things like "motor function in the upper right torso." My neurology textbook has a great figure from an old phrenology publication, it demonstrates the philosophical idea of there being brain regions for cognitive functions, but the partitions and labels are completely made up.
LanceH 7y ago
How much weight do you think they put in polygraphs? The bulk of the work polygraphs do is encouraging people to admit past deeds. Things a person has done in the past are rarely disqualifying. Hiding them is. The polygraph encourages the candidate to be forthcoming. It's not just the polygraph they are facing, but the background investigation as well. How well do you can lie in front of an investigator (with or without a polygraph) when you know they have been talking to past associates/neighbors/friends? And what is your goal in lying? Getting one over on the stupid military complex. Maybe, just maybe, they aren't as stupid as you think they are.
- stallmanite 7y ago
  Why use a known fraudulent technique? Explain why it’s more beneficial than just thoroughly vetting and interviewing/interrogating candidates. It’s embarrassing. Edit: It’s odd how you seem to be trying to make this a personal/emotional thing about wanting to lie to the MIC, etc. If I was a huge supporter of the intelligence community I’d want them to quit wasting time, money, and credibility on an ineffective scam. Also I explicitly stated that I’m not particularly intelligent so why you are painting me as a know-it-all for daring to question this practice. Enlighten me, please.
  - mc32 7y ago
    Wonder why they haven replaced it with fMRI? It can’t be worse.
    - deleted 7y ago
      [deleted]
  - nyolfen 7y ago
    why do cops interrogate people instead of just investigating them?
    - stallmanite 7y ago
      Interrogation is a real thing that works. Scientology e-meter type bs is not.

It’s crazy to me how much weight the intelligence agencies seem to put on polygraph tests. Some enterprising huckster should sell them on some high-dollar phrenology gear.

teddyh 7y ago

Polygraph tests “work” because those being tested believe that they work. But people don’t believe in phrenology, so it wouldn’t work.

mrlala 7y ago
This seems like the right answer. If the polygraph is able to essentially determine some sort of nervousness beyond normal, then someone who really believes they work and is not used to lying is going to be really damn nervous trying to lie and get caught up. But if someone knows what it really does, and then just knows you just have to be calm and say whatever you want, then it won't work.
- stallmanite 7y ago
  These are supposed to be smart people. Why would they believe it works? Wouldn’t they google polygraph tests before showing up?
  - domnomnom 7y ago
    What makes you so sure there isn't some sort of classified biomarker or correlate?
    - stallmanite 7y ago
      If it’s some other “secret” test then we’re no longer discussing the polygraph are we?
close04 7y ago
I agree with the first part but the second doesn't really follow. How will your belief that phrenology doesn't work influence the result of such a test (assuming you live in a world where such tests are even considered)? Phrenology doesn't allow the same "luxuries" as the polygraph because people are more or less stuck with the size and shape of their skull. Their guilt is exclusively in a chart not in their beliefs. Only thing that matters is if the people (judicial system) giving the test believe in it.
stallmanite 7y ago
Ok but are applicants for intelligence jobs that naive/uninformed? I’m not particularly intelligent or informed but I’ve come across articles and books debunking polygraphy many times over the past 20 years. It’s not very obscure information. Scary to think that our “best and brightest” can’t be bothered to do cursory research. Also if I was facing a polygraph you can be damn sure I’d research it ahead of time. Are candidates not doing this?
- javagram 7y ago
  Even being aware that something is fake doesn’t necessarily mean that you believe it. There are placebo effectiveness studies where the participants know they are taking a placebo pill, for instance, and yet still feel better than a control group who isn’t offered any treatment. https://www.health.harvard.edu/blog/placebo-can-work-even-know-placebo-201607079926 https://www.health.harvard.edu/blog/placebo-can-work-even-kn...
ChrisSD 7y ago
The fact that that polygraph tests don't work was well known certainly by the 90's (if not much much earlier). I struggle to believe anyone at all interested in the field wouldn't have some awareness of that, especially if they knew in advance they were going to be tested. And I'd dispute that they "work" even if the subject is unaware of the flaws. Studies have shown innocent people regularly react to assertions of criminality. It's basically left up to the individual tester to "interpret" whether or not they find the person to be lying. Sure sometimes it may inspire someone to admit to something they otherwise wouldn't. But these people are more likely to be largely honest people who made a mistake then hardened criminals or people actively working against the state.
- traderjane 7y ago
  How is this different from a clinician who has to diagnose autism without even such a handy instrument.
  - skissane 7y ago
    The main test they use for ASD, the Autism Diagnostic Observation Schedule (ADOS) – a Cochrane review [1] found that across 12 studies of diagnosis in preschoolers, the ADOS-1 [2] had a summary specificity of 0.8 – in other words, an average false positive rate of 20%. But, looking at the studies individually, the specificity ranged from 0.2 to 1.0 – in other words, the false positive rate ranged from 0% to 80% across the individual studies. And, that is in research contexts – there are good reasons to believe that the test will be less accurate in clinical use than in research contexts. In order to use the ADOS in research contexts, one has to demonstrate "research reliability", which is a process of scoring videos of the test being performed and comparing your scores with one's colleagues, repeating the process until you get sufficient agreement, and doing this again once every few months to make sure you stay in agreement. None of that rigour is required in clinical use. Added to that, I think in clinical contexts psychologists sometimes feel the urge to "put their thumb on the scales", they'll think "I know the test says this child doesn't have ASD, but I know they are struggling and an ASD diagnosis will get them funding that will help them, so I'll just fudge the numbers a bit to make it come out positive." Some psychologists who would never do that in research (it's scientific fraud) will do that sort of thing clinically. So, it seems very likely (if difficult to prove rigorously) that tests like the ADOS will have less accuracy in clinical contexts than research contexts, so however accurate they are measured to be in these studies, it is probably worse than that in real life. Sometimes, they suggest combining the ADOS with the ADI-R (Autism Diagnostic Interview-Revised), which is a structured interview of parents/caregivers. The combination is even called the "gold standard", with the suggestion that using the two in combination is more accurate than either alone. But, does the research actually support that belief? Well, that Cochrane review says they could only find one study (in preschoolers, they excluded studies of older individuals) which compared the accuracy of ADOS-1+ADI-R to ADOS-1 alone. It found a modest improvement in specificity (equivalent to false positive rate), at the cost of a modest worsening in sensitivity (equivalent to false negative rate), but the result wasn't statistically significant. So, this belief that using the combination improves accuracy doesn't appear to really be supported by the evidence. But, do you think clinicians tell parents any of this? In my personal experience, no. Either they are ignorant of the research, or they are keeping that knowledge close to their chests. How much faith would parents put in the results of the ADOS if they were told that in research contexts it had a 20% false positive rate on average? And that there is good reason to believe that in clinical contexts (their current context) the test probably performs worse than that? And that 20% is just an average over multiple studies, and in some individual studies its false positive rate was substantially higher? [1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6513463/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6513463/ [2] the ADOS-1 has been replaced by a new version, the ADOS-2. At the time of that Cochrane review, there hadn't yet been any studies published on using the ADOS-2 in preschoolers, so it was excluded from their analysis. But, the fundamentals of the test haven't changed, so I doubt the ADOS-2 performs significantly better than the ADOS-1 did.

aaronharnly 7y ago

One of my undergrad neuroscience professors used to like to say that phrenology was based on three hypotheses: 1. Cognitive capacities or traits are often centered in particular regions of the brain. 2. The degree of expression or strength of those capacities is correlated with the size of the region. 3. Those size differences can be detected from the outer surface of the skull. Only the third is false. (The first two are true to varying degrees for particular capacities or traits, of course...)

whatshisface 7y ago
There's a fourth assumption missing, which is also wrong: that the localized traits are things like "honesty" or "good character" instead of things like "motor function in the upper right torso." My neurology textbook has a great figure from an old phrenology publication, it demonstrates the philosophical idea of there being brain regions for cognitive functions, but the partitions and labels are completely made up.

LanceH 7y ago

How much weight do you think they put in polygraphs? The bulk of the work polygraphs do is encouraging people to admit past deeds. Things a person has done in the past are rarely disqualifying. Hiding them is. The polygraph encourages the candidate to be forthcoming. It's not just the polygraph they are facing, but the background investigation as well. How well do you can lie in front of an investigator (with or without a polygraph) when you know they have been talking to past associates/neighbors/friends? And what is your goal in lying? Getting one over on the stupid military complex. Maybe, just maybe, they aren't as stupid as you think they are.

stallmanite 7y ago
Why use a known fraudulent technique? Explain why it’s more beneficial than just thoroughly vetting and interviewing/interrogating candidates. It’s embarrassing. Edit: It’s odd how you seem to be trying to make this a personal/emotional thing about wanting to lie to the MIC, etc. If I was a huge supporter of the intelligence community I’d want them to quit wasting time, money, and credibility on an ineffective scam. Also I explicitly stated that I’m not particularly intelligent so why you are painting me as a know-it-all for daring to question this practice. Enlighten me, please.
- mc32 7y ago
  Wonder why they haven replaced it with fMRI? It can’t be worse.
  - deleted 7y ago
    [deleted]
- nyolfen 7y ago
  why do cops interrogate people instead of just investigating them?
  - stallmanite 7y ago
    Interrogation is a real thing that works. Scientology e-meter type bs is not.