4 ms·

This should be really easy to settle, like basic undergrad experimental design, double blind study. Hardly a thorough literature search, but I found this refere

by kortex 5y ago

This should be really easy to settle, like basic undergrad experimental design, double blind study. Hardly a thorough literature search, but I found this reference. > On average, hidden drug samples were indicated by dogs after 64 s searching time, with 87.7% indications being correct and 5.3% being false. In 7.0% of trials dogs failed to find the drug sample within 10 min. ... > Dogs were equally efficient at searching in well-known vs. unknown rooms with strange (i.e., non-target novelty) odors (83.2% correct indications), but they were less accurate when searching outside or inside cars (63.5% and 57.9% correct indications respectively). So that tells me two rough indicators: - There's a ~5% chance a sniffer dog/handler could put away an innocent person. This is a lower bound, the risk is higher with cues/confirmation bias - There's only about ~60-80% sensitivity in unfamiliar situations, of what I presume are pretty fresh samples. You don't have to bust out Bayes' theorem to realize that a cadaver dog accurately hitting scents in the woods months later, stretches credibility. https://www.sciencedirect.com/science/article/abs/pii/S0379073814000371 https://www.sciencedirect.com/science/article/abs/pii/S03790...

Zigurd 5y ago
Dogs are amazingly good at detecting things like cancer in double blind experiments. There are good reasons to think dogs can detect lots of things. That's not the problem: 1. No MD would order cancer treatment on the basis of a dog's sniff. 2. Analysis of all kinds of evidence, including the use of sniffer dogs, should not be managed by police departments, or even prosecutors. There have been many scandals of police evidence handling, even by supposedly professional evidence labs, have mishandled evidence and innocent people have been imprisoned. There needs to be independent analysis of evidence, incentivised to find the truth even if unfavorable to police and prosecutors. 3. Cops lie.
flatline 5y ago
Unfortunately there is more than statistics at play: http://www.wayne-county-forfeiture.com/content/drug-dogs-and-false-alerts-police-lie-and-dogs-wont-sniff-out-perjury http://www.wayne-county-forfeiture.com/content/drug-dogs-and...
jakelazaroff 5y ago
The problem with a purely quantitative analysis of a dog’s theoretical capabilities is that they can be trained to give false positives. See for example this Reason article [1] about dogs searching for drugs: > As a drug detection dog, Karma kept his nose down and treated every suspect the same. Public records show that from the time he arrived in Republic in January 2018 until his handler took a leave of absence to campaign for public office in 2020, Karma gave an "alert" indicating the presence of drugs 100 percent of the time during roadside sniffs outside vehicles. > Similar patterns abound nationwide, suggesting that Karma's career was not unusual. Lex, a drug detection dog in Illinois, alerted for narcotics 93 percent of the time during roadside sniffs, but was wrong in more than 40 percent of cases. Sella, a drug detection dog in Florida, gave false alerts 53 percent of the time. Bono, a drug detection dog in Virginia, incorrectly indicated the presence of drugs 74 percent of the time. [1] https://reason.com/2021/05/13/the-police-dog-who-cried-drugs-at-every-traffic-stop/ https://reason.com/2021/05/13/the-police-dog-who-cried-drugs...
- kortex 5y ago
  Right, that was being generous, assuming the ideal conditions, double blinded, so that any cues from the handler will only add noise and move the needle towards chance. In actuality, the police drug dogs are basically just an excuse to get into the vehicle, so they can justify anything they find post-hoc and ignore all other intrusions.
- iamthemonster 5y ago
  If I was a dog who was given a treat by my owner every time I alerted correctly, but never received a treat any other time, then you better believe I'll be alerting every time.
6gvONxR4sf7o 5y ago
If you do want to pull out bayes theorem, 5% of indications being false in an experimental setting could be way worse in a situation with a lower base rate. (I couldn’t find the base rate in a quick scan of the paper)
- kortex 5y ago
  Yeah I didn't compute the Bayes factor mostly because I'm on mobile and was heading out the door. But when the prior you are plugging in is "did this person murder their son", you need a pretty hefty likelihood ratio to push that into the territory of "lock this person up for life". If you have p(D|H) = 0.8 (hit rate) and p(D|H') = 0.05 (false positive), that gives an LR (likelihood ratio) of 16. But that's if you had like, a perfect detection situation. I'm not a fan, but it's probably enough to push a search into probable cause°, since if you find contraband, you now have field drug tests, which are also kinda crap°°, but it pushes your hypothesis. Then you seize the drugs and have the highly sensitive and selective lab tests of chemical analysis at your disposal. You have a chain of tests updating your prior. We have no such chain in the cabin test. It's just "dog signaled" -> some other circumstantial evidence -> jury convicts of murder. ° - that's assuming the handler is being unbiased, vs just giving the cops an excuse to search. °° - field drug tests basically only catch broad classes of molecules or functional groups, when performed correctly. Like drug dogs, often they too are just theater to intimidate/coerce more searching
ufmace 5y ago
I don't think you can really judge dogs in general based on statistics like this. We can't read a dogs mind and don't know all of their motivations. You can try and do standardized training and testing, but any particular dog might not "get" the training properly, or might decide to do something a little different one day for reasons we don't understand. I'm thinking, it's okay to have a dog to guide you to things, but ultimately, the thing is either there, or it's not, and that's what any convictions should be based upon. If a dog guides you to an actual body, fine, do all your analysis and charges based on that. This whole business of the dog claiming that a body was there months or years ago but we have no idea where it is now seems like bullshit, and I don't think anyone should ever be convicted on that alone. Ditto drugs - search dogs may be imperfect, but ultimately the drugs are either there or they're not. If the dog guides you to actual drugs, charge away I guess. If there aren't any drugs and the dog was wrong, then no charges. It's a little bad that someone got searched based on a false indication, but not nearly as bad as an actual conviction and sentence.

by kortex 5y ago

Zigurd 5y ago

Dogs are amazingly good at detecting things like cancer in double blind experiments. There are good reasons to think dogs can detect lots of things. That's not the problem: 1. No MD would order cancer treatment on the basis of a dog's sniff. 2. Analysis of all kinds of evidence, including the use of sniffer dogs, should not be managed by police departments, or even prosecutors. There have been many scandals of police evidence handling, even by supposedly professional evidence labs, have mishandled evidence and innocent people have been imprisoned. There needs to be independent analysis of evidence, incentivised to find the truth even if unfavorable to police and prosecutors. 3. Cops lie.

flatline 5y ago

Unfortunately there is more than statistics at play: http://www.wayne-county-forfeiture.com/content/drug-dogs-and-false-alerts-police-lie-and-dogs-wont-sniff-out-perjury http://www.wayne-county-forfeiture.com/content/drug-dogs-and...

jakelazaroff 5y ago

The problem with a purely quantitative analysis of a dog’s theoretical capabilities is that they can be trained to give false positives. See for example this Reason article [1] about dogs searching for drugs: > As a drug detection dog, Karma kept his nose down and treated every suspect the same. Public records show that from the time he arrived in Republic in January 2018 until his handler took a leave of absence to campaign for public office in 2020, Karma gave an "alert" indicating the presence of drugs 100 percent of the time during roadside sniffs outside vehicles. > Similar patterns abound nationwide, suggesting that Karma's career was not unusual. Lex, a drug detection dog in Illinois, alerted for narcotics 93 percent of the time during roadside sniffs, but was wrong in more than 40 percent of cases. Sella, a drug detection dog in Florida, gave false alerts 53 percent of the time. Bono, a drug detection dog in Virginia, incorrectly indicated the presence of drugs 74 percent of the time. [1] https://reason.com/2021/05/13/the-police-dog-who-cried-drugs-at-every-traffic-stop/ https://reason.com/2021/05/13/the-police-dog-who-cried-drugs...

kortex 5y ago
Right, that was being generous, assuming the ideal conditions, double blinded, so that any cues from the handler will only add noise and move the needle towards chance. In actuality, the police drug dogs are basically just an excuse to get into the vehicle, so they can justify anything they find post-hoc and ignore all other intrusions.
iamthemonster 5y ago
If I was a dog who was given a treat by my owner every time I alerted correctly, but never received a treat any other time, then you better believe I'll be alerting every time.

6gvONxR4sf7o 5y ago

If you do want to pull out bayes theorem, 5% of indications being false in an experimental setting could be way worse in a situation with a lower base rate. (I couldn’t find the base rate in a quick scan of the paper)

kortex 5y ago
Yeah I didn't compute the Bayes factor mostly because I'm on mobile and was heading out the door. But when the prior you are plugging in is "did this person murder their son", you need a pretty hefty likelihood ratio to push that into the territory of "lock this person up for life". If you have p(D|H) = 0.8 (hit rate) and p(D|H') = 0.05 (false positive), that gives an LR (likelihood ratio) of 16. But that's if you had like, a perfect detection situation. I'm not a fan, but it's probably enough to push a search into probable cause°, since if you find contraband, you now have field drug tests, which are also kinda crap°°, but it pushes your hypothesis. Then you seize the drugs and have the highly sensitive and selective lab tests of chemical analysis at your disposal. You have a chain of tests updating your prior. We have no such chain in the cabin test. It's just "dog signaled" -> some other circumstantial evidence -> jury convicts of murder. ° - that's assuming the handler is being unbiased, vs just giving the cops an excuse to search. °° - field drug tests basically only catch broad classes of molecules or functional groups, when performed correctly. Like drug dogs, often they too are just theater to intimidate/coerce more searching

ufmace 5y ago

I don't think you can really judge dogs in general based on statistics like this. We can't read a dogs mind and don't know all of their motivations. You can try and do standardized training and testing, but any particular dog might not "get" the training properly, or might decide to do something a little different one day for reasons we don't understand. I'm thinking, it's okay to have a dog to guide you to things, but ultimately, the thing is either there, or it's not, and that's what any convictions should be based upon. If a dog guides you to an actual body, fine, do all your analysis and charges based on that. This whole business of the dog claiming that a body was there months or years ago but we have no idea where it is now seems like bullshit, and I don't think anyone should ever be convicted on that alone. Ditto drugs - search dogs may be imperfect, but ultimately the drugs are either there or they're not. If the dog guides you to actual drugs, charge away I guess. If there aren't any drugs and the dog was wrong, then no charges. It's a little bad that someone got searched based on a false indication, but not nearly as bad as an actual conviction and sentence.