6 ms·

You raise a good point. Interestingly, your comments run contrary to what a lot of law enforcement officers have said to us. In general, the police are hyper v

You raise a good point. Interestingly, your comments run contrary to what a lot of law enforcement officers have said to us. In general, the police are hyper vigilant about buying only devices that are independently validated to be very accurate. Every conversation we have had has eventually lead to "is it NHTSA approved?". The reason for this need for third party validation is that the police are incredibly court room sensitive. If there is any chance a defense attorney would be able to pull out a study showing low specificity or sensitivity for a device, the police will simply not buy it. Third party validation gives them that guarantee. You are right that the police want more convictions (or less time consuming convictions), but the way they do that is by having very accurate devices that are defensible in court, not inaccurate devices that risk cases being thrown out. That is why making sure our device has low false positive is very important.

neuralzen 7y ago
I admire what you're doing, and hope you can adhere to scientific integrity while developing and selling your product, but that simply isn't true. At least in Washington state. A few years ago Washington State purchased a whole bunch of Draeger 9510 (alcohol breathalyzer) units without ever having tested them. After two years of sitting unused, the warranties were going to expire and they finally got around to testing them, and purchasing extended warranties at that point. However their initial tests failed multiple accuracy tests to the point that the state's lab halted testing, since it was so off. A few months later Draeger patched the firmware and OS application (the breath/alcohol measurement firmware being written by a single contractor in Texas, and versioned as 0.7) and while testing looked better, it still wasn't accurate and they just pushed it out the door for deployment...they had already bought them, after all. I'll link the records below...they didn't even have the correct versions of the software and firmware accounted for and signed off, which led to other problems (and Draegar has had other problems of accuracy in other court cases, such as Florida, but that's a side tangent). My point is, a lot of state governments (at least in the US) spend money first and verify later, and 'verify' can be a damnably loose term. Loose enough to violate WAC 448-15-020 (in Washington state's case) which requires a 'reasonable degree of scientific accuracy'. EDIT: Apparently the original link (below) is no longer available in WSP's records, which is interesting. You can view the document via archive.org, here https://web.archive.org/web/20170211114622/http://www.wsp.wa.gov/breathtest/docs/webdms/Draeger/Valid_Docs/Draeger%20Validation%20Approval.pdf https://web.archive.org/web/20170211114622/http://www.wsp.wa... original (defunct) http://www.wsp.wa.gov/breathtest/docs/webdms/Draeger/Valid_Docs/Draeger%20Validation%20Approval.pdf http://www.wsp.wa.gov/breathtest/docs/webdms/Draeger/Valid_D...
- SannTek 7y ago
  Thanks for the info. Definitely something we have not run into or considered yet. I guess we just naively assumed that all police precincts were as scrupulous as those we talked to :(. There is some stuff we can do, however. For instance, we can directly build the QC schedule into the device so it does not work unless you get it calibrated. We can also have QC cartridges that have a barcode that the device reads to make sure you actually ran a calibration check properly and the device passed. We were so focused on internal methods for making the device accurate, but maybe we should put some thought into how we can make sure the police use it properly.
  - michaelmrose 7y ago
    Everyone says they are ethical including the unethical I'm not sure how you possibly took this at face value.
  - neuralzen 7y ago
    Really, if you're producing something that is reliably accurate, I would expect that would be all anyone would expect, along with proper training, verified updates, and audited software/firmware, etc. In the case I cited above it was more an issue with the lack of accuracy of the product, and sloppy follow through from the vendor, with money and political pressure thrown at the problem - something which the police departments themselves likely have less influence on anyway.
  - inflatableDodo 7y ago
    maybe we should put some thought into how we can make sure the police use it properly. This times a million. Do not just make the device accurate, make sure it is very hard for a motivated individual to trick it into being inaccurate, in either direction. Please excuse the terminology, was learned from very elderly engineers, but above foolproof (stupidity), there is twatproof (deliberate misuse) and cuntproof (educated malice). Fully cuntproofing is very hard, but any device like this should be at least fully twatproofed, with at least some attempt at cuntproofing.
    - 7y ago
mbrameld 7y ago
> In general, the police are hyper vigilant about buying only devices that are independently validated to be very accurate. How do you reconcile that with the reality of the NIK tests which are notoriously unreliable and ubiquitous among police departments?
lowdose 7y ago
Like the lie detector?

neuralzen 7y ago

I admire what you're doing, and hope you can adhere to scientific integrity while developing and selling your product, but that simply isn't true. At least in Washington state. A few years ago Washington State purchased a whole bunch of Draeger 9510 (alcohol breathalyzer) units without ever having tested them. After two years of sitting unused, the warranties were going to expire and they finally got around to testing them, and purchasing extended warranties at that point. However their initial tests failed multiple accuracy tests to the point that the state's lab halted testing, since it was so off. A few months later Draeger patched the firmware and OS application (the breath/alcohol measurement firmware being written by a single contractor in Texas, and versioned as 0.7) and while testing looked better, it still wasn't accurate and they just pushed it out the door for deployment...they had already bought them, after all. I'll link the records below...they didn't even have the correct versions of the software and firmware accounted for and signed off, which led to other problems (and Draegar has had other problems of accuracy in other court cases, such as Florida, but that's a side tangent). My point is, a lot of state governments (at least in the US) spend money first and verify later, and 'verify' can be a damnably loose term. Loose enough to violate WAC 448-15-020 (in Washington state's case) which requires a 'reasonable degree of scientific accuracy'. EDIT: Apparently the original link (below) is no longer available in WSP's records, which is interesting. You can view the document via archive.org, here https://web.archive.org/web/20170211114622/http://www.wsp.wa.gov/breathtest/docs/webdms/Draeger/Valid_Docs/Draeger%20Validation%20Approval.pdf https://web.archive.org/web/20170211114622/http://www.wsp.wa... original (defunct) http://www.wsp.wa.gov/breathtest/docs/webdms/Draeger/Valid_Docs/Draeger%20Validation%20Approval.pdf http://www.wsp.wa.gov/breathtest/docs/webdms/Draeger/Valid_D...

SannTek 7y ago
Thanks for the info. Definitely something we have not run into or considered yet. I guess we just naively assumed that all police precincts were as scrupulous as those we talked to :(. There is some stuff we can do, however. For instance, we can directly build the QC schedule into the device so it does not work unless you get it calibrated. We can also have QC cartridges that have a barcode that the device reads to make sure you actually ran a calibration check properly and the device passed. We were so focused on internal methods for making the device accurate, but maybe we should put some thought into how we can make sure the police use it properly.
- michaelmrose 7y ago
  Everyone says they are ethical including the unethical I'm not sure how you possibly took this at face value.
- neuralzen 7y ago
  Really, if you're producing something that is reliably accurate, I would expect that would be all anyone would expect, along with proper training, verified updates, and audited software/firmware, etc. In the case I cited above it was more an issue with the lack of accuracy of the product, and sloppy follow through from the vendor, with money and political pressure thrown at the problem - something which the police departments themselves likely have less influence on anyway.
- inflatableDodo 7y ago
  maybe we should put some thought into how we can make sure the police use it properly. This times a million. Do not just make the device accurate, make sure it is very hard for a motivated individual to trick it into being inaccurate, in either direction. Please excuse the terminology, was learned from very elderly engineers, but above foolproof (stupidity), there is twatproof (deliberate misuse) and cuntproof (educated malice). Fully cuntproofing is very hard, but any device like this should be at least fully twatproofed, with at least some attempt at cuntproofing.
  - 7y ago

mbrameld 7y ago

> In general, the police are hyper vigilant about buying only devices that are independently validated to be very accurate. How do you reconcile that with the reality of the NIK tests which are notoriously unreliable and ubiquitous among police departments?

lowdose 7y ago

Like the lie detector?