4 ms·

I was thinking more along the lines of things like speed cameras - they capture more speeding violations, don't discriminate based on race/gender/whether or not

I was thinking more along the lines of things like speed cameras - they capture more speeding violations, don't discriminate based on race/gender/whether or not you're driving an expensive car/etc., and they note only the fact that the driver violated the speed limit. The driver gets a fine in the mail shortly after. It improves on the model where the police only pull over a fraction of violators, and you spend 15 minutes waiting, subject to intimidation, further questioning by the officer, the officer gets to glance in your windows to see if there's anything else that violates the law, and in the end you still get a fine. Yet, almost everyone I've talked to about speed cameras hates them and sees them as invasive, sometimes going so far as calling them Orwellian. I agree about the TSA checkpoints, though. There are plenty of examples on both sides of the arguments. My point is that the general assumption seems to gravitate to "they'll implement it stupidly, so they shouldn't have that capability at all". It's not necessarily a binary choice between no security and stupid implementation.

pwg 12y ago
> and they note only the fact that the _vehicle_ violated the speed limit. The _vehicle's owner_ gets a fine in the mail shortly after. There, fixed that for you. Now, I'll admit, a large part of the time the owner is also the driver, but with speed cameras it is not the "driver" who receives the fine, but the _owner of the vehicle_. The one difference with "speed cameras" and the NSA's data mining is that measurement of the rate of travel of a vehicle (assuming a properly calibrated and non-malfunctioning speed measurement device) is so close to 100% accurate as to be considered 100% accurate. No "connect the dots, infer here, infer there" data mining in the NSA's dataset will ever be even within several orders of magnitude of accurate as compared to a properly functioning, properly calibrated, speed measurement device. And that underlying, non zero, error rate is what triggers the base rate fallacy issue in statistical analysis. When an occurrence of X is extremely rare in a very large population without X, then the test for X has to be impossibly accurate for the results to not be swamped by false positives. And this is a completely counter intuitive result, which is why it is so often overlooked. Pertinent quote from the Shiner article at https://www.schneier.com/blog/archives/2006/03/data_mining_for.html https://www.schneier.com/blog/archives/2006/03/data_mining_f...: "Let's look at some numbers. We'll be optimistic. We'll assume the system has a 1 in 100 false positive rate (99% accurate), and a 1 in 1,000 false negative rate (99.9% accurate). Assume one trillion possible indicators to sift through: that's about ten events -- e-mails, phone calls, purchases, web surfings, whatever -- per person in the U.S. per day. Also assume that 10 of them are actually terrorists plotting. This unrealistically-accurate system will generate one billion false alarms for every real terrorist plot it uncovers. Every day of every year, the police will have to investigate 27 million potential plots in order to find the one real terrorist plot per month. Raise that false-positive accuracy to an absurd 99.9999% and you're still chasing 2,750 false alarms per day -- but that will inevitably raise your false negatives, and you're going to miss some of those ten real plots. This isn't anything new. In statistics, it's called the "base rate fallacy," and it applies in other domains as well. For example, even highly accurate medical tests are useless as diagnostic tools if the incidence of the disease is rare in the general population. Terrorist attacks are also rare, any "test" is going to result in an endless stream of false alarms."
- TeMPOraL 12y ago
  > There, fixed that for you. Now, I'll admit, a large part of the time the owner is also the driver, but with speed cameras it is not the "driver" who receives the fine, but the _owner of the vehicle_. Exactly as it should be. It's your responsibility as a vehicle owner to not let bad drivers use your vehicle. RE base rate fallacy - true, but I guess that's why they want even more data. A single test may swamp you with false positives, but positive results from many different sources can correlate into useful info.
  - simoncion 12y ago
    > It's your responsibility as a vehicle owner to not let bad drivers use your vehicle. I read the fine print in everything that I sign. I never noticed any such language in either the state driving regulations, or the restrictions imposed by my automobile insurer. I did notice obligations to operate one's vehicle in a safe and prudent manner at all times.
- csandreasen 12y ago
  Schneier is making the exact same mistake in that article that I pointed out in the Wikipedia article - he is propping up an overly simplistic hypothetical system with a given error rate, then stating that the authorities have to waste their time investigating every single instance without applying any further filtering. I don't know where Schneier is getting his numbers from, as there is significantly less than 1 billion people in the US at any given point in time, and no one in the FBI is going to advocate for using a system that requires them to investigate nearly one tenth of the U.S. population every day. This also doesn't match with real-world numbers that we know on the NSA metadata program. Schneier was saying immediately after 9/11, the FBI was getting thousands of tips per month, which may very well have been the case back then. But the Presidential Review Group report stated that in 2012 a total of 12 tips were passed to the FBI[1]. To me, that indicates that they've become much better at getting rid false positives. Schneier's hypothetical program also isn't analogous because it starts by scanning everyone to determine if one is or is not a terrorist, but the actual NSA metadata program (according to Review Group, same article cited above) starts out by examining people the NSA has already identified as terrorists through other means - the controversy isn't in the accuracy rate, it's in the method of collection. [1] http://www.huffingtonpost.com/geoffrey-r-stone/nsa-meta-data_b_4499934.html http://www.huffingtonpost.com/geoffrey-r-stone/nsa-meta-data...

pwg 12y ago

> and they note only the fact that the _vehicle_ violated the speed limit. The _vehicle's owner_ gets a fine in the mail shortly after. There, fixed that for you. Now, I'll admit, a large part of the time the owner is also the driver, but with speed cameras it is not the "driver" who receives the fine, but the _owner of the vehicle_. The one difference with "speed cameras" and the NSA's data mining is that measurement of the rate of travel of a vehicle (assuming a properly calibrated and non-malfunctioning speed measurement device) is so close to 100% accurate as to be considered 100% accurate. No "connect the dots, infer here, infer there" data mining in the NSA's dataset will ever be even within several orders of magnitude of accurate as compared to a properly functioning, properly calibrated, speed measurement device. And that underlying, non zero, error rate is what triggers the base rate fallacy issue in statistical analysis. When an occurrence of X is extremely rare in a very large population without X, then the test for X has to be impossibly accurate for the results to not be swamped by false positives. And this is a completely counter intuitive result, which is why it is so often overlooked. Pertinent quote from the Shiner article at https://www.schneier.com/blog/archives/2006/03/data_mining_for.html https://www.schneier.com/blog/archives/2006/03/data_mining_f...: "Let's look at some numbers. We'll be optimistic. We'll assume the system has a 1 in 100 false positive rate (99% accurate), and a 1 in 1,000 false negative rate (99.9% accurate). Assume one trillion possible indicators to sift through: that's about ten events -- e-mails, phone calls, purchases, web surfings, whatever -- per person in the U.S. per day. Also assume that 10 of them are actually terrorists plotting. This unrealistically-accurate system will generate one billion false alarms for every real terrorist plot it uncovers. Every day of every year, the police will have to investigate 27 million potential plots in order to find the one real terrorist plot per month. Raise that false-positive accuracy to an absurd 99.9999% and you're still chasing 2,750 false alarms per day -- but that will inevitably raise your false negatives, and you're going to miss some of those ten real plots. This isn't anything new. In statistics, it's called the "base rate fallacy," and it applies in other domains as well. For example, even highly accurate medical tests are useless as diagnostic tools if the incidence of the disease is rare in the general population. Terrorist attacks are also rare, any "test" is going to result in an endless stream of false alarms."

TeMPOraL 12y ago
> There, fixed that for you. Now, I'll admit, a large part of the time the owner is also the driver, but with speed cameras it is not the "driver" who receives the fine, but the _owner of the vehicle_. Exactly as it should be. It's your responsibility as a vehicle owner to not let bad drivers use your vehicle. RE base rate fallacy - true, but I guess that's why they want even more data. A single test may swamp you with false positives, but positive results from many different sources can correlate into useful info.
- simoncion 12y ago
  > It's your responsibility as a vehicle owner to not let bad drivers use your vehicle. I read the fine print in everything that I sign. I never noticed any such language in either the state driving regulations, or the restrictions imposed by my automobile insurer. I did notice obligations to operate one's vehicle in a safe and prudent manner at all times.
csandreasen 12y ago
Schneier is making the exact same mistake in that article that I pointed out in the Wikipedia article - he is propping up an overly simplistic hypothetical system with a given error rate, then stating that the authorities have to waste their time investigating every single instance without applying any further filtering. I don't know where Schneier is getting his numbers from, as there is significantly less than 1 billion people in the US at any given point in time, and no one in the FBI is going to advocate for using a system that requires them to investigate nearly one tenth of the U.S. population every day. This also doesn't match with real-world numbers that we know on the NSA metadata program. Schneier was saying immediately after 9/11, the FBI was getting thousands of tips per month, which may very well have been the case back then. But the Presidential Review Group report stated that in 2012 a total of 12 tips were passed to the FBI[1]. To me, that indicates that they've become much better at getting rid false positives. Schneier's hypothetical program also isn't analogous because it starts by scanning everyone to determine if one is or is not a terrorist, but the actual NSA metadata program (according to Review Group, same article cited above) starts out by examining people the NSA has already identified as terrorists through other means - the controversy isn't in the accuracy rate, it's in the method of collection. [1] http://www.huffingtonpost.com/geoffrey-r-stone/nsa-meta-data_b_4499934.html http://www.huffingtonpost.com/geoffrey-r-stone/nsa-meta-data...