3 ms·

An interesting application of Bayesian reasoning to the problem of screening for low-base-rate phenomena. (And terrorist criminal activity is a lower-base-rate

An interesting application of Bayesian reasoning to the problem of screening for low-base-rate phenomena. (And terrorist criminal activity is a lower-base-rate phenomenon in the United States, so far, than prostate cancer or other dangers that are screened for.) The mathematics, of course, is exquisitely sensitive to exactly how sensitive and specific a screening program is. (By the way, so far news reports are saying that the NSA program that is all over the front page of Hacker News, deservedly so in my opinion, is not a screening program but a data collection program, with analysis of the collected data triggered only by other kinds of law enforcement evidence-gathering. We'll see what further reporting says about that issue.)

suredo 13y ago
There is only one reason for collecting this data, to analyze it... and the biggest obstacle is the gathering of that data.
- rdtsc 13y ago
  Well I guess technically there is this pesky thing getting in the way called the Constitution. But they re-defined some terms, moved some things around, decided that 'search' probably occurs when a human looks at the data (LOL, founding fathers didn't know about databases ;-). So it is not just a technical challenge, given their budgets they can solve the technical side it seems. Imagine also (and I've mentioned this before) there is no statute of limitations on historical data in case of warrant/nsl, so if warrant is every issued (and we know they are never denied) they can get _all_ your data from years and years back. Emails you sent 10 years ago to secret lovers, jokes about punching the president in the face when you were 13 years old, stuff like that. Make no mistake they will find stuff to blackmail, jail, or scare anyone into anything if they dig hard and long enough into everyone's past. I suspect more of the people involved in this will decide to do what Manning did. They see the waste, the lies, wars on terror, wars on drugs. If anything, NSA and other XYZ agencies, love to hire patriotic people, and a fraction of a percent of them maybe still are and will say "fuck it, everyone needs to know about this".

suredo 13y ago

There is only one reason for collecting this data, to analyze it... and the biggest obstacle is the gathering of that data.

rdtsc 13y ago
Well I guess technically there is this pesky thing getting in the way called the Constitution. But they re-defined some terms, moved some things around, decided that 'search' probably occurs when a human looks at the data (LOL, founding fathers didn't know about databases ;-). So it is not just a technical challenge, given their budgets they can solve the technical side it seems. Imagine also (and I've mentioned this before) there is no statute of limitations on historical data in case of warrant/nsl, so if warrant is every issued (and we know they are never denied) they can get _all_ your data from years and years back. Emails you sent 10 years ago to secret lovers, jokes about punching the president in the face when you were 13 years old, stuff like that. Make no mistake they will find stuff to blackmail, jail, or scare anyone into anything if they dig hard and long enough into everyone's past. I suspect more of the people involved in this will decide to do what Manning did. They see the waste, the lies, wars on terror, wars on drugs. If anything, NSA and other XYZ agencies, love to hire patriotic people, and a fraction of a percent of them maybe still are and will say "fuck it, everyone needs to know about this".