Many of us with some (Bayesian) statistics/game-theory backgrounds are familiar with these kinds of puzzles. There are many (many) such anecdotes littered throu

Many of us with some (Bayesian) statistics/game-theory backgrounds are familiar with these kinds of puzzles. There are many (many) such anecdotes littered through math textbooks (HIV +/- testing comes to mind) as well as some real-life scenarios (UC Berkeley hiring practices and women comes to mind -- 1970s case). HOWEVER, let me just say that two metrics in this little puzzle are simply wrong (or at the very least unfair): P(+ | bad guy) is simply a LOT larger than 0.99 if you want to play the game correctly. The + comes from a POSITIVE outcome -- that is, a terrorist attack is thwarted and someone is thrown in jail. Some noise in the system (i.e. pizza orders) does NOT skew the positives down. After all, discerning between pizza orders and terrorist activity is part of the algorithmic process -- the automated system may flag both cases, but the buck doesn't stop there (or anywhere close). P(+ | good guy), again, suffers from the same problems as P(+ | bad guy). Unless there is evidence of 1% of the people being monitored being thrown in jail for terrorism charges, that number is a lot (lot) less than 1%. There have been plenty of bogus terrorism charges (see Guantanamo) and I think there may be something here. But if we want to play this game correctly, we need to be careful. To do this simulation, we need numbers that we will simply never have access to: e.g. how many terrorist threats did PRISM avert?