4 ms·
> Except that we don't know how well this 'focus group' forecasting performs relative to voluminous reporting by analysts at State, CIA, RAND, Congressional Res
by jacques_chester 12y ago
> Except that we don't know how well this 'focus group' forecasting performs relative to voluminous reporting by analysts at State, CIA, RAND, Congressional Research, think tanks, etc.
Actually, we do. Philip Tetlock, who is mentioned in the article, spent more than a decade collecting tens of thousands of predictions from geopolitical experts. He found that humans are terrible at making predictions about complex phenomena; typically we are stomped by simple linear models or probablistic random walks.
Some humans were slightly better at predicting outcomes, but compared to simple models, they still sucked.
It didn't matter what other variables were involved. Education, access to classified information, experience, seniority, nationality, profession ... none of them mattered. A Russian politics expert is about as likely to pick Putin's next move as a financial economist who reads the New York Times.
Tetlock's book, Expert Political Judgement, is comprehensive. I reviewed it here: http://chester.id.au/2012/07/29/review-expert-political-judgement/ http://chester.id.au/2012/07/29/review-expert-political-judg...
- rrggrr 12y agoYou don't understand how DC works. On a given issue analysis and projections are produced by numerous agencies. For example a Country desk at State may deliver an accurate estimate, but another group at state may water it down, or it may be stuffed by a competing estimate from CIA. Since its politically safer to bury your head in the sand than it is to fight for an estimate, the process is ripe for manipulation. All this concept does is pass the buck without - apparently - any comparison to all the analysis produced. Tl;dr - This solution speaks to the failings of the intel process and not the people.
- jacques_chester 12y agoTetlock's work was based on individuals giving unfiltered predictions.
- deleted 12y ago[deleted]