That's a failure mode that's had quite a bit of attention at Less Wrong; see e.g. http://lesswrong.com/lw/ia/focus_your_uncertainty/ http://lesswrong.com/lw/ia/

That's a failure mode that's had quite a bit of attention at Less Wrong; see e.g. http://lesswrong.com/lw/ia/focus_your_uncertainty/ http://lesswrong.com/lw/ia/focus_your_uncertainty/ and http://lesswrong.com/lw/ii/conservation_of_expected_evidence/ http://lesswrong.com/lw/ii/conservation_of_expected_evidence... .