It is possible to become better calibrated for long-term predictions. This takes a longer amount of time, but is still doable. It's an intuitive process, and th

It is possible to become better calibrated for long-term predictions. This takes a longer amount of time, but is still doable. It's an intuitive process, and the way I do it relies on discovering the true probabilities associated with various feelings of confidence I have in a given domain. The best tool I've yet found to do this is PredictionBook.com. It could use things like tags support (so you could see your accuracy for sports separately from your accuracy for development), but it's still useful overall. I've gotten much better at being accurate, especially for 90% predictions, which is a useful benchmark.