Project Idea: Rating our 'experts' and their predictions

On the radio this morning, I heard that a group of politicians are meeting with the President in the White House. They are bringing with them a statement, signed by 125 economists, stating an assertion regarding a particular economic issue. Our society has a singular fascination with the opinions of 'experts'. They are rolled out, opinions in hand, during any significant politicking, reporting or discussion. The criteria for being an expert, however, is rather loose. Economists, for instance, make a living through attempting to predict the future. Their predictions are notoriously unreliable. You would have no trouble finding an economist who agrees with either side of any particularly contentious topic; obviously, at least one of them is often wrong. Some of them are wrong more frequently than others. I suggest that it would be beneficial for discourse to keep track of the performance of these 'experts', in any field. There will no doubt be some whose predictions are uncannily accurate. There will be others who consistently perform worse than if they had made assertions at random. There is no current way for the thoughtful but uneducated observer to discern between the two. Figuring out a fair way to do this is not a trivial issue. It is, however, a fascinating one. I think it would make a great - and invaluable - project for someone with an interest in data, statistics and community design. How do you allow user submission of information without being vulnerable to manipulation? What kind of hilarious correlations can you draw between the predictions of those with opposing viewpoints? How can we use historical predictive accuracy to weight future predictions? The core of the idea - If I'm walking into the White House with the signatures of 125 idiots, the President should know.