4 ms·
Mr. Gates writes, "I have been struck by how important measurement is to improving the human condition. You can achieve incredible progress if you set a clear
by cs702 14y ago
Mr. Gates writes, "I have been struck by how important measurement is to improving the human condition. You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal — in a feedback loop ... This may seem basic, but it is amazing how often it is not done and how hard it is to get right."
He's talking from experience. Between the late 1990's and early 2000's, the Bill & Melinda Gates Foundation wasted around $1.7 billion on a broad-ranging, intensive, national effort to study small schools... based on a flawed measurement of Bayesian probabilities!
The foundation looked only at the probability that a school would be small given high school performance, and seemingly forgot to look at the probability that a school would be high-performing given a small school size. The two are not the same. $1.7 billion, poof!
The following analysis, by a professor of statistics, explains in more detail how the foundation got it wrong: http://press.princeton.edu/chapters/s8863.pdf http://press.princeton.edu/chapters/s8863.pdf
- Snoptic 14y agoThe math should be trivial to fix, the expensive collected data is still valid.
- adaml_623 14y agoGuy spent $1.7 Billion testing a hypothesis. Admitted he was wrong. Probably would have been better to fail faster, etc. But it was his money. Kudos to him.
- sliverstorm 14y agoThe $1.7B was not necessarily wasted, if it provided the BMGF learnings that improve future efforts.