> How do people determine the "current rate of progress"? There is absolutely no empirical standard to evaluate the performance of AI systems. I would measure

> How do people determine the "current rate of progress"? There is absolutely no empirical standard to evaluate the performance of AI systems. I would measure using something similar to Yudkowsky's challenge: "What is the *least* impressive feat that you would bet big money at 9-1 odds *cannot possibly* be done in 2 years?" [1] Pay a panel of experts to list their predictions each year, including an incentive to get it right, and then measure the percentage of those predictions that fail anyway. [1] https://twitter.com/ESYudkowsky/status/910566159249899520 https://twitter.com/ESYudkowsky/status/910566159249899520