I'm guessing they mean something like this: repeatedly pose a random reinforcement-learning problem and record the agent's score. The "IQ" is the Kolmogorov com

I'm guessing they mean something like this: repeatedly pose a random reinforcement-learning problem and record the agent's score. The "IQ" is the Kolmogorov complexity of the problems at which the agent goes from usually acing it to usually bombing. If you pick a random program (as your random problem), its K-complexity is probably near the length of the program, so you don't need to be able to compute the (uncomputable) K-complexity to get a good estimate of the above score. (Based on reading a few words before about Shane Legg's work in the same area, not yet published AFAIK. This is just a guess since there were very few details on that too.) Update: from the paper (thanks to rubidium), on first skim, the above is basically right. They sum up weighted rewards instead of looking for a K-complexity that separates good from bad performance -- which I'd thought of but wasn't sure how to weight scores from problems of different complexity.