4 ms·

All environments are not created the same, and therefore not equal. If you feed 100 environments that are nearly identical to each other but favor A, and 4 othe

All environments are not created the same, and therefore not equal. If you feed 100 environments that are nearly identical to each other but favor A, and 4 other environments that all favor B but are all dissimilar to each other, then although B is more generalized, A would still win. Since you have no formal way to compare environments to each other, you can't prevent this from happening. Therefore you just pushed the subjectively as to which AI is smarter to which environments are chosen by the user to run against.

xamuel 7y ago
The paper addresses this. Rather than (futilely) attempt to nail down the "one true electorate", the paper defines an infinite family of comparators, depending on a hyperparameter which is, essentially, the choice of which environments get to vote and how to count their votes. Crucially, this doesn't change the truth of the structural theorems (except that some of the theorems require the hyperparameter satisfy certain constraints). Indeed, any attempt to come up with "one true comparison of intelligence" (as opposed to a parametrized family) should be viewed with skepticism, because it really must depend on a lot of arbitrary choices.