4 ms·
That core point is like a variation on a lemma in a paper I recently published (Lemma 2 in [1]): There can be no single deterministic interactive reward-giving
by xamuel 7y ago
That core point is like a variation on a lemma in a paper I recently published (Lemma 2 in [1]): There can be no single deterministic interactive reward-giving environment which can, by its lonesome, serve as a good proxy for general intelligence of deterministic agents.
The proof is fairly simple. Suppose E were such an environment. I claim that every intelligent agent is exactly as intelligent (according to intelligence as measured by performance in E) as some "blind" agent, where by "blind" I mean an agent which totally ignores its surroundings.
Let A be any deterministic agent. If we were to place A in the environment E, then A would take certain actions, call them a1, a2, a3, ..., based on A's interaction with E.
Now define a new agent B as follows. B totally ignores everything and instead just blindly takes actions a1, a2, a3, ...
By construction, B acts exactly the same as A within E, therefore, as measured by E-performance, A and B are equally intelligent. So, for any particular environment, every agent is just as intelligent as some "blind" agent. But that's clearly a very bad property for an alleged intelligence measure to possess!
[1] https://philpapers.org/archive/ALEIVU.pdf https://philpapers.org/archive/ALEIVU.pdf
- elcomet 7y agoI'm not sure I follow. What if you change the initial conditions of the environment? Then A would act differently, but B would still act the same because it is blind. So now they are not taking the same actions. It seems impossible to have a blind agent that follows a non-blind in all situations.
- xamuel 7y agoThe argument is for deterministic environments--no initial condition dependence. If initial conditions are allowed, then you can consider the following environment: initial condition is an arbitrary source-code; the environment proceeds to implement said source-code. Clearly this "environment" is too all-encompasing to be considered as a single environment.
- aristus 7y agoIsn't that basically Searle's Chinese Room argument?
- xamuel 7y agoNot really, unless I'm missing something. They have thematic similarities, I suppose.