3 ms·
>1. The agent is capable of extreme self improvement on exceedingly short timescales (minutes to days). This really depends what you mean by "self-improvement"
by eli_gottlieb 12y ago
>1. The agent is capable of extreme self improvement on exceedingly short timescales (minutes to days).
This really depends what you mean by "self-improvement". It will probably be easy for an agent to "self-improve" in the sense of just optimizing its code to run more efficiently, in terms of space and time, thus giving it more time and space to think and allowing it to converge towards calculative rationality more quickly. It will probably be very hard (proving you've got the optimal compression/generalization algorithm is incomputable) for an agent to "self-improve" in the sense of improving its compression/learning algorithms to learn more from the same quantity of sensory data.
>2. The AI is pretty much a rational bayesian
All real agents are bounded-rational: "rational Bayesians" like Solomonoff Inducers can't finish calculating their first move before the real world drops a rock on their head. But a bounded-rational agent put in front of a problem well within its bounds can pretend, well enough, to be calculatively rational.
>3. A resource conflict will occur between humans and the AI, and humans will lose because the AI is so much smarter/faster/more powerful than us.
There are already resource conflicts between humans and humans. We tend to solve them with guns.