4 ms·
Yes, sorry, that was a bit vague. What I mean is that there are probably classes of systems that consume resources too efficiently for intelligent agents to com
by breuleux 2y ago
Yes, sorry, that was a bit vague. What I mean is that there are probably classes of systems that consume resources too efficiently for intelligent agents to compete. Because an intelligent agent aims to "achieve goals," the space of actions they can undertake is constrained: if they need to do A in order to achieve goal B, they must do A in such a way that it does not undermine B. If they need to build paperclips to bind documents, it would be counterproductive to start an uncontrollable reaction that transforms all matter in the universe, including documents, into paperclips.
So, insofar that intelligent agents must self-limit to controllable methods to extract resources to achieve their goals, it is quite possible that the optimal resource extraction system lies outside of that set. The "thing" that is the very best at consuming the resources that an intelligent agent wishes to consume may be something they would never dare to create, and if it was created, either through hubris or happenstance, the agent would be unable to control it and would therefore be unable to achieve their goals.