GPT-4 is not alive or anything like that but it is intelligent. And it's approximately human level now. Computer hardware historically improves exponentially.

GPT-4 is not alive or anything like that but it is intelligent. And it's approximately human level now. Computer hardware historically improves exponentially. We should anticipate a system that thinks 100 times faster than a human in a few years or less. This is barely even speculative. Two orders of magnitude performance improvement has been achieved many, many times over in the history of computing. This is a very specific application and there is still probably relatively low hanging fruit for optimizing it with hardware and software customization. New compute-in-memory paradigms are coming soon. The problem with hyperspeed GPT is that in an environment where companies and countries are competing, there is a strong incentive to give the AI very broad goals. Because having it wait a day for the humans to make decisions would mean 100 days of progress made by competitors (if it is at that speed). The human operators are effectively frozen relative to the activity of this speed of agent. They become speculators on the sidelines for the most part. Also, you can't prevent people from instructing it to pursue its own selfish goals or having self-preservation or reproductive (copying code/model weights) goals. These will be open source models. I think the only answer is that you ban the manufacture of AI hardware that exceeds a certain level of performance relative to humans. Also immediately start strongly discouraging people from simulating lifelike qualities in intelligent digital systems.