5 ms·

OK, but the hard part was the "intelligent" part, right? If you have the "intelligent" thing, and you want to get an "intelligent agent", all you have to do at

OK, but the hard part was the "intelligent" part, right? If you have the "intelligent" thing, and you want to get an "intelligent agent", all you have to do at most is wrap it in a shell script that runs the equivalent of an OODA loop, e.g. AutoGPT. So we should expect a "superintelligent agent" to appear pretty much the next day after a "superintelligence" is created.

lukev 3y ago
Perhaps, I don't discount the possibility that an LLM could be used as a primitive as part of a larger more complex system that could rightfully be called intelligent. But a script running a LLM in an OODA loop is a lot easier to monitor and control than some inscrutable "ghost in the parameters" that a lot of AI alarmists seem worried about. You can actually see a transcript of its "thoughts", for one!
- Footkerchief 3y ago
  We already have self-replicating scripts that can prove very hard to detect and destroy, even without the benefit of LLM-powered fuzzing and social engineering.
  - lukev 3y ago
    First of all, we're assuming that agents are (at least initially) designed intentionally, not as harmful viruses. Second, running a LLM inference (nevermind training!) involves lots and lots of system resources. Much more difficult to hide than a tiny backdoor.
joe_the_user 3y ago
OK, but the hard part was the "intelligent" part, right? If you have the "intelligent" thing, and you want to get an "intelligent agent", all you have to do at most is wrap it in a shell script that runs the equivalent of an OODA loop, e.g. AutoGPT. No, not really. Neither AutoGPT or any similar actually work. And the only justification for expecting them to work the future is same naively extrapolation more of one kind of quality will automatically give a thing more of a correlated quality. The humming bird building airplanes: "If we continue to increase the capacity of these flying machines, eventually they will have 'super-bird abilities' and at that point they will naturally steal all of nectar..."
- mquander 3y ago
  AutoGPT doesn't work because GPT-4 is too stupid to make good plans, not because it's missing the secret sauce of "agentiness." If some future software was really great at observing, planning, and reasoning, then an AutoGPT-like wrapper would make it good at autonomously doing things, too.

OK, but the hard part was the "intelligent" part, right? If you have the "intelligent" thing, and you want to get an "intelligent agent", all you have to do at most is wrap it in a shell script that runs the equivalent of an OODA loop, e.g. AutoGPT. So we should expect a "superintelligent agent" to appear pretty much the next day after a "superintelligence" is created.

lukev 3y ago

Perhaps, I don't discount the possibility that an LLM could be used as a primitive as part of a larger more complex system that could rightfully be called intelligent. But a script running a LLM in an OODA loop is a lot easier to monitor and control than some inscrutable "ghost in the parameters" that a lot of AI alarmists seem worried about. You can actually see a transcript of its "thoughts", for one!

Footkerchief 3y ago
We already have self-replicating scripts that can prove very hard to detect and destroy, even without the benefit of LLM-powered fuzzing and social engineering.
- lukev 3y ago
  First of all, we're assuming that agents are (at least initially) designed intentionally, not as harmful viruses. Second, running a LLM inference (nevermind training!) involves lots and lots of system resources. Much more difficult to hide than a tiny backdoor.

joe_the_user 3y ago

OK, but the hard part was the "intelligent" part, right? If you have the "intelligent" thing, and you want to get an "intelligent agent", all you have to do at most is wrap it in a shell script that runs the equivalent of an OODA loop, e.g. AutoGPT. No, not really. Neither AutoGPT or any similar actually work. And the only justification for expecting them to work the future is same naively extrapolation more of one kind of quality will automatically give a thing more of a correlated quality. The humming bird building airplanes: "If we continue to increase the capacity of these flying machines, eventually they will have 'super-bird abilities' and at that point they will naturally steal all of nectar..."

mquander 3y ago
AutoGPT doesn't work because GPT-4 is too stupid to make good plans, not because it's missing the secret sauce of "agentiness." If some future software was really great at observing, planning, and reasoning, then an AutoGPT-like wrapper would make it good at autonomously doing things, too.