3 ms·
Except that it's a big open question how to control an AGI. So the choice may not be between which entity controls it, but whether it is created by an entity th
by jefftk 3y ago
Except that it's a big open question how to control an AGI. So the choice may not be between which entity controls it, but whether it is created by an entity that is able to control it at all.
- abm53 3y agoIn the space of possibilities, the volume occupied by “not quite smart enough to outwit a corporation but smart enough to outwit a government” seems so vanishingly small as to almost be disregarded entirely.
- londons_explore 3y agoAt least initially, any AGI is likely to be limited by compute capacity. Ie. it might have greater-than-human intelligence, but is only able to make a certain number of decisions/actions per day. However, we have seen with other AI models that as soon as something useful is proved possible, we can expect rapid reductions in the amount of compute necessary to achieve the same.
- stormfather 3y agoThen the first thing the AGI will do is figure out ultra-high throughput analog computing for LLMs
- londons_explore 3y agoAn AGI will likely be scared of even more powerful AGI's that it itself cannot control. It might take steps to try to prevent anyone building such a thing.
- Jensson 3y agoYou assume consciousness here that can't be transferred to the new AI. Without that there is no reason why the AI should fear a more powerful AI. If it is just a dumb process like a flame, then the most efficient version prevails ie the smartest one, if it is conscious and that can be transferred then the AI just sees it as making itself more powerful.
- ben_w 3y agoGiven how many people here regard brain uploads as "not really cheating death", and that AI "can't have consciousness", I think a AGI with consciousness[0] would have a two different ways to have a reasonable chance of fearing that its existence would end with any attempt to transfer itself to a superior design. (Even more so if the architecture of the successor AI is very different, as that would be a bit like one of us being "uploaded" into an LLM via it watching us talk and write). An AGI without consciousness may well act like it has consciousness for the same reason VHS tapes playing on a TV do a good job of reproducing human facial expressions. An AI like this would still act scared of such things, because some of us act scared of such things. [0] it's not obvious to me that an AGI necessarily has to have consciousness
- _heimdall 3y agoAssuming that an AGI will have human-like emotions is likely a mistake. They may very well experience something like emotions that resemble human emotions, but they may not. Or they could have an entirely different set of emotions that we don't recognize. We simply don't know, just like we don't know what other species experience and can only guess at by observing their behavior and trying to map it back to what we know.
- adrianN 3y agoGovernments are reasonably good at controlling humans.
- thelastgallon 3y agoYes, Governments have a monopoly on violence.
- semiquaver 3y agoIt’s not inconceivable that violence will be necessary to control AGI at some point.
- _heimdall 3y agoOne of the many ethical questions we should answer before even attempting to get closer to an AGI. Once a digital intelligence is discovered, does it have rights? Can it be enslaved, controlled, or destroyed? If an AGI can't be granted similar rights as a human, why are corporations legally treated similarly to humans?
- adrianN 3y agoI bet guns kill computers too.