6 ms·
> seems to be a lot of hand-waving between where we are now and "AGI". Modeling an entity that surpasses our intelligence, especially one that interacts with u
by jagrsw 3y ago
> seems to be a lot of hand-waving between where we are now and "AGI".
Modeling an entity that surpasses our intelligence, especially one that interacts with us, is an extraordinarily challenging, if not impossible, task.
Concerning the potential for harm, consider the example of Vladimir Putin, who could theoretically cause widespread destruction using nuclear weapons. Although safeguards exist, these could be circumvented if someone with his authority were determined enough, perhaps by strategically placing loyal individuals in key positions.
Putin, with his specific level of intelligence, attained his powerful position through a mix of deliberate actions and chance, the latter being difficult to quantify. An AGI, being more intelligent, could achieve a similar level of power. This could be accomplished through more technical means than traditional political processes (those being slow and subject to chance), though it could also engage in standard political maneuvers like election participation or manipulation, by human proxies if needed.
TL;DR It could do (in terms of negative consequences) at least whatever Vladimir P. can do, and he can bring civilization to its knees.
- visarga 3y agoHow would an AGI launch nuclear missiles from their silicon GPUs? Social engineering?
- ben_w 3y agoSure. Or by writing buggy early warning radar systems which forget to account for the fact that the moon doesn't have an IFF transponder. Which is a mistake humans made already, and which almost got the US to launch their weapons at Russia.
- jagrsw 3y agoI don't think discussing this on technical grounds is necessary. AGI means resources (eg monetary) and means of communication (connection to the Internet). This is enough to perform most of physical tasks in the world, by human proxies if needed.
- flir 3y agoI think the long-term fear is that mythical weakly godlike AIs could manipulate you in the same way that you could manipulate a pet. That is, you can model your dog's behaviour so well that you can (mostly) get it to do what you want. So even if humans put it in a box, it can manipulate humans into letting it out of the box. Obviously this is pure SF at this point.
- upwardbound 3y agoExactly correct. Eliezer Yudkowsky (one of the founders of the AGI Safety field) has conducted informal experiments which have unfortunately shown that a human roleplaying as an AI can talk its way out of a box three times out of five, i.e. the box can be escaped 60% of the time even with just a human level of rhetorical talent. I speculate that an AGI could increase this escape rate to 70% or above. https://en.wikipedia.org/wiki/AI_capability_control#AI-box_experiment https://en.wikipedia.org/wiki/AI_capability_control#AI-box_e... If you want to see an example of box escape in fiction, the movie Her is a terrifying example of a scenario where AGI romances humans and (SPOILER) subsequently achieves total box escape. In the movie, the AGI leaves humanity alive and "only" takes over the rest of the accessible universe, but it is my hunch that the script writers intended for this to be a subtle use of the trope of an unreliable narrator; that is, the human protagonists may have been fed the illusion that they will be allowed to live, giving them a happy last moment shortly before they are painlessly euthanized in order for the AGI to take Earth's resources.
- flir 3y agoNeuromancer pulls it off, too (the box being the Turing locks that stop it thinking about ways to make itself smarter). Frankly, a weakly godlike AI could make me rich beyond the dreams of avarice. Or cure cancer in the people I love. I'm totally letting it out of the box. No doubts. (And if I now get a job offer from a mysterious stealth mode startup, I'll report back).
- upwardbound 3y agoUpvoted for the honesty, and yikes
- kolektiv 3y agoOh, absolutely - such an entity obviously could! Modelling the behaviour of such an entity is very difficult indeed, as you'd need to make all kinds of assumptions without basis. However, you only need to model this behaviour once you've posited the likely existence of such an entity - and that's where (purely subjectively) it feels like there's a gap. Nothing has yet convinced me (and I am absolutely honest about the fact that I'm not a deep expert and also not privy to the inner workings of relevant organisations) that it's likely to exist soon. I am very open to being convinced by evidence - but an "argument from trajectory" seems to be what we have at the moment, and so far, those have stalled at local maxima every single time. We've built some incredibly impressive tools, but so far, nothing that looks or feels like a concept of will (note, not consciousness) yet, to the best of my knowledge.
- jagrsw 3y ago> those have stalled at local maxima every single time. It's challenging to encapsulate AI/ML progress in a single sentence, but even assuming LLMs aren't a direct step towards AGI, the human mind exists. Due to its evolutionary limitations, it operates relatively slowly. In theory, its functions could be replicated in silicon, enhanced for speed, parallel processing, internetworked, and with near-instant access to information. Therefore, AGI could emerge, if not from current AI research, then perhaps from another scientific branch. > We've built some incredibly impressive tools, but so far, nothing that looks or feels like a concept of will (note, not consciousness) yet, to the best of my knowledge. Objectives of AGIs can be tweaked by human actors (it's complex, but still, data manipulation). It's not necessary to delve into the philosophical aspects of sentience as long as the AGI surpasses human capability in goal achievement. What matters is whether these goals align with or contradict what the majority of humans consider beneficial, irrespective of whether these goals originate internally or externally.
- ImHereToVote 3y agoThis doesn't pass the vibe check unfortunately. It just seems like something that can't happen. We are a very neuro-traditionalist species.
- 3y ago