2 ms·
I believe that a prerequisite for intelligence is the ability to understand human language, including the ability to sort intent from the literal meaning. Thus
by joe_the_user 12y ago
I believe that a prerequisite for intelligence is the ability to understand human language, including the ability to sort intent from the literal meaning. Thus all arguments of AIs being like genies who do what you say, not what you intend, seems like bs.
Similarly, Nick Bostrom argues an AI would have to make survival something like hidden agenda since survival would help it do all sorts of other things. But lots of humans are quite happy to sacrifice their lives for various sensible and senseless causes, more so than less intelligent beings as far as I can see. So the ability of AIs to willingly die or go into oblivion for us is probably there.
We don't yet know all the qualities of an general AI system and we can't create one either - we don't understand human-level intelligence, which to say we don't understand ourselves. If an AI system could appear at random, it might be a problem but our continuous failure over many years to intentionally build one seems to indicate that a system of this sort will only appear through a rather detailed understanding of its architecture and qualities. At that point, it seems fairly certain the group creating the entity will be able to create it so that it wishes to aid and obey them, to the extent they specify (and probably will create it in such a fashion but you never know).
My argument that the AI will likely just follow the will of its creators, as many high intelligent human servants have followed the wills of their masters and employers and so-forth, sidesteps the "will it be moral?" question. The problem is that humans haven't been particularly moral in their process of creating more and more mechanisms to magnify their intentions. There's no indication that the creation of an AI would change this and so the advent of an AI might be disastrous in many ways. But disastrous through human intent rather than a thing escaping human intent.
Weird this danger gets much less attention. Or not.