3 ms·
The argument that an imaginary AGI is going to take over the world because of an imaginary principle is not actually very convincing though. It would be better
by astrange 4y ago
The argument that an imaginary AGI is going to take over the world because of an imaginary principle is not actually very convincing though. It would be better if you said they'd do something humans already did; at least we know that'd be possible. Most of this "instrumental convergence" assumes that it has an infinite attention span, infinite energy, all its plans work properly… the only reason it all goes that well is you're imagining it did.
I think last week someone on twitter told me an AI could be bad because it'd "kill us all by releasing viruses into the atmosphere", but if you had a baby it could also grow up to do that, and (correctly) nobody worries about this. Not clear why these are supposed to be different.
More importantly, it mostly relies on thinking AGIs behave like computer programs in that 1. they don't get bored 2. they can be copied or copy themselves. Is that necessarily true? If someone copied you could you take over the world easier? Maybe you'd just fight over who gets the bank account.