3 ms·

> AIs becoming sentient and causing harm for their own ends. No one serious is worried about sentience. It really doesn't matter whether it's sentient, just wh

> AIs becoming sentient and causing harm for their own ends. No one serious is worried about sentience. It really doesn't matter whether it's sentient, just whether we've made a thing that is better at modelling and planning than us and have given it goals that don't perfectly align with how we'd like the world to be. We still aren't that close to something dangerous, but we could end up making one in the next few decades and we have no idea how to ensure we train it to do the right thing once it becomes powerful (humans given power tend to misalign quite badly with our desires as well and homo sapiens discovered contraception and misaligned with the "desires" of evolution fairly recently).

ben_w 4y ago
> No one serious is worried about sentience. Eh. Not for that reason, but I am seriously worried about sentience. As you say, it doesn't matter if it has a feeling about whatever harm it might cause. It does matter (at least to me) if we create one that has the capacity to suffer and is then caused to. That said: > We still aren't that close to something dangerous ICBM early warning radars. Not just Stanislav Petrov; there's also the Thule incident where somebody forgot that the Moon doesn't have an IFF transponder. https://en.wikipedia.org/wiki/Thule_Site_J https://en.wikipedia.org/wiki/Thule_Site_J AI doesn't need to be at any particular quality to kill, it only needs to be incorrect and in use.