3 ms·
> For this to be true, we'd had to assume sentience + malevolence. No we don't. A "dumb," "parrot-like" AI is perfectly capable of large amounts of damage if i
by dwohnitmok 4y ago
> For this to be true, we'd had to assume sentience + malevolence.
No we don't. A "dumb," "parrot-like" AI is perfectly capable of large amounts of damage if it just happens to be really good at doing something that, in the limit, isn't very good for humans and is good at replication.
See for example Meta's Diplomacy-playing bots. They were perfectly capable of natural-language deception at the expense of other humans without any notion f sentience or malevolence.
In particular,
> It can only output text.
As we've seen with both private hobbyists and large companies, there is a race to hook up text to computers with real world access. Outputting text isn't much of a limit if that text ends up as instructions to command a real world system.
- sturza 4y agoIf i understood your point correctly: the places we decide to use them and our confidence in their output can become negative if we decide to trust the output as is, without any human supervision/decision. So, what you’re saying is that we carry risk if we have a high enough confidence in its output to let is unsupervised - this counteracts my point of the necessity of malevolence, as the (even slightly)wrong reinforcement can have negative consequences also. At the same time, we do this exercise every time we have a new policy in the wild, and sometimes we get unintended consequences.