4 ms·

For this to be true, we'd had to assume sentience + malevolence. The shortest explanation for why an AI give us correct data currently is a range of confidence.

by sturza 4y ago

For this to be true, we'd had to assume sentience + malevolence. The shortest explanation for why an AI give us correct data currently is a range of confidence. We instructed it to pick the one with the highest number. We instructed it to give us a similar next data etc. We reinforced it, yes, but we'd have to be naive to believe that taking a small part of what makes something "intelligent" and training the shit out of it, then it actually turns intelligent. chatGPT is "just" a large language model, it does not have senses, it cannot move, it cannot live and die. It can only output text. The next gen will output text (maybe) exponentially better. That does not make it "intelligent". But, if we'd combine it with Boston Dynamics, Tesla Autopilot, internet, quantum computing, nano tech and other specifically trained parts (movement, sensing etc) into one, and release it into the world to see what it does, then your argument could (maybe)have a higher confidence of the outcome you fear.

dwohnitmok 4y ago
> For this to be true, we'd had to assume sentience + malevolence. No we don't. A "dumb," "parrot-like" AI is perfectly capable of large amounts of damage if it just happens to be really good at doing something that, in the limit, isn't very good for humans and is good at replication. See for example Meta's Diplomacy-playing bots. They were perfectly capable of natural-language deception at the expense of other humans without any notion f sentience or malevolence. In particular, > It can only output text. As we've seen with both private hobbyists and large companies, there is a race to hook up text to computers with real world access. Outputting text isn't much of a limit if that text ends up as instructions to command a real world system.
- sturza 4y ago
  If i understood your point correctly: the places we decide to use them and our confidence in their output can become negative if we decide to trust the output as is, without any human supervision/decision. So, what you’re saying is that we carry risk if we have a high enough confidence in its output to let is unsupervised - this counteracts my point of the necessity of malevolence, as the (even slightly)wrong reinforcement can have negative consequences also. At the same time, we do this exercise every time we have a new policy in the wild, and sometimes we get unintended consequences.

by sturza 4y ago

dwohnitmok 4y ago

> For this to be true, we'd had to assume sentience + malevolence. No we don't. A "dumb," "parrot-like" AI is perfectly capable of large amounts of damage if it just happens to be really good at doing something that, in the limit, isn't very good for humans and is good at replication. See for example Meta's Diplomacy-playing bots. They were perfectly capable of natural-language deception at the expense of other humans without any notion f sentience or malevolence. In particular, > It can only output text. As we've seen with both private hobbyists and large companies, there is a race to hook up text to computers with real world access. Outputting text isn't much of a limit if that text ends up as instructions to command a real world system.

sturza 4y ago
If i understood your point correctly: the places we decide to use them and our confidence in their output can become negative if we decide to trust the output as is, without any human supervision/decision. So, what you’re saying is that we carry risk if we have a high enough confidence in its output to let is unsupervised - this counteracts my point of the necessity of malevolence, as the (even slightly)wrong reinforcement can have negative consequences also. At the same time, we do this exercise every time we have a new policy in the wild, and sometimes we get unintended consequences.