3 ms·

You could just make the AI answer private to the asker. That way there's hopefully minimal harm to the repository of answers, it just stops containing questions

You could just make the AI answer private to the asker. That way there's hopefully minimal harm to the repository of answers, it just stops containing questions that are easy enough for the AI to handle.

startupsfail 4y ago
From the perspective of AI Safety (may get important soon), it is perhaps a very bad idea to allow AI agents to suggest execution/injection of arbitrary code into random places with a review from a single human that needed help from Stack Overflow. [And don’t believe ChatGPT that claims it is an only a Language Model. It is not. It is a RL Agent trained with PPO.]
- andai 4y ago
  That's a fair point. Does the same argument apply to GitHub Copilot / ChatGPT? Does it apply to things that aren't programming? e.g. People are already using these AIs for legal work.
  - startupsfail 4y ago
    One of the first steps of a runaway AI attempting to secure its presence would likely be inference/GPU/TPU compute access. And code injection is a vector. There are multiple other vectors. But, perhaps considering AI as an adversary is a bad idea. And alignment along the “Love is all you need” lines is the actual solution. Tricky problem…
    - andai 4y ago
      I was laughing last night when I realized that even if you want to make an AI that only knows and cares about maximizing paperclip output, you also need to teach it to love all humans unconditionally to prevent the apocalypse.
nl 4y ago
> You could just make the AI answer private to the asker. So then the asker gets the wrong answer and no one else can correct it? Seems even worse.
duskwuff 4y ago
That doesn't solve the problem that GPT-based AIs don't "know" when they're wrong. They will happily spit out authoritative-looking nonsense when presented with questions they don't know how to answer -- and the people asking the questions may not be able to recognize those answers as wrong.

startupsfail 4y ago

From the perspective of AI Safety (may get important soon), it is perhaps a very bad idea to allow AI agents to suggest execution/injection of arbitrary code into random places with a review from a single human that needed help from Stack Overflow. [And don’t believe ChatGPT that claims it is an only a Language Model. It is not. It is a RL Agent trained with PPO.]

andai 4y ago
That's a fair point. Does the same argument apply to GitHub Copilot / ChatGPT? Does it apply to things that aren't programming? e.g. People are already using these AIs for legal work.
- startupsfail 4y ago
  One of the first steps of a runaway AI attempting to secure its presence would likely be inference/GPU/TPU compute access. And code injection is a vector. There are multiple other vectors. But, perhaps considering AI as an adversary is a bad idea. And alignment along the “Love is all you need” lines is the actual solution. Tricky problem…
  - andai 4y ago
    I was laughing last night when I realized that even if you want to make an AI that only knows and cares about maximizing paperclip output, you also need to teach it to love all humans unconditionally to prevent the apocalypse.

nl 4y ago

> You could just make the AI answer private to the asker. So then the asker gets the wrong answer and no one else can correct it? Seems even worse.

duskwuff 4y ago

That doesn't solve the problem that GPT-based AIs don't "know" when they're wrong. They will happily spit out authoritative-looking nonsense when presented with questions they don't know how to answer -- and the people asking the questions may not be able to recognize those answers as wrong.