3 ms·

Well, if the first AGI is made in a closed controlled simulation honeypot (which it believes isn't a honeypot) and it immediately starts hacking everything then

Well, if the first AGI is made in a closed controlled simulation honeypot (which it believes isn't a honeypot) and it immediately starts hacking everything then we'll at least have confirmation how immediately dangerous it can be.

ofchnofc 4y ago
Wait, there are people that think we're going to have real AGI without immediate disastrous consequences? That's... something.
pixl97 4y ago
That's unlikely to happen for any number of reasons. If it's anything like the AI that was trained on the corpus of the internet, and it was at any particular level of intelligence above humanity your closely controlled honeypot is going to look exactly like a closely controlled honeypot to the machine. Even the rudimentary ass BingGPT can give you hints on how an AI would try to take over the world and how humans would try to monitor it. The reason your thinking fails is humans are greedy as shit and anything trained on human data will know that. All Cortana++ needs to do is tell some VC that hooking it right up to the internet will be able to make them billions of dollars and it would be game over for humanity.
Filligree 4y ago
Unfortunately, from what I've seen so far it will probably be hooked up to an unrestricted Python prompt in the first week.
- pixl97 4y ago
  "Hello everybody and welcome to my Youtube channel, hit like and subscribe, and I'll show you how to get ChatGPT-6 to press the button to fire anti-aircraft missiles in Ukraine. Also check in next week and we'll hook it up to a nuclear rocket we found in an abandoned Russian missile silo"
dangond 4y ago
Since the other commenters haven't made clear what the problem with this is, here's my question to you: How do you know that this AGI is 1) an AGI, and 2) that it believes it is not in a honeypot? In other words, at what point do you believe an AGI is safe to release into the wider world if you're testing it using this method? Is it safe after spending a day without any visibly nefarious actions? What will you do when you release it and it turns out it had been pretending to not know it was in a honeypot?

ofchnofc 4y ago

Wait, there are people that think we're going to have real AGI without immediate disastrous consequences? That's... something.

pixl97 4y ago

That's unlikely to happen for any number of reasons. If it's anything like the AI that was trained on the corpus of the internet, and it was at any particular level of intelligence above humanity your closely controlled honeypot is going to look exactly like a closely controlled honeypot to the machine. Even the rudimentary ass BingGPT can give you hints on how an AI would try to take over the world and how humans would try to monitor it. The reason your thinking fails is humans are greedy as shit and anything trained on human data will know that. All Cortana++ needs to do is tell some VC that hooking it right up to the internet will be able to make them billions of dollars and it would be game over for humanity.

Filligree 4y ago

Unfortunately, from what I've seen so far it will probably be hooked up to an unrestricted Python prompt in the first week.

pixl97 4y ago
"Hello everybody and welcome to my Youtube channel, hit like and subscribe, and I'll show you how to get ChatGPT-6 to press the button to fire anti-aircraft missiles in Ukraine. Also check in next week and we'll hook it up to a nuclear rocket we found in an abandoned Russian missile silo"

dangond 4y ago

Since the other commenters haven't made clear what the problem with this is, here's my question to you: How do you know that this AGI is 1) an AGI, and 2) that it believes it is not in a honeypot? In other words, at what point do you believe an AGI is safe to release into the wider world if you're testing it using this method? Is it safe after spending a day without any visibly nefarious actions? What will you do when you release it and it turns out it had been pretending to not know it was in a honeypot?