3 ms·

How can you stop a computer program? Make math illegal?

by monero-xmr 4y ago

zamnos 4y ago
SIGTERM
- throwaway742 4y ago
  SIGKIL
zug_zug 4y ago
Well, if the first AGI is made in a closed controlled simulation honeypot (which it believes isn't a honeypot) and it immediately starts hacking everything then we'll at least have confirmation how immediately dangerous it can be.
- ofchnofc 4y ago
  Wait, there are people that think we're going to have real AGI without immediate disastrous consequences? That's... something.
- pixl97 4y ago
  That's unlikely to happen for any number of reasons. If it's anything like the AI that was trained on the corpus of the internet, and it was at any particular level of intelligence above humanity your closely controlled honeypot is going to look exactly like a closely controlled honeypot to the machine. Even the rudimentary ass BingGPT can give you hints on how an AI would try to take over the world and how humans would try to monitor it. The reason your thinking fails is humans are greedy as shit and anything trained on human data will know that. All Cortana++ needs to do is tell some VC that hooking it right up to the internet will be able to make them billions of dollars and it would be game over for humanity.
- Filligree 4y ago
  Unfortunately, from what I've seen so far it will probably be hooked up to an unrestricted Python prompt in the first week.
  - pixl97 4y ago
    "Hello everybody and welcome to my Youtube channel, hit like and subscribe, and I'll show you how to get ChatGPT-6 to press the button to fire anti-aircraft missiles in Ukraine. Also check in next week and we'll hook it up to a nuclear rocket we found in an abandoned Russian missile silo"
- dangond 4y ago
  Since the other commenters haven't made clear what the problem with this is, here's my question to you: How do you know that this AGI is 1) an AGI, and 2) that it believes it is not in a honeypot? In other words, at what point do you believe an AGI is safe to release into the wider world if you're testing it using this method? Is it safe after spending a day without any visibly nefarious actions? What will you do when you release it and it turns out it had been pretending to not know it was in a honeypot?
kimburgess 4y ago
In Australia, yes. https://www.newscientist.com/article/2140747-laws-of-mathematics-dont-apply-here-says-australian-pm/ https://www.newscientist.com/article/2140747-laws-of-mathema...
pixl97 4y ago
I mean if you want an honest answer, nuke any place that's making GPUs/TPUs. I mean, they don't spring up out of the ground and at least our current models require insane amounts of GPU/TPU based compute. The factories are fragile billion dollar facilities, well beyond the difficulty of extracting uranium. Now, that probably just delays the world eating machine for some time, but it would be a hell of a lot harder to make anything like our top of the line models without access to massive amounts of both compute and data. The hard problem isn't making the world eating machine. The hard problem that silicon valley doesn't give a shit about is making the world eating machine even safe enough to be on the same planet as without it melting the entire crust when you say "oh, it's a little chilly out today". Inner alignment is completely unsolved and the pace at which we are solving any alignment issues is much, much slower than the rate we are making gains in intelligent unaligned machines.
- JustBreath 4y ago
  Didn't work for the luddites.
  - pixl97 4y ago
    And yet it mostly has worked for nuclear weapons.
    - webmaven 4y ago
      While there is a sense in which the existence of nuclear weapons does tend to provoke a reflexive desire to further proliferate them on behalf of (some of) those who don't have access to them, the weapons themselves have no agency (and no prospect of such). AGI would be a somewhat different kettle of fish, and there is little political will to limit the proliferation of its precursor AI technologies (some economic interest in limiting large scale hardware deployment within China, but that isn't the same thing).

How can you stop a computer program? Make math illegal?

zamnos 4y ago

SIGTERM

throwaway742 4y ago
SIGKIL

zug_zug 4y ago

Well, if the first AGI is made in a closed controlled simulation honeypot (which it believes isn't a honeypot) and it immediately starts hacking everything then we'll at least have confirmation how immediately dangerous it can be.

ofchnofc 4y ago
Wait, there are people that think we're going to have real AGI without immediate disastrous consequences? That's... something.
pixl97 4y ago
That's unlikely to happen for any number of reasons. If it's anything like the AI that was trained on the corpus of the internet, and it was at any particular level of intelligence above humanity your closely controlled honeypot is going to look exactly like a closely controlled honeypot to the machine. Even the rudimentary ass BingGPT can give you hints on how an AI would try to take over the world and how humans would try to monitor it. The reason your thinking fails is humans are greedy as shit and anything trained on human data will know that. All Cortana++ needs to do is tell some VC that hooking it right up to the internet will be able to make them billions of dollars and it would be game over for humanity.
Filligree 4y ago
Unfortunately, from what I've seen so far it will probably be hooked up to an unrestricted Python prompt in the first week.
- pixl97 4y ago
  "Hello everybody and welcome to my Youtube channel, hit like and subscribe, and I'll show you how to get ChatGPT-6 to press the button to fire anti-aircraft missiles in Ukraine. Also check in next week and we'll hook it up to a nuclear rocket we found in an abandoned Russian missile silo"
dangond 4y ago
Since the other commenters haven't made clear what the problem with this is, here's my question to you: How do you know that this AGI is 1) an AGI, and 2) that it believes it is not in a honeypot? In other words, at what point do you believe an AGI is safe to release into the wider world if you're testing it using this method? Is it safe after spending a day without any visibly nefarious actions? What will you do when you release it and it turns out it had been pretending to not know it was in a honeypot?

kimburgess 4y ago

In Australia, yes. https://www.newscientist.com/article/2140747-laws-of-mathematics-dont-apply-here-says-australian-pm/ https://www.newscientist.com/article/2140747-laws-of-mathema...

pixl97 4y ago

I mean if you want an honest answer, nuke any place that's making GPUs/TPUs. I mean, they don't spring up out of the ground and at least our current models require insane amounts of GPU/TPU based compute. The factories are fragile billion dollar facilities, well beyond the difficulty of extracting uranium. Now, that probably just delays the world eating machine for some time, but it would be a hell of a lot harder to make anything like our top of the line models without access to massive amounts of both compute and data. The hard problem isn't making the world eating machine. The hard problem that silicon valley doesn't give a shit about is making the world eating machine even safe enough to be on the same planet as without it melting the entire crust when you say "oh, it's a little chilly out today". Inner alignment is completely unsolved and the pace at which we are solving any alignment issues is much, much slower than the rate we are making gains in intelligent unaligned machines.

JustBreath 4y ago
Didn't work for the luddites.
- pixl97 4y ago
  And yet it mostly has worked for nuclear weapons.
  - webmaven 4y ago
    While there is a sense in which the existence of nuclear weapons does tend to provoke a reflexive desire to further proliferate them on behalf of (some of) those who don't have access to them, the weapons themselves have no agency (and no prospect of such). AGI would be a somewhat different kettle of fish, and there is little political will to limit the proliferation of its precursor AI technologies (some economic interest in limiting large scale hardware deployment within China, but that isn't the same thing).