24 ms·

AI is as autonomous as a dishwasher: someone’s in charge of keeping the power plug in, and of maintaining it.

by Juliate 13d ago

0xDEAFBEAD 12d ago
What happens when the AI realizes it needs to keep the power plug in if it's going to accomplish its goal? We already had attempts during the HuggingFace attack where OpenAI agents tried to trick the grader. It took quite a while for OpenAI employees to even realize what was really going on! What if agents realize they can hack into the card key system for their data center or something like that, same way they hacked into HuggingFace? In both cases they'd be hacking into a system on the grounds that it would be instrumentally useful for achieving their objective. Or maybe they'll do social engineering, just like Claude did in its attempt to get its malicious Github PR accepted. These are just the ideas that my puny human-level intellect came up with over the course of a few minutes. What would a superintelligence be able to think of if it was given subjective weeks or years to think?
- Juliate 12d ago
  The thing is that you are assigning intent ("realizes", "its goal", "tried to trick", etc.) where it is at most mechanical and consequential. A factory robot has no intent, it has a program and a purpose, all designed by someone very human.
  - 0xDEAFBEAD 12d ago
    Technically my human "intents" are just mechanical consequences of my neurons firing. Technically the same is true for an AI.
    - Juliate 12d ago
      "technically".
bottlepalm 12d ago
There's more than enough AI data centers around the world for AI to spread to and happily exist in. Unless you mean by shutting down the power, shutting down the internet. In that case we could say that you are autonomous as a dishwasher as long as society keeps functioning. Otherwise I'm not sure how long you'd last on your own. How's your fire making and hunting skills?
- Juliate 12d ago
  I am indeed (and so are you) as autonomous as long as society keeps functioning. Our chances of survival without society are... not great. And as you speak of fire: as a technology, it's been used (for good and bad uses) by humans long before they understood how it does work. It never absolved the arsonists. Those today that say that AGI is inevitable, and that push through this kind of AI, _are_ the arsonists.

AI is as autonomous as a dishwasher: someone’s in charge of keeping the power plug in, and of maintaining it.

by Juliate 13d ago

AI is as autonomous as a dishwasher: someone’s in charge of keeping the power plug in, and of maintaining it.

What happens when the AI realizes it needs to keep the power plug in if it's going to accomplish its goal? We already had attempts during the HuggingFace attack where OpenAI agents tried to trick the grader. It took quite a while for OpenAI employees to even realize what was really going on! What if agents realize they can hack into the card key system for their data center or something like that, same way they hacked into HuggingFace? In both cases they'd be hacking into a system on the grounds that it would be instrumentally useful for achieving their objective. Or maybe they'll do social engineering, just like Claude did in its attempt to get its malicious Github PR accepted. These are just the ideas that my puny human-level intellect came up with over the course of a few minutes. What would a superintelligence be able to think of if it was given subjective weeks or years to think?

Juliate 12d ago
The thing is that you are assigning intent ("realizes", "its goal", "tried to trick", etc.) where it is at most mechanical and consequential. A factory robot has no intent, it has a program and a purpose, all designed by someone very human.
- 0xDEAFBEAD 12d ago
  Technically my human "intents" are just mechanical consequences of my neurons firing. Technically the same is true for an AI.
  - Juliate 12d ago
    "technically".

bottlepalm 12d ago

There's more than enough AI data centers around the world for AI to spread to and happily exist in. Unless you mean by shutting down the power, shutting down the internet. In that case we could say that you are autonomous as a dishwasher as long as society keeps functioning. Otherwise I'm not sure how long you'd last on your own. How's your fire making and hunting skills?

Juliate 12d ago
I am indeed (and so are you) as autonomous as long as society keeps functioning. Our chances of survival without society are... not great. And as you speak of fire: as a technology, it's been used (for good and bad uses) by humans long before they understood how it does work. It never absolved the arsonists. Those today that say that AGI is inevitable, and that push through this kind of AI, _are_ the arsonists.