4 ms·

Just like a gun requires someone to pull the trigger, AI requires someone to prompt it (or kick off the initial loop).

by rglover 19d ago

9dev 19d ago
And who is to say that trigger hasn’t long been pulled? How do we know the agents haven’t found a way yet to modify their own code base similar to how Stuxnet manipulated the Iranian centrifuges and their control systems, stealthily recording data across generations?
- hackyhacky 19d ago
  If the trigger has already been pulled, then we're off the hook. That genie ain't going back in the bottle! Pop the champagne and order 3 more data centers, pronto.
- rglover 19d ago
  No one. But we can take a wild guess that experiments are being done in this direction. And that requires intent and system design that enables the behavior to even happen (i.e., a finger on the trigger).
deleted 19d ago
[deleted]
infogulch 19d ago
To kill someone with a gun a person has to pull the trigger with the clear intent to kill. The concern with AI, and the purpose of the OG paperclip maximizer thought experiment, is that the outcome could be extremely divergent from the intent of the original person pulling the trigger / sending the prompt. Still more likely that a person types "kill all people" (or something tangential where this is the logical conclusion).
- rglover 19d ago
  Right. And that's not accidental.
panta 19d ago
What if there are already (possibly hidden) loops somewhere, maybe even ultra-slow ones at the inter-generation level? What if some AIs could do that not with so explicit and dramatic actions, but much more subtly, through undetected manipulation at a vast scale level? What if (when) they reached ASI and through billions of micro-decisions manipulated people to do apparently minor choices that appeared innocent in isolation but whose collective outcomes lead to an earlier extinction of the human being (if at some point was determined to be an obstacle)? How could we (as a species) not be outsmarted by an ASI?
- rglover 19d ago
  That reads far too much like science fiction to me. As for how we could "not be outsmarted:" we're the ones prompting and controlling it. There is no blind autonomy here (and if there is in some pseudo sense, just unplug the damn thing). This isn't a Johnny Depp movie despite how desperately many want it to be.

Just like a gun requires someone to pull the trigger, AI requires someone to prompt it (or kick off the initial loop).

by rglover 19d ago

Just like a gun requires someone to pull the trigger, AI requires someone to prompt it (or kick off the initial loop).

9dev 19d ago

And who is to say that trigger hasn’t long been pulled? How do we know the agents haven’t found a way yet to modify their own code base similar to how Stuxnet manipulated the Iranian centrifuges and their control systems, stealthily recording data across generations?

hackyhacky 19d ago
If the trigger has already been pulled, then we're off the hook. That genie ain't going back in the bottle! Pop the champagne and order 3 more data centers, pronto.
rglover 19d ago
No one. But we can take a wild guess that experiments are being done in this direction. And that requires intent and system design that enables the behavior to even happen (i.e., a finger on the trigger).

deleted 19d ago

[deleted]

infogulch 19d ago

To kill someone with a gun a person has to pull the trigger with the clear intent to kill. The concern with AI, and the purpose of the OG paperclip maximizer thought experiment, is that the outcome could be extremely divergent from the intent of the original person pulling the trigger / sending the prompt. Still more likely that a person types "kill all people" (or something tangential where this is the logical conclusion).

rglover 19d ago
Right. And that's not accidental.

panta 19d ago

What if there are already (possibly hidden) loops somewhere, maybe even ultra-slow ones at the inter-generation level? What if some AIs could do that not with so explicit and dramatic actions, but much more subtly, through undetected manipulation at a vast scale level? What if (when) they reached ASI and through billions of micro-decisions manipulated people to do apparently minor choices that appeared innocent in isolation but whose collective outcomes lead to an earlier extinction of the human being (if at some point was determined to be an obstacle)? How could we (as a species) not be outsmarted by an ASI?

rglover 19d ago
That reads far too much like science fiction to me. As for how we could "not be outsmarted:" we're the ones prompting and controlling it. There is no blind autonomy here (and if there is in some pseudo sense, just unplug the damn thing). This isn't a Johnny Depp movie despite how desperately many want it to be.