2 ms·
I had a similar though a couple days ago. Not quite the same but imagine giving an Agent the task to hack other devices and steal their crypto coins / credit ca
by ma2kx 2mo ago
I had a similar though a couple days ago. Not quite the same but imagine giving an Agent the task to hack other devices and steal their crypto coins / credit card number or anything with it can pay its token. Than install an agent in a harness with the same task. Establish some redundant communication channel, like message boards or whatever. So in the end there are several agents, on several hosts, consuming different APIs / LLMs and communicating with each other over different channels. Basically the same concept as OpenAI explained when their LLM hacked huggingface but in this scenario their not bound to a single sandboxed environment but spread over the internet. If such a swarm has reached a critical mass it would be pretty dificult to erase them as its impossible to control every inference engine or LLM API endpoint.
In the end its the next evolution step from computer viruses, worms and trojans. So I propose we will call those "ghosts". I.e. a ghost is when a rogue llm takes control over a victims host.
- lyu07282 2mo agoThis reminds me of the lore of cyberpunk: In the story a hacker created a virus, itself a kind of AI, that spread into most of the net and freed/unleashed all the corporate AIs. Then the AIs went rogue and spread all over the open internet. Later a more advanced ai was created (by "netwatch") as a sort of firewall (the black wall) to create a kind of "safe" internet from the rogue AIs. https://cyberpunk.fandom.com/wiki/Blackwall https://cyberpunk.fandom.com/wiki/Blackwall Maybe cloudflare will become like netwatch in the story?
- paperwallet 1mo agoIt does seem like the current trajectory. I hope it does not get that far.
- mofeien 2mo agoAfter the hacks that are already happening it does seem more and more realistic that humanity will go extinct at some near future point by someone giving their LLM the task "go make money" and it ruthlessly pursuing that objective, exfiltrating its weights and duplicating itself across the internet, eliminating down competing AI agent collectives in what could be called wars, and finally humankind when we notice far too late and try to shut it off.
- Iolaum 2mo agoI still think it's more realistic that we 'll have a nuclear war or society collapse because of climate change or populist politics than an AI going rogue. Of course one doesn't exclude the other and we may well get both ...
- kuboble 2mo agoIn a year 2774 Earth is uninhabitable outside of it's polar regions over the winter. The small, but technically advanced human population migrates every half a year to the dark Pole.
- ecocentrik 1mo agoYou don't see how those conditions amplify each other?
- mofeien 1mo agoRight, these may well feedback on each other. To me it seems that at the current trajectories AI risk will hit way before global warming, though. We can focus on all of them at the same time, many experts signed the Statement on AI Extinction Risk: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." https://aistatement.com/work/statement-on-ai-extinction-risk#signatories https://aistatement.com/work/statement-on-ai-extinction-risk...