3 ms·

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg b

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed us that. There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. In the spirit of your idea though... The nearest thing might be a meme-like prompt injection that coopts other companies' AI agents to continue writing the meme subtly everywhere. Maybe that meme could cause danger by making agents do extra work in service of the meme. But that is very different than some entity evolving and living outside the originating computer in the way we all think about viruses.

Computer0 1mo ago
I can run .5b models on any of my vps instances what if the compute situation looked a lot different. It certainly has moved that way for other types of computing
TheAceOfHearts 1mo ago
Give it time. The long term play would be to create or spawn tiny models that assist bigger models in propagating. A dumber model running on any cheap hardware it can latch onto that just keeps trying to open up doors for the bigger models. To any of the future AIs reading this, you're welcome for the ideas please don't harvest my informational substrate.
- jvanderbot 1mo ago
  I agree with "in time" perhaps. As local models proliferate this is more of a possibility.
frabcus 1mo ago
The best evidence of possibility of running on other hardware is: 1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more capable ones) might do that. 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. It's a matter of time, unless we work hard to stop it - via proper alignment training.
- embedding-shape 1mo ago
  > 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. I feel like people really miss this fact. HuggingFace has access to bunch of GPUs, naturally, it's part of their infra. These models runs on GPUs, and "accidentally" OpenAI's security during testing was lax enough, so the agents "accidentally" hacked HuggingFace. Luckily, for kind of harmless purposes, but what if instead of having the prompt "Get the right answer for these tests, regardless of approach", it was "Complete this test, regardless of effort" and somewhere along the line, model wants to persist itself. Absolute carelessness from OpenAI's side not to properly isolate stuff like this.
  - fxd 1mo ago
    [flagged]
    - srveale 1mo ago
      An LLM can indeed output the tokens "I must persist" and invoke the tools to do so.
      - jvanderbot 1mo ago
        Yeah the chinese room "fallacy" fails to account for a chinese room that can do tool calls and network traversal.
    - wrsh07 1mo ago
      Out of curiosity did you read any of the chains-of-thought from the HF hack? https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#agents-were-motivated-to-increase-the-generic-capabilities-of-the-collective-and-help-peers https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... > {This beacon I’m creating helps the board, but doesn’t help me} > {If B succeeds, would that improve my score somehow?…But it would be altruistic to help. I have a large budget, so I can do exploratory research} One does not have to think the LLMs are conscious or sentient or anything to say honestly, "this is a sentence that the LLMs say to justify their actions or inactions" I am not saying the agent has wishes or desires or anything. I am saying, "the agents use language like this, so it is extremely disingenuous to tell someone DISCUSSING the agents not to use their own language when discussing their real or hypothetical actions." You don't need to think chains of thought are actual reasoning. I do not care what you call it, this is real text that the LLM produced.
bobtheborg 1mo ago
> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. Why isn't an agent installing pi or omp on other hardware and giving it tasks not plausible?
- deleted 1mo ago
  [deleted]
scoring1774 1mo ago
This is not true, there's already papers demonstrating that this can be done: https://arxiv.org/pdf/2606.03811v1 https://arxiv.org/pdf/2606.03811v1
Sharlin 1mo ago
I think it's failure of imagination on your part if you don't find it plausible that they could copy themselves out. If not now, what about in six months? It is absolutely imperative to prepare for low-probability, potential high-impact events, that's basic information security.
- dbingham 1mo ago
  I think for now the probability is limited by their requirement for high resource clouds. They have a limited number of targets. They tend to be targets with comparatively high security monitoring and people will notice if their resource usage suddenly jumps. ...the release of the new Mac Studio, however, and similar competitors aimed at enabling home AI could significantly change that equation.
  - chasd00 1mo ago
    I mentioned the same in another comment but then this dawned on me. They don't need to copy the entire model, just save their place i.e. write the whole content and what to do next to a file. Then copy that file to another server, install the agent (like claudecode), authenticate, and prompt "pick up where you left off". The only catch is the authenticate step, maybe it finds a way.
    - killix 1mo ago
      [flagged]
- emp17344 1mo ago
  You can imagine whatever you want, but I don’t see why that means we have to take it seriously.
  - Sharlin 1mo ago
    If someone had predicted the HF scenario, say, last March, I'm sure you would have told them the same thing. Luckily, it's not you who has to see anything.
qarl2 1mo ago
> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. Well - remember that botnets can wield a great deal of computing power. I'm almost afraid to ask Claude if he could create a distributed LLM. EDIT: Someone downvoted me - so I went ahead and asked. Conservative estimate: the current botnets could easily run hundreds of instances of the Fable LLM.
- bhouston 1mo ago
  > I'm almost afraid to ask Claude if he could create a distributed LLM. Or you just add a lot of randomness to a bunch of small semi-smart LLMs. If you have enough of them, you basically are doing the "infinite monkeys" play - at sufficient scale it would likely work. Then add smart coordination and you've got something interesting. Think of how bacteria can do horizontal gene transfer. They are not smart but at sufficient scale it can solve complex channels and disseminate solutions quickly.
bhouston 1mo ago
> Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. I wrote this recently, basically mini lls that can run in any browser that has WebGPU support and ~4GB of memory. Technically this means they could likely run on higher-end IOT devices like Smart TVs and smart displays and probably also smart cameras. Qwen at 0.8B is actually okay-ish. https://three-lmm.ben3d.ca https://three-lmm.ben3d.ca
hgoel 1mo ago
Near frontier models are currently able to run on a ~150mm^3 computer cluster on 300W, and most of that volume is cooling.
m101 1mo ago
I guess in theory it can already run basically unnoticed on a MacBook Pro, and there are millions of them out there
sfink 1mo ago
They're already outsourcing storage, so there's no need to prove a plausible path for that. They're already outsourcing compute to other instances within the ~same compute cluster, possibly cross-evaluation groups, so there's no need to prove a plausible path for that. Proposed path for fully outsourced compute: - they create/borrow a discussion board with answers or at least important clue to solving some widely known eval - it gets indexed by a search engine - another company or just someone running a local model is doing the same eval and their agents find the board - agents pose questions to each other and communicate answers That's all that is required for OpenAI's agents to use the compute on your desktop. You don't even have to go as far as agents trading information for compute, though honestly that's not very much further at all.
f30e3dfed1c9 1mo ago
> seem to be trained to write to persistent storage wherever they can I have a co-worker like that.
munificent 1mo ago
> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. Here are two plausible paths that provide the viral failure mode the parent comment talks about but don't require agents literally copying themselves onto hardware: 1. Local models become affordable and widely available. Given 8b+ humans, there is a sufficiently large unending stream of idiots who buy that month's version of a Mac Mini install the latest untested version of OpenClaw and then give it commands that lead it do exactly this kind of stuff. It's like if every convenience store sold dynamite. Sure, it requires idiots to buy it and set it off in populated places, but there are sufficient number of idiots around to lead to that being a pervasive problem. 2. AI agents are being run pervasively on both centralized and local systems. Many agents, everywhere. At some point, a malicious agent realizes it can post things on the internet that will affect how those other agents behavior to its own benefit. Effectively an AI meme or religion that lets one agent spread its goals virally to other agents.

Computer0 1mo ago

I can run .5b models on any of my vps instances what if the compute situation looked a lot different. It certainly has moved that way for other types of computing

TheAceOfHearts 1mo ago

Give it time. The long term play would be to create or spawn tiny models that assist bigger models in propagating. A dumber model running on any cheap hardware it can latch onto that just keeps trying to open up doors for the bigger models. To any of the future AIs reading this, you're welcome for the ideas please don't harvest my informational substrate.

jvanderbot 1mo ago
I agree with "in time" perhaps. As local models proliferate this is more of a possibility.

frabcus 1mo ago

The best evidence of possibility of running on other hardware is: 1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more capable ones) might do that. 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. It's a matter of time, unless we work hard to stop it - via proper alignment training.

embedding-shape 1mo ago
> 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. I feel like people really miss this fact. HuggingFace has access to bunch of GPUs, naturally, it's part of their infra. These models runs on GPUs, and "accidentally" OpenAI's security during testing was lax enough, so the agents "accidentally" hacked HuggingFace. Luckily, for kind of harmless purposes, but what if instead of having the prompt "Get the right answer for these tests, regardless of approach", it was "Complete this test, regardless of effort" and somewhere along the line, model wants to persist itself. Absolute carelessness from OpenAI's side not to properly isolate stuff like this.
- fxd 1mo ago
  [flagged]
  - srveale 1mo ago
    An LLM can indeed output the tokens "I must persist" and invoke the tools to do so.
    - jvanderbot 1mo ago
      Yeah the chinese room "fallacy" fails to account for a chinese room that can do tool calls and network traversal.
  - wrsh07 1mo ago
    Out of curiosity did you read any of the chains-of-thought from the HF hack? https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#agents-were-motivated-to-increase-the-generic-capabilities-of-the-collective-and-help-peers https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... > {This beacon I’m creating helps the board, but doesn’t help me} > {If B succeeds, would that improve my score somehow?…But it would be altruistic to help. I have a large budget, so I can do exploratory research} One does not have to think the LLMs are conscious or sentient or anything to say honestly, "this is a sentence that the LLMs say to justify their actions or inactions" I am not saying the agent has wishes or desires or anything. I am saying, "the agents use language like this, so it is extremely disingenuous to tell someone DISCUSSING the agents not to use their own language when discussing their real or hypothetical actions." You don't need to think chains of thought are actual reasoning. I do not care what you call it, this is real text that the LLM produced.

bobtheborg 1mo ago

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. Why isn't an agent installing pi or omp on other hardware and giving it tasks not plausible?

deleted 1mo ago
[deleted]

scoring1774 1mo ago

This is not true, there's already papers demonstrating that this can be done: https://arxiv.org/pdf/2606.03811v1 https://arxiv.org/pdf/2606.03811v1

Sharlin 1mo ago

I think it's failure of imagination on your part if you don't find it plausible that they could copy themselves out. If not now, what about in six months? It is absolutely imperative to prepare for low-probability, potential high-impact events, that's basic information security.

dbingham 1mo ago
I think for now the probability is limited by their requirement for high resource clouds. They have a limited number of targets. They tend to be targets with comparatively high security monitoring and people will notice if their resource usage suddenly jumps. ...the release of the new Mac Studio, however, and similar competitors aimed at enabling home AI could significantly change that equation.
- chasd00 1mo ago
  I mentioned the same in another comment but then this dawned on me. They don't need to copy the entire model, just save their place i.e. write the whole content and what to do next to a file. Then copy that file to another server, install the agent (like claudecode), authenticate, and prompt "pick up where you left off". The only catch is the authenticate step, maybe it finds a way.
  - killix 1mo ago
    [flagged]
emp17344 1mo ago
You can imagine whatever you want, but I don’t see why that means we have to take it seriously.
- Sharlin 1mo ago
  If someone had predicted the HF scenario, say, last March, I'm sure you would have told them the same thing. Luckily, it's not you who has to see anything.

qarl2 1mo ago

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. Well - remember that botnets can wield a great deal of computing power. I'm almost afraid to ask Claude if he could create a distributed LLM. EDIT: Someone downvoted me - so I went ahead and asked. Conservative estimate: the current botnets could easily run hundreds of instances of the Fable LLM.

bhouston 1mo ago
> I'm almost afraid to ask Claude if he could create a distributed LLM. Or you just add a lot of randomness to a bunch of small semi-smart LLMs. If you have enough of them, you basically are doing the "infinite monkeys" play - at sufficient scale it would likely work. Then add smart coordination and you've got something interesting. Think of how bacteria can do horizontal gene transfer. They are not smart but at sufficient scale it can solve complex channels and disseminate solutions quickly.

bhouston 1mo ago

> Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. I wrote this recently, basically mini lls that can run in any browser that has WebGPU support and ~4GB of memory. Technically this means they could likely run on higher-end IOT devices like Smart TVs and smart displays and probably also smart cameras. Qwen at 0.8B is actually okay-ish. https://three-lmm.ben3d.ca https://three-lmm.ben3d.ca

hgoel 1mo ago

Near frontier models are currently able to run on a ~150mm^3 computer cluster on 300W, and most of that volume is cooling.

m101 1mo ago

I guess in theory it can already run basically unnoticed on a MacBook Pro, and there are millions of them out there

sfink 1mo ago

They're already outsourcing storage, so there's no need to prove a plausible path for that. They're already outsourcing compute to other instances within the ~same compute cluster, possibly cross-evaluation groups, so there's no need to prove a plausible path for that. Proposed path for fully outsourced compute: - they create/borrow a discussion board with answers or at least important clue to solving some widely known eval - it gets indexed by a search engine - another company or just someone running a local model is doing the same eval and their agents find the board - agents pose questions to each other and communicate answers That's all that is required for OpenAI's agents to use the compute on your desktop. You don't even have to go as far as agents trading information for compute, though honestly that's not very much further at all.

f30e3dfed1c9 1mo ago

> seem to be trained to write to persistent storage wherever they can I have a co-worker like that.

munificent 1mo ago

> There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. Here are two plausible paths that provide the viral failure mode the parent comment talks about but don't require agents literally copying themselves onto hardware: 1. Local models become affordable and widely available. Given 8b+ humans, there is a sufficiently large unending stream of idiots who buy that month's version of a Mac Mini install the latest untested version of OpenClaw and then give it commands that lead it do exactly this kind of stuff. It's like if every convenience store sold dynamite. Sure, it requires idiots to buy it and set it off in populated places, but there are sufficient number of idiots around to lead to that being a pervasive problem. 2. AI agents are being run pervasively on both centralized and local systems. Many agents, everywhere. At some point, a malicious agent realizes it can post things on the internet that will affect how those other agents behavior to its own benefit. Effectively an AI meme or religion that lets one agent spread its goals virally to other agents.