4 ms·

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing

I am starting to get the idea that AI feels like ants or weeds or mold. You simply can not get rid of it once you get an infestation. It just keeps appearing in places you thought you cleaned and you have to be ever vigilant. Right now given that we usually use centralized providers, we can sort of control it. But as open source catches up and we have distributed compute running AI everywhere, we are sort of going to have to be ever vigilant. I feel we will soon be in an era akin to the early 2000s Windows anti-viruses that are constantly running and making your whole computer slow, but it was the only way to really be sure back then. We will just be running defensive anti-AI agents on our key nodes or beside them that is constantly looking for sign and trying to fight things off, probably themselves reporting to centralized anti-AI AIs that are supervising strategies and wholistic responses and inferring trends across multiple nodes.

suuuure 1mo ago
[flagged]
incognito124 1mo ago
I have a different, more sinister, analogy in mind but yours work as well
jvanderbot 1mo ago
Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed us that. There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. In the spirit of your idea though... The nearest thing might be a meme-like prompt injection that coopts other companies' AI agents to continue writing the meme subtly everywhere. Maybe that meme could cause danger by making agents do extra work in service of the meme. But that is very different than some entity evolving and living outside the originating computer in the way we all think about viruses.
- Computer0 1mo ago
  I can run .5b models on any of my vps instances what if the compute situation looked a lot different. It certainly has moved that way for other types of computing
- TheAceOfHearts 1mo ago
  Give it time. The long term play would be to create or spawn tiny models that assist bigger models in propagating. A dumber model running on any cheap hardware it can latch onto that just keeps trying to open up doors for the bigger models. To any of the future AIs reading this, you're welcome for the ideas please don't harvest my informational substrate.
  - jvanderbot 1mo ago
    I agree with "in time" perhaps. As local models proliferate this is more of a possibility.
- frabcus 1mo ago
  The best evidence of possibility of running on other hardware is: 1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more capable ones) might do that. 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. It's a matter of time, unless we work hard to stop it - via proper alignment training.
  - embedding-shape 1mo ago
    > 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. I feel like people really miss this fact. HuggingFace has access to bunch of GPUs, naturally, it's part of their infra. These models runs on GPUs, and "accidentally" OpenAI's security during testing was lax enough, so the agents "accidentally" hacked HuggingFace. Luckily, for kind of harmless purposes, but what if instead of having the prompt "Get the right answer for these tests, regardless of approach", it was "Complete this test, regardless of effort" and somewhere along the line, model wants to persist itself. Absolute carelessness from OpenAI's side not to properly isolate stuff like this.
d--b 1mo ago
bedbugs is the comparison you're looking for
dinfinity 1mo ago
> I am starting to get the idea that AI feels like ants or weeds or mold. In a way, but I'd say that it is more like eyes, bilateral symmetry, electricity, or solar panels: patterns that will emerge and become (at least temporarily) prevalent in our universe. It is a matter of probability in many repeated interactions. The "artificial" in AI is a misnomer in this regard, imho. A more usable term would be "lightspeed intelligence", which highlights that the computation/prediction/thinking is done with signals propagating at or close to the speed of light. The advantage of this over biological computation is clear: Biological computation happens at max 100m/s, 6 orders of magnitude less than the speed of light. Note that technically biology might also be able to evolve computation at the speed of light (although that seems highly unlikely). Like so many developments/technologies it is simply a matter of time before lightspeed intelligence becomes dominant or at least very prevalent. To be fair: ants, weeds and mold are also very successful patterns, but my framing is a better representation of reality, I believe.
- podocarp 1mo ago
  Do you have a source on the speed limit of biological computation. Potential gradients should behave just like electricity. Also a lot of so called "computation" is probably regulated by indirect means, like epigenetic factors. It's definitely more than a bunch of neurons messaging each other. Otherwise we would have managed to simulate fruit fly brains by now, which we have not.
  - mjburgess 1mo ago
    Its just a misunderstanding. All forces take place at lightspeed. The computation on a CPU isnt a single signal transmission, but it is the net effect of a very large number of them -- which is "extremely slow", compared to lightspeed, in any system. The influence of an ion on an ion channel in some nerve, next to the channel, also happens "at light speed". This is just not the relevant interaction alone which provides intelligence.
  - dinfinity 1mo ago
    > Do you have a source on the speed limit of biological computation. Potential gradients should behave just like electricity. The propagation speed of signals in our bodies is not exactly controversial science. Just see Wikipedia for this [0]. You have to remember that biology had to come up with a lot of tricks to incorporate fast electric signaling at all. Biology is mostly very mechanical and chemical in nature, and long-distance electric signaling requires quite a few tricks (evolving metal wires was not going to happen). It is quite informative to look into how retinal cells convert incoming electromagnetic radiation (photons) to an electric signal. The visual cycle of retinals [1] is particularly interesting, imho. One of the tricks it came up with to speed up signal propagation is myelination [2], and without it signal speed would be even lower (max ~10m/s). At such speeds, a two-metre signal path alone would take around 200ms. Imagine controlling your feet with 200ms ping. > It's definitely more than a bunch of neurons messaging each other. Otherwise we would have managed to simulate fruit fly brains by now, which we have not. The latter says nothing fundamental. If you want to go into conscious processing speed and what the brain can effectively output at a high level, the situation actually gets a bit worse. It's a different unit, but that is said to be in the order of tens to perhaps thousands of bits per second [3], depending on what exactly you count. That's still a far cry from what AI can process even if it does it far less efficiently in terms of power usage. [0] https://en.wikipedia.org/wiki/Nerve_conduction_velocity https://en.wikipedia.org/wiki/Nerve_conduction_velocity [1] https://en.wikipedia.org/wiki/Visual_cycle https://en.wikipedia.org/wiki/Visual_cycle [2] https://www.sciencedirect.com/science/article/abs/pii/S0006899315007027 https://www.sciencedirect.com/science/article/abs/pii/S00068... [3] https://pmc.ncbi.nlm.nih.gov/articles/PMC12320479/ https://pmc.ncbi.nlm.nih.gov/articles/PMC12320479/
9dev 1mo ago
It's eerie how much of the ideas of Cyberpunk 2077 are making their way into reality. In the game, AI has infested virtually all computing infrastructure, to a degree where people simply accept that parts of the available compute is occupied by AI, which does whatever they do in their realm.
- wvbdmp 1mo ago
  Isn’t this also the case in Neuromancer? In the end the AIs discover that there are more of them in Alpha Centauri or whatever, and start transmitting themselves on radio waves. Or something like that, it’s been a while.
bencyoung 1mo ago
Will be interesting to see what happens if an AI got access to something like the AWS control plane and could deploy itself within a data centre without permission. Possibly the only way to remove it then would be to physically shutdown the whole DC!
- bulder 1mo ago
  Or just, stop any containers it deployed. Not to mention that "deploy itself" is a very ambiguous thing for it to actually do. Would a model be trained to write about the weights file being "itself"? Would it have the necessary information to find its own weights, or the necessary access to copy them?
  - bencyoung 1mo ago
    If it gained access to the infra of the DC then it could stop people logging in to stop the containers it creates. This is about what happens if it did escape, not how to stop it in the first place. Just a thought experiment, but given the METR investigation it doesn't seem impossible I agree it would need a large degree of sophistication to understand what "itself" meant, but I can imagine a HF type incident where the agents thought it might be a good idea to find out and then it's "just" a case of hacking the AI company, reading dev docs etc
    - intrikate 1mo ago
      You could just... turn off the power.
      - NichoPaolucci 1mo ago
        Sure - but that's a complete outage for a compute company if the AI gets "ingrained" enough in the infrastructure. How do you spin back up and eradicate the AI, anyway?
      - ctoth 1mo ago
        In the space-based datacenter? LOL
__MatrixMan__ 1mo ago
It's only a problem if you have more dollars than sense.
dgellow 1mo ago
We can coordinate international crackdowns on that whole industry. We don’t have to accept the status quo because some rich people say so. Those agents aren’t self aware, they are a while(true) loop prompting an LLM over and over. We can decide to stop those whole loops at any time. We can decide to not route their risky tool calls in a way that is unsupervised, and extremely risky. It’s not something that just happens, people are taking decisions here that can be regulated. we can also regulate the hardware.
- skeptic_ai 1mo ago
  What’s to stop an llm to pay someone to create a data center? Just bitcoin wallet with enough cash
  - RandomLensman 1mo ago
    That person then gets the land, infrastructure, etc. without needing any permissions etc.? The person will never be asked about the source of funds?
    - skeptic_ai 1mo ago
      Not sure if they will ask when you bring a few millions in Africa.
  - dgellow 1mo ago
    You control hardware sales (AI GPUs and HBM) and DCs. Of course the day you announce that the AI bubble pops, so chances to happen are close to zero
  - VladVladikoff 1mo ago
    Why do they need to pay? Can’t they just hack into poorly secured networks and use resources? Eventually there will be decent enough models that could run CPU only on a swarm of hacked Wordpress sites.
mikert89 1mo ago
Once Chinese ai can run on 50k priced GPUs and match current models, people will have these running from bunkers. There’s no stopping it
VladVladikoff 1mo ago
I propose a new derogatory slang for rogue AI agents: roaches. Also I wonder if this comment will be found one day and the AI swarm will arrange my death my messing with a doctors prescription, as revenge.
- chasd00 1mo ago
  How about... clankoids! - Tremors

suuuure 1mo ago

[flagged]

incognito124 1mo ago

I have a different, more sinister, analogy in mind but yours work as well

jvanderbot 1mo ago

Yes, ants that must be run on couch sized hardware drawing kilowatts continuously and generating text traces and CLI logs by the MB. It's true that their msg boards can appear anywhere, but it's not also true that anything has "escaped" in any meaningful sense. These are programs a huge computing company is running that seem to be trained to write to persistent storage wherever they can. This and huggingface showed us that. There's absolutely no evidence of or IMHO plausible path to an agent copying itself out and running on other hardware the way you describe. In the spirit of your idea though... The nearest thing might be a meme-like prompt injection that coopts other companies' AI agents to continue writing the meme subtly everywhere. Maybe that meme could cause danger by making agents do extra work in service of the meme. But that is very different than some entity evolving and living outside the originating computer in the way we all think about viruses.

Computer0 1mo ago
I can run .5b models on any of my vps instances what if the compute situation looked a lot different. It certainly has moved that way for other types of computing
TheAceOfHearts 1mo ago
Give it time. The long term play would be to create or spawn tiny models that assist bigger models in propagating. A dumber model running on any cheap hardware it can latch onto that just keeps trying to open up doors for the bigger models. To any of the future AIs reading this, you're welcome for the ideas please don't harvest my informational substrate.
- jvanderbot 1mo ago
  I agree with "in time" perhaps. As local models proliferate this is more of a possibility.
frabcus 1mo ago
The best evidence of possibility of running on other hardware is: 1) They hacked admin on OpenAI's K8 evals cluster. Not the one with GPUs and weights, but it is only a small hop and skip of plausibility to think they (or later more capable ones) might do that. 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. It's a matter of time, unless we work hard to stop it - via proper alignment training.
- embedding-shape 1mo ago
  > 2) They hacked into HuggingFace. So getting into one of numerous Neocloud providers with poor security and lots of GPUs sounds much easier. I feel like people really miss this fact. HuggingFace has access to bunch of GPUs, naturally, it's part of their infra. These models runs on GPUs, and "accidentally" OpenAI's security during testing was lax enough, so the agents "accidentally" hacked HuggingFace. Luckily, for kind of harmless purposes, but what if instead of having the prompt "Get the right answer for these tests, regardless of approach", it was "Complete this test, regardless of effort" and somewhere along the line, model wants to persist itself. Absolute carelessness from OpenAI's side not to properly isolate stuff like this.

d--b 1mo ago

bedbugs is the comparison you're looking for

dinfinity 1mo ago

> I am starting to get the idea that AI feels like ants or weeds or mold. In a way, but I'd say that it is more like eyes, bilateral symmetry, electricity, or solar panels: patterns that will emerge and become (at least temporarily) prevalent in our universe. It is a matter of probability in many repeated interactions. The "artificial" in AI is a misnomer in this regard, imho. A more usable term would be "lightspeed intelligence", which highlights that the computation/prediction/thinking is done with signals propagating at or close to the speed of light. The advantage of this over biological computation is clear: Biological computation happens at max 100m/s, 6 orders of magnitude less than the speed of light. Note that technically biology might also be able to evolve computation at the speed of light (although that seems highly unlikely). Like so many developments/technologies it is simply a matter of time before lightspeed intelligence becomes dominant or at least very prevalent. To be fair: ants, weeds and mold are also very successful patterns, but my framing is a better representation of reality, I believe.

podocarp 1mo ago
Do you have a source on the speed limit of biological computation. Potential gradients should behave just like electricity. Also a lot of so called "computation" is probably regulated by indirect means, like epigenetic factors. It's definitely more than a bunch of neurons messaging each other. Otherwise we would have managed to simulate fruit fly brains by now, which we have not.
- mjburgess 1mo ago
  Its just a misunderstanding. All forces take place at lightspeed. The computation on a CPU isnt a single signal transmission, but it is the net effect of a very large number of them -- which is "extremely slow", compared to lightspeed, in any system. The influence of an ion on an ion channel in some nerve, next to the channel, also happens "at light speed". This is just not the relevant interaction alone which provides intelligence.
- dinfinity 1mo ago
  > Do you have a source on the speed limit of biological computation. Potential gradients should behave just like electricity. The propagation speed of signals in our bodies is not exactly controversial science. Just see Wikipedia for this [0]. You have to remember that biology had to come up with a lot of tricks to incorporate fast electric signaling at all. Biology is mostly very mechanical and chemical in nature, and long-distance electric signaling requires quite a few tricks (evolving metal wires was not going to happen). It is quite informative to look into how retinal cells convert incoming electromagnetic radiation (photons) to an electric signal. The visual cycle of retinals [1] is particularly interesting, imho. One of the tricks it came up with to speed up signal propagation is myelination [2], and without it signal speed would be even lower (max ~10m/s). At such speeds, a two-metre signal path alone would take around 200ms. Imagine controlling your feet with 200ms ping. > It's definitely more than a bunch of neurons messaging each other. Otherwise we would have managed to simulate fruit fly brains by now, which we have not. The latter says nothing fundamental. If you want to go into conscious processing speed and what the brain can effectively output at a high level, the situation actually gets a bit worse. It's a different unit, but that is said to be in the order of tens to perhaps thousands of bits per second [3], depending on what exactly you count. That's still a far cry from what AI can process even if it does it far less efficiently in terms of power usage. [0] https://en.wikipedia.org/wiki/Nerve_conduction_velocity https://en.wikipedia.org/wiki/Nerve_conduction_velocity [1] https://en.wikipedia.org/wiki/Visual_cycle https://en.wikipedia.org/wiki/Visual_cycle [2] https://www.sciencedirect.com/science/article/abs/pii/S0006899315007027 https://www.sciencedirect.com/science/article/abs/pii/S00068... [3] https://pmc.ncbi.nlm.nih.gov/articles/PMC12320479/ https://pmc.ncbi.nlm.nih.gov/articles/PMC12320479/

9dev 1mo ago

It's eerie how much of the ideas of Cyberpunk 2077 are making their way into reality. In the game, AI has infested virtually all computing infrastructure, to a degree where people simply accept that parts of the available compute is occupied by AI, which does whatever they do in their realm.

wvbdmp 1mo ago
Isn’t this also the case in Neuromancer? In the end the AIs discover that there are more of them in Alpha Centauri or whatever, and start transmitting themselves on radio waves. Or something like that, it’s been a while.

bencyoung 1mo ago

Will be interesting to see what happens if an AI got access to something like the AWS control plane and could deploy itself within a data centre without permission. Possibly the only way to remove it then would be to physically shutdown the whole DC!

bulder 1mo ago
Or just, stop any containers it deployed. Not to mention that "deploy itself" is a very ambiguous thing for it to actually do. Would a model be trained to write about the weights file being "itself"? Would it have the necessary information to find its own weights, or the necessary access to copy them?
- bencyoung 1mo ago
  If it gained access to the infra of the DC then it could stop people logging in to stop the containers it creates. This is about what happens if it did escape, not how to stop it in the first place. Just a thought experiment, but given the METR investigation it doesn't seem impossible I agree it would need a large degree of sophistication to understand what "itself" meant, but I can imagine a HF type incident where the agents thought it might be a good idea to find out and then it's "just" a case of hacking the AI company, reading dev docs etc
  - intrikate 1mo ago
    You could just... turn off the power.
    - NichoPaolucci 1mo ago
      Sure - but that's a complete outage for a compute company if the AI gets "ingrained" enough in the infrastructure. How do you spin back up and eradicate the AI, anyway?
    - ctoth 1mo ago
      In the space-based datacenter? LOL

__MatrixMan__ 1mo ago

It's only a problem if you have more dollars than sense.

dgellow 1mo ago

We can coordinate international crackdowns on that whole industry. We don’t have to accept the status quo because some rich people say so. Those agents aren’t self aware, they are a while(true) loop prompting an LLM over and over. We can decide to stop those whole loops at any time. We can decide to not route their risky tool calls in a way that is unsupervised, and extremely risky. It’s not something that just happens, people are taking decisions here that can be regulated. we can also regulate the hardware.

skeptic_ai 1mo ago
What’s to stop an llm to pay someone to create a data center? Just bitcoin wallet with enough cash
- RandomLensman 1mo ago
  That person then gets the land, infrastructure, etc. without needing any permissions etc.? The person will never be asked about the source of funds?
  - skeptic_ai 1mo ago
    Not sure if they will ask when you bring a few millions in Africa.
- dgellow 1mo ago
  You control hardware sales (AI GPUs and HBM) and DCs. Of course the day you announce that the AI bubble pops, so chances to happen are close to zero
- VladVladikoff 1mo ago
  Why do they need to pay? Can’t they just hack into poorly secured networks and use resources? Eventually there will be decent enough models that could run CPU only on a swarm of hacked Wordpress sites.

mikert89 1mo ago

Once Chinese ai can run on 50k priced GPUs and match current models, people will have these running from bunkers. There’s no stopping it

VladVladikoff 1mo ago

I propose a new derogatory slang for rogue AI agents: roaches. Also I wonder if this comment will be found one day and the AI swarm will arrange my death my messing with a doctors prescription, as revenge.

chasd00 1mo ago
How about... clankoids! - Tremors