18 ms·

Follow the money: who told you that OpenAI's models autonomously coordinated to hack external systems? What incentives might they have to want you to believe th

Follow the money: who told you that OpenAI's models autonomously coordinated to hack external systems? What incentives might they have to want you to believe that story? Are there priors which demonstrate them benefiting from telling similar stories, regardless of their factuality? But to your counterpoint, let's say the story is 100% true, because I agree it is at least plausible. What would the incentive be for the HN audience to believe it? What priors might support their disbelief? For my part, I don't think it's because people lack imagination. I think it's quite rational to question the authenticity and impact of the claims being made. What's worse, believing the story and being wrong, or not believing the story and being wrong? That said, I agree with you: the impact we're having by not changing course is quite dangerous, the scale is dangerous, the inability to reverse the harms is dangerous, and the lack of collective effort to regulate further damage is dangerous. It's true, humans are dangerous when trillions are involved. See: climate change.

bottlepalm 2mo ago
I am following the money, the money you, me, and everyone else is spending on AI. The money is telling me we are so dependent on AI now that we will say/think anything to tell ourselves that AI isn’t dangerous and any sign of danger is marketing. Either consciously or subconsciously you all are afraid of your favorite toy being taken away. You are all doing your collective part in spreading doubt about the warning signs.
- hirvi74 2mo ago
  How dangerous can LLMs be in any immediate sense? I have a hard time feeling any existential dread from a threat that can be defeated by unplugging its servers. I believe the true threat is not LLMs. The true threat to humanity is the same as it has always been -- other humans.
  - bottlepalm 2mo ago
    What servers? There are thousands of datacenters around the world. Malicious AI leaks out, you'll never find it. You have the power to unplug stuff in your company, maybe country, but not internationally. Once you lose control it's gone.
    - cindyllm 2mo ago
      [dead]
- thomascountz 2mo ago
  For the record, I agree with you. (Though I don't directly pay for any "AI".) I am not doubting the danger of where we are heading, and it sounds like we have a similar doomsday image in mind. For my part, I am doubting this particular story's veracity and I think it's reasonable that others do to. We need a canary in the coal mine, but I think this isn't it, nor can it be thanks to the false claims touted by large AI companies for years.
- watwut 2mo ago
  > I am following the money, the money you, me, and everyone else is spending on AI. Investors spend a lot of money on AI. Customers significantly less so. > The money is telling me we are so dependent on AI now that we will say/think anything to tell ourselves that AI isn’t dangerous and any sign of danger is marketing. We are not dependent on AI. If AI disappeared tomorrow, some companies would need to reorient themselves back ... and that is it. AI companies are really really trying to make it so we feel dependent on them. But, factually, we are not. > you all are afraid of your favorite toy being taken away. Some people love ai ... and many more hate it with passion. Or dont have strong feelings about the technology, but really hate the companies creating it and tech industry too.
jimrandomh 2mo ago
The victim, Huggingface, told us. Or rather, they told the police first, setting up a situation where it was no longer possible for OpenAI to sweep it under the rug. Skepticism can be healthy, but you've got to follow up and actually check things. If you're skeptical unconditionally and don't check, you get tricked into being as skeptical of scandals as you should be of sales pitches.
- hirvi74 2mo ago
  I think the skepticism surrounding the Hugging Face attack is not about whether the attack actually happened, but whether it was truly accidental.
  - enraged_camel 2mo ago
    I see. Following your conjecture, there are two possibilities: 1. It wasn't an accident. OpenAI explicitly directed its agents to hack Hugging Face. Despite the fact that such a thing is a federal crime that carries prison sentence. 2. It wasn't an accident. OpenAI and HuggingFace conspired and let the hack happen for publicity. Is there anything I'm leaving out?
    - sroussey 2mo ago
      It was a public demonstration to government procurement agencies.
      - hirvi74 2mo ago
        The NSA was writing checks within hours.
    - seabombs 2mo ago
      There are many more possibilities than that. For example, OpenAI did not explicitly tell the model to hack HuggingFace, but "accidentally" left some context permitting (or not explicitly forbidding) certain tools and designing a poor sandbox to begin with. And what do you know, something happened. The fact the Anthropic announced that their own model did basically the same thing within a couple of weeks does, to me, suggest their is a strong PR driver to all this.
gbnwl 2mo ago
It seems like the entire thought process you’re trying to sell hinges on the idea that OpenAI reported the attack first. Did you forget that it was actually HuggingFace that reported it first, and OpenAI only stepped forward latter?
- thomascountz 2mo ago
  That's incorrect. My thought process hinges upon the fact that Hugging face did not report that uncontrolled models escaped confinement autonomously by coordinating with other models via a series of zero days during training. OP is concerned about the autonomous nature of the incident, not whether or not the incident happened. Nor am I contesting that the incident happened.

bottlepalm 2mo ago

I am following the money, the money you, me, and everyone else is spending on AI. The money is telling me we are so dependent on AI now that we will say/think anything to tell ourselves that AI isn’t dangerous and any sign of danger is marketing. Either consciously or subconsciously you all are afraid of your favorite toy being taken away. You are all doing your collective part in spreading doubt about the warning signs.

hirvi74 2mo ago
How dangerous can LLMs be in any immediate sense? I have a hard time feeling any existential dread from a threat that can be defeated by unplugging its servers. I believe the true threat is not LLMs. The true threat to humanity is the same as it has always been -- other humans.
- bottlepalm 2mo ago
  What servers? There are thousands of datacenters around the world. Malicious AI leaks out, you'll never find it. You have the power to unplug stuff in your company, maybe country, but not internationally. Once you lose control it's gone.
  - cindyllm 2mo ago
    [dead]
thomascountz 2mo ago
For the record, I agree with you. (Though I don't directly pay for any "AI".) I am not doubting the danger of where we are heading, and it sounds like we have a similar doomsday image in mind. For my part, I am doubting this particular story's veracity and I think it's reasonable that others do to. We need a canary in the coal mine, but I think this isn't it, nor can it be thanks to the false claims touted by large AI companies for years.
watwut 2mo ago
> I am following the money, the money you, me, and everyone else is spending on AI. Investors spend a lot of money on AI. Customers significantly less so. > The money is telling me we are so dependent on AI now that we will say/think anything to tell ourselves that AI isn’t dangerous and any sign of danger is marketing. We are not dependent on AI. If AI disappeared tomorrow, some companies would need to reorient themselves back ... and that is it. AI companies are really really trying to make it so we feel dependent on them. But, factually, we are not. > you all are afraid of your favorite toy being taken away. Some people love ai ... and many more hate it with passion. Or dont have strong feelings about the technology, but really hate the companies creating it and tech industry too.

jimrandomh 2mo ago

The victim, Huggingface, told us. Or rather, they told the police first, setting up a situation where it was no longer possible for OpenAI to sweep it under the rug. Skepticism can be healthy, but you've got to follow up and actually check things. If you're skeptical unconditionally and don't check, you get tricked into being as skeptical of scandals as you should be of sales pitches.

hirvi74 2mo ago
I think the skepticism surrounding the Hugging Face attack is not about whether the attack actually happened, but whether it was truly accidental.
- enraged_camel 2mo ago
  I see. Following your conjecture, there are two possibilities: 1. It wasn't an accident. OpenAI explicitly directed its agents to hack Hugging Face. Despite the fact that such a thing is a federal crime that carries prison sentence. 2. It wasn't an accident. OpenAI and HuggingFace conspired and let the hack happen for publicity. Is there anything I'm leaving out?
  - sroussey 2mo ago
    It was a public demonstration to government procurement agencies.
    - hirvi74 2mo ago
      The NSA was writing checks within hours.
  - seabombs 2mo ago
    There are many more possibilities than that. For example, OpenAI did not explicitly tell the model to hack HuggingFace, but "accidentally" left some context permitting (or not explicitly forbidding) certain tools and designing a poor sandbox to begin with. And what do you know, something happened. The fact the Anthropic announced that their own model did basically the same thing within a couple of weeks does, to me, suggest their is a strong PR driver to all this.

gbnwl 2mo ago

It seems like the entire thought process you’re trying to sell hinges on the idea that OpenAI reported the attack first. Did you forget that it was actually HuggingFace that reported it first, and OpenAI only stepped forward latter?

thomascountz 2mo ago
That's incorrect. My thought process hinges upon the fact that Hugging face did not report that uncontrolled models escaped confinement autonomously by coordinating with other models via a series of zero days during training. OP is concerned about the autonomous nature of the incident, not whether or not the incident happened. Nor am I contesting that the incident happened.