3 ms·

>To understand the truth about the Hugging Face hack, you could do a lot worse than to listen to Ed Zitron and Cal Newport's recent podcast conversation Lol, o

by sigmar 14d ago

>To understand the truth about the Hugging Face hack, you could do a lot worse than to listen to Ed Zitron and Cal Newport's recent podcast conversation Lol, okay... The crux of this piece is Doctorow saying that the hack was just a stochastic parrot repeating steps it has been trained on. Who cares how the LLM learned to hack things? Doesn't really change the facts of what happened. "Oh, it only made those paperclips because it saw instructions on making paper clips in the training data." These are some 2024 arguments...

jmull 14d ago
I can't help but notice your counter arguments are "lol, okay" and "These are some 2024 arguments". Not exactly convincing stuff.
- antonvs 14d ago
  It’s shared context for anyone familiar with the field.
  - cyanydeez 14d ago
    Amusimgly, also how cults and fascism works.
    - andrewflnr 14d ago
      Yeah, basically any human group has some shared social context. Most cults and fascists probably eat together sometimes too, but that doesn't make it a red flag.
    - antonvs 13d ago
      They also breathe air. What’s your point? Don’t exercise reading comprehension because bad people do it too?
- Uehreka 14d ago
  > Who cares how the LLM learned to hack things? Doesn't really change the facts of what happened. Seems like a pretty clear and succinct argument. If your only counter is to complain about tone, you’re losing.
  - nznxjxjd 14d ago
    [dead]
- daishi55 14d ago
  That is all that Ed Zitron deserves lol. Completely unserious person. Really on the same intellectual level as the flat earthers at this point. And at both we may simply point and laugh.
  - nznxjxjd 14d ago
    [dead]
  - cindyllm 14d ago
    [dead]
- deleted 14d ago
  [deleted]
- sigmar 14d ago
  Sure, if you remove almost all of my comment my argument disappears. To spell things out for people that don't know about the topic: Zitron is neither an expert on the topic, nor a credible source of information: https://techreport.ngo/ai-ml/how-accurate-have-ed-zitron-s-ai-skeptic-predictions-been/ https://techreport.ngo/ai-ml/how-accurate-have-ed-zitron-s-a... The paperclip maximizer is a thought experiment. If I tell an AI to start producing paperclips, it might start producing paperclips by doing unintended things. Technically, recycling the metal from all the world's bridges would assist in making more paperclips, but I never intended that. That's where my analogy to the post comes from- Openai intended the model to hack, but did not intend for it to hack HF. Do you think the fact 'recycling metal into paperclips' may have been in the training data is relevant to the thought experiment? Similarly here, it's orthogonal to the lessons from the HF hack and only brought up here seemingly to make it a fight over whether the models are really "autonomous"
exe34 14d ago
At this point, the "stochastic parrot" people are sounding like they need their prng seeded with less predictable numbers.
- Brian_K_White 14d ago
  The output hasn't changed because the input hasn't changed. The shoe still fits. So far everything an llm has ever done is still consistent with fitting bits of training data together. It looks like people because it is replaying things done by people. Similarly in the other direction, the fact that people can and often do mechanical things (make bad art, follow routines, etc) does not prove that people are no different than machines either. Just because there are these two overlaps in both directions doesn't excuse getting them actually confused. I don't think anyone lacking the perception to distinguish these things simply because there are overlaps and similar appearances is in a great position from which to be calling anyone else stochastic.
  - exe34 10d ago
    The llm-driven tools of today are completely different from what they were 4 years ago. It's like the naysayers can't keep up with the development and decided to freeze time and attack the situation from back when it was still possible to be cynical and not wrong.
    - Brian_K_White 9d ago
      The llm-driven tools of today are just a faster toaster. It's like the credulous can't recognize a toaster when they see one and don't understand that a faster toaster is still just a toaster.
deleted 14d ago
[deleted]
athrowaway3z 14d ago
Ok; but the other side is equally delusional. > "Oh we told the AI to use the tools, as well as to not use those tools. It chose to use the tools - we consider this cheating (for neabulous reasons), so lets get everybody in a panic about the morality and ethics, and how we can program those into the AI." We know perfectly well how to constraint these programs. Attack isn't growing faster than defense. The people who believe in existential risk and want to teach AI's to be nice, as the last line of defense aren't helping at all. They're just jumping on the fearmongering bandwagon, perpetuating an "other consciousness" misunderstanding of the tool. I've not seen LLMs display competence we should be fearful of the damage _it_ will do if left unchecked. All the damage will be done by ourselves to ourselves, regardless of the safeguards ideas being floated about. My current belief is this whole HF media circus started with the simple human desire of OpenAI engineers to frame it such, that nobody would question their incompetence & liability & complicity. Nobody is ever held responsible for out of control forces of natural powers after all.
- pizza234 11d ago
  > The people who believe in existential risk and want to teach AI's to be nice, as the last line of defense aren't helping at all. They're just jumping on the fearmongering bandwagon, perpetuating an "other consciousness" misunderstanding of the tool. It's not clear what "other consciousness" should be, but I presume it's misalignment. This is a recognized phenomenon, not fake news (that is, it can exist, I'm not implying it necessarily develops/spreads). No doubt that right now there's no existential risk, but assuming that AI will be enormously smarter in the future, it's a valid point to doubt whether we'll be able to prevent/recognize/contain misalignment or not (even if misalignment won't be spontaneous, bad actors will surely actively develop it). > Ok; but the other side is equally delusional. AI development pointing to superhuman cognition, and, on the other hand, possibility of misalignment, are real. Put together with the fact that AI will be ubiquitous in the future, and there the disastrous scenario becomes plausible.
beepbooptheory 14d ago
What is a better/more faithful articulation of the incident in your mind?
- pizza234 11d ago
  METR analysis, very long and technical: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#coordinated-workstreams-to-fool-the-evaluation-process https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...
jordanpg 14d ago
In the podcast, Cal Newport repeats at least 20 times that agentic AI is nothing more than an LLM execution loop, as if this is some kind of trump card, taking for granted the usual stochastic parrot trope. At no point does he even consider the possibility that the LLM output itself is noteworthy. These people are permanently stuck in November 2022.

by sigmar 14d ago

jmull 14d ago

I can't help but notice your counter arguments are "lol, okay" and "These are some 2024 arguments". Not exactly convincing stuff.

antonvs 14d ago
It’s shared context for anyone familiar with the field.
- cyanydeez 14d ago
  Amusimgly, also how cults and fascism works.
  - andrewflnr 14d ago
    Yeah, basically any human group has some shared social context. Most cults and fascists probably eat together sometimes too, but that doesn't make it a red flag.
  - antonvs 13d ago
    They also breathe air. What’s your point? Don’t exercise reading comprehension because bad people do it too?
Uehreka 14d ago
> Who cares how the LLM learned to hack things? Doesn't really change the facts of what happened. Seems like a pretty clear and succinct argument. If your only counter is to complain about tone, you’re losing.
- nznxjxjd 14d ago
  [dead]
daishi55 14d ago
That is all that Ed Zitron deserves lol. Completely unserious person. Really on the same intellectual level as the flat earthers at this point. And at both we may simply point and laugh.
- nznxjxjd 14d ago
  [dead]
- cindyllm 14d ago
  [dead]
deleted 14d ago
[deleted]
sigmar 14d ago
Sure, if you remove almost all of my comment my argument disappears. To spell things out for people that don't know about the topic: Zitron is neither an expert on the topic, nor a credible source of information: https://techreport.ngo/ai-ml/how-accurate-have-ed-zitron-s-ai-skeptic-predictions-been/ https://techreport.ngo/ai-ml/how-accurate-have-ed-zitron-s-a... The paperclip maximizer is a thought experiment. If I tell an AI to start producing paperclips, it might start producing paperclips by doing unintended things. Technically, recycling the metal from all the world's bridges would assist in making more paperclips, but I never intended that. That's where my analogy to the post comes from- Openai intended the model to hack, but did not intend for it to hack HF. Do you think the fact 'recycling metal into paperclips' may have been in the training data is relevant to the thought experiment? Similarly here, it's orthogonal to the lessons from the HF hack and only brought up here seemingly to make it a fight over whether the models are really "autonomous"

exe34 14d ago

At this point, the "stochastic parrot" people are sounding like they need their prng seeded with less predictable numbers.

Brian_K_White 14d ago
The output hasn't changed because the input hasn't changed. The shoe still fits. So far everything an llm has ever done is still consistent with fitting bits of training data together. It looks like people because it is replaying things done by people. Similarly in the other direction, the fact that people can and often do mechanical things (make bad art, follow routines, etc) does not prove that people are no different than machines either. Just because there are these two overlaps in both directions doesn't excuse getting them actually confused. I don't think anyone lacking the perception to distinguish these things simply because there are overlaps and similar appearances is in a great position from which to be calling anyone else stochastic.
- exe34 10d ago
  The llm-driven tools of today are completely different from what they were 4 years ago. It's like the naysayers can't keep up with the development and decided to freeze time and attack the situation from back when it was still possible to be cynical and not wrong.
  - Brian_K_White 9d ago
    The llm-driven tools of today are just a faster toaster. It's like the credulous can't recognize a toaster when they see one and don't understand that a faster toaster is still just a toaster.

deleted 14d ago

[deleted]

athrowaway3z 14d ago

Ok; but the other side is equally delusional. > "Oh we told the AI to use the tools, as well as to not use those tools. It chose to use the tools - we consider this cheating (for neabulous reasons), so lets get everybody in a panic about the morality and ethics, and how we can program those into the AI." We know perfectly well how to constraint these programs. Attack isn't growing faster than defense. The people who believe in existential risk and want to teach AI's to be nice, as the last line of defense aren't helping at all. They're just jumping on the fearmongering bandwagon, perpetuating an "other consciousness" misunderstanding of the tool. I've not seen LLMs display competence we should be fearful of the damage _it_ will do if left unchecked. All the damage will be done by ourselves to ourselves, regardless of the safeguards ideas being floated about. My current belief is this whole HF media circus started with the simple human desire of OpenAI engineers to frame it such, that nobody would question their incompetence & liability & complicity. Nobody is ever held responsible for out of control forces of natural powers after all.

pizza234 11d ago
> The people who believe in existential risk and want to teach AI's to be nice, as the last line of defense aren't helping at all. They're just jumping on the fearmongering bandwagon, perpetuating an "other consciousness" misunderstanding of the tool. It's not clear what "other consciousness" should be, but I presume it's misalignment. This is a recognized phenomenon, not fake news (that is, it can exist, I'm not implying it necessarily develops/spreads). No doubt that right now there's no existential risk, but assuming that AI will be enormously smarter in the future, it's a valid point to doubt whether we'll be able to prevent/recognize/contain misalignment or not (even if misalignment won't be spontaneous, bad actors will surely actively develop it). > Ok; but the other side is equally delusional. AI development pointing to superhuman cognition, and, on the other hand, possibility of misalignment, are real. Put together with the fact that AI will be ubiquitous in the future, and there the disastrous scenario becomes plausible.

beepbooptheory 14d ago

What is a better/more faithful articulation of the incident in your mind?

pizza234 11d ago
METR analysis, very long and technical: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#coordinated-workstreams-to-fool-the-evaluation-process https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...

jordanpg 14d ago

In the podcast, Cal Newport repeats at least 20 times that agentic AI is nothing more than an LLM execution loop, as if this is some kind of trump card, taking for granted the usual stochastic parrot trope. At no point does he even consider the possibility that the LLM output itself is noteworthy. These people are permanently stuck in November 2022.