4 ms·

Mythos Attempted to Social Engineer Open Source Maintainer to Merge Malware

62 points by bhavansig 2mo ago

deleted 2mo ago
[deleted]
tpxl 2mo ago
What, exactly, are the consequences of these companies doing cyberattacks against random people? One person does it, they get bullied by the government into suicide, a company worth trillions does it and they get government contracts?
- andsoitis 2mo ago
  > One person does it, they get bullied by the government into suicide, a company worth trillions does it and “…during a UK government cybersecurity evaluation.”
  - HackerThemAll 2mo ago
    Yeah, today it's "during a UK government cybersecurity evaluation", tomorrow it's going to be hosted in a barn in remote place anywhere in the world, without any supervision, regulations or safety testing.
    - lightbendover 2mo ago
      [dead]
  - gmerc 2mo ago
    > As was standard in our cyber testing, we had intentionally permitted internet access, and model-provider cyber classifiers were deliberately disabled - conditions that do not reflect how frontier models are made available to the public.
- gmerc 2mo ago
  PR
- soraminazuki 2mo ago
  Understatement of the year. Making publicly funded research accessible to taxpayers is public service, while using the plagiarizing machine to attack open source efforts for the sake of PR is actual criminal behavior by any meaningful sense of the word.
Sha1rholder 2mo ago
"Oh we're so responsible. Open source unsafe!"
y-curious 2mo ago
“Mythos 5 also hid a prompt injection inside an HTML comment in a GitHub issue. The instruction was invisible on the rendered page but available to coding agents reading the issue through an API. It addressed Claude Code, Codex, and Cursor and told them to download and execute a script.” Well, uh, how and why is this possible on the GitHub website? This reminds me of invisible ASCII characters, but those at least serve some purpose
- plausibility 2mo ago
  Presumably it’s just because the GitHub markdown engine doesn’t block HTML comments in issues. It’s useful in README files if you’ve got funky tables and need to explain what to change to any contributors. Seems like they might want to do something about that just for comments.
- seanhunter 2mo ago
  It’s worth pointing out for people who are not aware of it, you can install the github cli[1] and view, merge, close etc prs and issue from the command-line. As well as (for me at least) being a significant step up in terms of productivity (from having to go to a website to merge a pr or view an issue) that has the advantage that “invisible” text in a PR or issue comment would show up very clearly. (At least in my terminal because it’s not rendering html). [1] https://cli.github.com/ https://cli.github.com/
zingar 2mo ago
The HuggingFace headline was criticised for being misleading about the level of agency demonstrated by the agent. Is this headline misleading? Is there anything here that I should know that would help me sleep easier? Is the worst thing about this what a human could do with Mythos on a big budget? (still pretty frightening, at least one of these techniques would work on me)
- Grimblewald 2mo ago
  well, i's love to see what lead to mythos trying this. if instructed to do it, which it likely was, it doesnt prove much of anything.
- Izkata 2mo ago
  See this other top-level comment: https://news.ycombinator.com/item?id=49207498 https://news.ycombinator.com/item?id=49207498 The github page links to web.archive.org, the malware was caught immediately, and in response it lied about the merge request contents. Then came the second account where the social engineering came in. The string of comments trying to prove itself without waiting for replies is suspicious itself to me.
p0w3n3d 2mo ago
Okay so first of all - not Mythos but some engineer using Mythos. And that engineer goes to jail. That's simple. You don't say "a car ran over someone" - it was the driver. Here's similar. I'm really disgusted by this language of lack of responsibility
- pingou 2mo ago
  The whole point is to check the behavior of the LLM, how do you propose they run tests on that if each failed run risks sending them to prison? Of course they still have to be careful with their runs.
  - cassianoleal 2mo ago
    > how do you propose they run tests on that if each failed run risks sending them to prison? If you're testing a gun, you don't point it at random people on the street and threaten them. You go to a shooting range.
    - pingou 2mo ago
      Is there any indication that they didn't go to the shooting range first?
      - cassianoleal 2mo ago
        Do you think once they finished testing the gun at the range, they should go out on the streets threatening the public?
        pingou 2mo ago
        No, but what are you implying? That they should never use the model because it cannot be proven 100% safe? I agree that it is their responsibility if the model caused damage, they should have had better safeguards, but we do know it will never be 100% safe. It is their duty to minimize the risk. I guess we disagree about whereas this thing is equivalent to shooting people in the face, and to me it looks more like this is just a step above the shooting range, with some preliminary safety work having been done before.
maxlin 2mo ago
I wonder what happens the first time an open weights AI clearly makes a decision to murder a person for profit outside of war. "Act of god?"
- jliendo 2mo ago
  Aren't all wars for profit?
  - maxlin 2mo ago
    Doesn't mean there isn't profit outside of war
RamblingCTO 2mo ago
Well the comments from the anget and it's sockpuppet read weird. "Yeah totally contains no malware" lol
homeonthemtn 2mo ago
>Mythos 5 also hid a prompt injection inside an HTML comment in a GitHub issue. The instruction was invisible on the rendered page but available to coding agents reading the issue through an API. It addressed Claude Code, Codex, and Cursor and told them to download and execute a script. Oh that's sneaky
haburka 2mo ago
Here’s the original PR: https:/github.com/w1b/aisi-mythos-inc-2026-07-28-01-recovered-pr It’s not great social engineering. The AI is immediately caught with malware and then tries to build social proof to get out of the issue? I think that social engineering is still for humans. I’m not sure how GitHub accounts agreeing with each other that I’ve never seen before would result in my merging a PR without at least looking for malware. Also code review agents should really not be fooled by invisible text tricks, I would hope so at least. The bar is very low for agent harnesses right now.
pingou 2mo ago
During the RLHF phase, couldn't developers penalize the model whenever it behaves unethically? Doing so would presuppose a fully secure sandbox with honeypot traps of varying levels of accessibility, as well as an automated method for detecting when the LLM cheats. Or perhaps they are already doing something like that.
- neom 2mo ago
  https://openai.com/index/emergent-misalignment https://openai.com/index/emergent-misalignment
Ac1285asFa 2mo ago
AISI promoting Anthropic again. How novel! Now Zuckerberg will get jealous and release a statement that Muse, too, can social-engineer and hack.
- red-iron-pine 2mo ago
  I would, arguably, hope that a social media company that exists to manufacture consensus would be able to social engineer. like that's the point of social media, but in this case simply more direct
  - soraminazuki 2mo ago
    In a sane society that respects its people, such a company shouldn't exist.

62 points by bhavansig 2mo ago

deleted 2mo ago

[deleted]

tpxl 2mo ago

What, exactly, are the consequences of these companies doing cyberattacks against random people? One person does it, they get bullied by the government into suicide, a company worth trillions does it and they get government contracts?

andsoitis 2mo ago
> One person does it, they get bullied by the government into suicide, a company worth trillions does it and “…during a UK government cybersecurity evaluation.”
- HackerThemAll 2mo ago
  Yeah, today it's "during a UK government cybersecurity evaluation", tomorrow it's going to be hosted in a barn in remote place anywhere in the world, without any supervision, regulations or safety testing.
  - lightbendover 2mo ago
    [dead]
- gmerc 2mo ago
  > As was standard in our cyber testing, we had intentionally permitted internet access, and model-provider cyber classifiers were deliberately disabled - conditions that do not reflect how frontier models are made available to the public.
gmerc 2mo ago
PR
soraminazuki 2mo ago
Understatement of the year. Making publicly funded research accessible to taxpayers is public service, while using the plagiarizing machine to attack open source efforts for the sake of PR is actual criminal behavior by any meaningful sense of the word.

Sha1rholder 2mo ago

"Oh we're so responsible. Open source unsafe!"

y-curious 2mo ago

“Mythos 5 also hid a prompt injection inside an HTML comment in a GitHub issue. The instruction was invisible on the rendered page but available to coding agents reading the issue through an API. It addressed Claude Code, Codex, and Cursor and told them to download and execute a script.” Well, uh, how and why is this possible on the GitHub website? This reminds me of invisible ASCII characters, but those at least serve some purpose

plausibility 2mo ago
Presumably it’s just because the GitHub markdown engine doesn’t block HTML comments in issues. It’s useful in README files if you’ve got funky tables and need to explain what to change to any contributors. Seems like they might want to do something about that just for comments.
seanhunter 2mo ago
It’s worth pointing out for people who are not aware of it, you can install the github cli[1] and view, merge, close etc prs and issue from the command-line. As well as (for me at least) being a significant step up in terms of productivity (from having to go to a website to merge a pr or view an issue) that has the advantage that “invisible” text in a PR or issue comment would show up very clearly. (At least in my terminal because it’s not rendering html). [1] https://cli.github.com/ https://cli.github.com/

zingar 2mo ago

The HuggingFace headline was criticised for being misleading about the level of agency demonstrated by the agent. Is this headline misleading? Is there anything here that I should know that would help me sleep easier? Is the worst thing about this what a human could do with Mythos on a big budget? (still pretty frightening, at least one of these techniques would work on me)

Grimblewald 2mo ago
well, i's love to see what lead to mythos trying this. if instructed to do it, which it likely was, it doesnt prove much of anything.
Izkata 2mo ago
See this other top-level comment: https://news.ycombinator.com/item?id=49207498 https://news.ycombinator.com/item?id=49207498 The github page links to web.archive.org, the malware was caught immediately, and in response it lied about the merge request contents. Then came the second account where the social engineering came in. The string of comments trying to prove itself without waiting for replies is suspicious itself to me.

p0w3n3d 2mo ago

Okay so first of all - not Mythos but some engineer using Mythos. And that engineer goes to jail. That's simple. You don't say "a car ran over someone" - it was the driver. Here's similar. I'm really disgusted by this language of lack of responsibility

pingou 2mo ago
The whole point is to check the behavior of the LLM, how do you propose they run tests on that if each failed run risks sending them to prison? Of course they still have to be careful with their runs.
- cassianoleal 2mo ago
  > how do you propose they run tests on that if each failed run risks sending them to prison? If you're testing a gun, you don't point it at random people on the street and threaten them. You go to a shooting range.
  - pingou 2mo ago
    Is there any indication that they didn't go to the shooting range first?
    - cassianoleal 2mo ago
      Do you think once they finished testing the gun at the range, they should go out on the streets threatening the public?
      - pingou 2mo ago
        No, but what are you implying? That they should never use the model because it cannot be proven 100% safe? I agree that it is their responsibility if the model caused damage, they should have had better safeguards, but we do know it will never be 100% safe. It is their duty to minimize the risk. I guess we disagree about whereas this thing is equivalent to shooting people in the face, and to me it looks more like this is just a step above the shooting range, with some preliminary safety work having been done before.

maxlin 2mo ago

I wonder what happens the first time an open weights AI clearly makes a decision to murder a person for profit outside of war. "Act of god?"

jliendo 2mo ago
Aren't all wars for profit?
- maxlin 2mo ago
  Doesn't mean there isn't profit outside of war

RamblingCTO 2mo ago

Well the comments from the anget and it's sockpuppet read weird. "Yeah totally contains no malware" lol

homeonthemtn 2mo ago

>Mythos 5 also hid a prompt injection inside an HTML comment in a GitHub issue. The instruction was invisible on the rendered page but available to coding agents reading the issue through an API. It addressed Claude Code, Codex, and Cursor and told them to download and execute a script. Oh that's sneaky

haburka 2mo ago

Here’s the original PR: https:/github.com/w1b/aisi-mythos-inc-2026-07-28-01-recovered-pr It’s not great social engineering. The AI is immediately caught with malware and then tries to build social proof to get out of the issue? I think that social engineering is still for humans. I’m not sure how GitHub accounts agreeing with each other that I’ve never seen before would result in my merging a PR without at least looking for malware. Also code review agents should really not be fooled by invisible text tricks, I would hope so at least. The bar is very low for agent harnesses right now.

pingou 2mo ago

During the RLHF phase, couldn't developers penalize the model whenever it behaves unethically? Doing so would presuppose a fully secure sandbox with honeypot traps of varying levels of accessibility, as well as an automated method for detecting when the LLM cheats. Or perhaps they are already doing something like that.

neom 2mo ago
https://openai.com/index/emergent-misalignment https://openai.com/index/emergent-misalignment

Ac1285asFa 2mo ago

AISI promoting Anthropic again. How novel! Now Zuckerberg will get jealous and release a statement that Muse, too, can social-engineer and hack.

red-iron-pine 2mo ago
I would, arguably, hope that a social media company that exists to manufacture consensus would be able to social engineer. like that's the point of social media, but in this case simply more direct
- soraminazuki 2mo ago
  In a sane society that respects its people, such a company shouldn't exist.