3 ms·

The situation is getting crazy ... personally I have already uninstalled node, python and all package managers from my machine and instead only use them in devc

The situation is getting crazy ... personally I have already uninstalled node, python and all package managers from my machine and instead only use them in devcontainers / VMs. But even if the dev community comes up with super hardened security, I fear in at least a year the models will be good enough in social engineering that we are still running a losing game.

wolfi1 5mo ago
how do containers solve the problem? if they are connected to the internet (and they are) you have got the same problem, if the credentials can be read by the container, at least to my understanding
- mentalgear 5mo ago
  For credential stealing, that is true, but at least it would protect your local machine. But I just read these worms also try container escape ...
  - silon42 5mo ago
    We need to prevent direct connections to internet for containers... once you have a proxy, predefined credentials (api keys) can maybe be added there (per container/target).
    - cyanydeez 5mo ago
      the model most people are talking about is in the cloud. for the harness to do useful work, it needs to talk to the cloud the trouble is, we need protocols that are software determined that force AI interaxtions into limited scope but currently theyre all just bash adjacent and inherit your tools.
  - fnoef 5mo ago
    You need to use full isolated VM with its own kernel. But then again, I've read somewhere that this malware is also trying to escape the VM isolation as well...
    - deleted 5mo ago
      [deleted]
- jcgl 5mo ago
  On my personal machine, I run OpenSnitch. Much better defense against data exfil if you reject outbound connections to unexpected/unwanted hosts.
  - pingou 5mo ago
    That wouldn't help in that case as exfiltrated data is committed to public GitHub repositories. Unless you have to accept every time an app posts or requests data from known hosts?
    - gus_ 5mo ago
      Personally I don't allow outbound connections from almost any app, except web browsers to port 80/443. So nodejs, pip, ruby, curl, wget, etc, opening unexpected outbound connections is a big red flag for me. In some cases, maybe you need to allow permanently git to open outbound resquests to github.com (or gitlab, etc), but at least in my case, I'm okey allowing these connections manually. > preinstall script: bun run index.js > Dual exfiltration: > stolen data is committed as Git objects to public GitHub repositories (api.github.com) > and sent as RSA+AES encrypted HTTPS POSTs to hxxps://t.m-kosche[.]com/api/public/otel/v1/traces (disguised as OpenTelemetry traces) > The Bun installer command (command -v bun >/dev/null 2>&1 || (curl -fsSL https://bun.sh/install https://bun.sh/install | bash && export PATH=$HOME/.bun/bin:$PATH)) prepends every injected hook to guarantee Bun availability > A separate gh-token-monitor daemon (decrypted from J7, deployed by class so) installs to ~/.local/bin/gh-token-monitor.sh with its own systemd service and LaunchAgent. It polls stolen GitHub tokens at 60-second intervals with a 24-hour TTL This attack in particular would have caused OpenSnitch to go crazy, giving you the opportunity to review what's going on.
      - jcgl 5mo ago
        > Personally I don't allow outbound connections from almost any app, except web browsers to port 80/443. So nodejs, pip, ruby, curl, wget, etc, opening unexpected outbound connections is a big red flag for me. Yep, exactly. Reject by default, with reasonably judicious always-allow rules.
      - 5mo ago
- TacticalCoder 5mo ago
  Watertight subdivision in a ship doesn't promise: "there'll never ever be water in this ship". It says: "If there's water in this ship due to one hole, it'll stay in one compartment". Note that I said one hole: you have the titanic, many compartment gets holes, that one ship is still going to sink. (btw that the Titanic sunk is not an excuse not to secure other ships. And it did save a great many other ships to have watertight subdivision.) So... Although there are exploits escaping containers and VMs and then bad guys doing lateral moves across machines, you still want defense in depth.
  - sitkack 5mo ago
    One thing could be more clear, is that the Titanic's bulkheads didn't go all the way to the top deck. They did not seal completely and were not watertight.
- CGamesPlay 5mo ago
  Things on my workstation that the container does not have access to: browser cookies, 1Password cli, SSH keys (even if I allow the SSH agent socket), cargo publish tokens (unless it’s a rust project), npm tokens (unless it’s an npm project), and not to mention anything relating to my other clients (don’t compromise my employer when I vibe install some dep for a random side project).
HWR_14 5mo ago
Without node, how do you control your cloud resources? Cloudflare requires wrangler. AWS has a lot of node clis. Etc.
- anygivnthursday 5mo ago
  From CI pipelines with pinned package versions and OIDC instead of permanent secrets limited, least privileges.
- rubiquity 5mo ago
  AWS has CLIs and SDKs in many languages.
zahlman 5mo ago
> the models will be good enough in social engineering Never mind questions of how good the models will/can get. I'm confused why people expect that, in principle, models getting really good at social engineering would have such huge impact. Seems to me like it has diminishing returns and is severely bottlenecked by the fact that the target operates at human speed. The amount of effort involved in the XZ hack, for example, was immense, and it couldn't have been accelerated because it worked specifically by wearing the existing maintainer down over time. You could generate and send all the necessary nastygrams in seconds and it wouldn't speed up the human consuming them (and in fact, having all of them arrive at once would raise suspicion). And there is a limit to how persuasive that input can be. Take any of the random nastygrams directed at the XZ maintainer; maybe they could have been made more nasty, more pointed, more aware of the maintainer's personal weaknesses and fears — but would that have actually been overall more effective? I think not, or at most only a tiny bit.
- Terr_ 5mo ago
  I'm imagining an influence campaign where someone weaponizes targeted advertising and recommendation engines against Mr. Trusted Maintainer, so that it's a The Truman Show effect, where the background of their entire digital bubble pushing the idea that they need retire and delegate.
  - pyinstallwoes 5mo ago
    This definitely happens. Perhaps more on a state actor level of sophistication but yeah. I’ve seen it. It’s wild.

wolfi1 5mo ago

how do containers solve the problem? if they are connected to the internet (and they are) you have got the same problem, if the credentials can be read by the container, at least to my understanding

mentalgear 5mo ago
For credential stealing, that is true, but at least it would protect your local machine. But I just read these worms also try container escape ...
- silon42 5mo ago
  We need to prevent direct connections to internet for containers... once you have a proxy, predefined credentials (api keys) can maybe be added there (per container/target).
  - cyanydeez 5mo ago
    the model most people are talking about is in the cloud. for the harness to do useful work, it needs to talk to the cloud the trouble is, we need protocols that are software determined that force AI interaxtions into limited scope but currently theyre all just bash adjacent and inherit your tools.
- fnoef 5mo ago
  You need to use full isolated VM with its own kernel. But then again, I've read somewhere that this malware is also trying to escape the VM isolation as well...
  - deleted 5mo ago
    [deleted]
jcgl 5mo ago
On my personal machine, I run OpenSnitch. Much better defense against data exfil if you reject outbound connections to unexpected/unwanted hosts.
- pingou 5mo ago
  That wouldn't help in that case as exfiltrated data is committed to public GitHub repositories. Unless you have to accept every time an app posts or requests data from known hosts?
  - gus_ 5mo ago
    Personally I don't allow outbound connections from almost any app, except web browsers to port 80/443. So nodejs, pip, ruby, curl, wget, etc, opening unexpected outbound connections is a big red flag for me. In some cases, maybe you need to allow permanently git to open outbound resquests to github.com (or gitlab, etc), but at least in my case, I'm okey allowing these connections manually. > preinstall script: bun run index.js > Dual exfiltration: > stolen data is committed as Git objects to public GitHub repositories (api.github.com) > and sent as RSA+AES encrypted HTTPS POSTs to hxxps://t.m-kosche[.]com/api/public/otel/v1/traces (disguised as OpenTelemetry traces) > The Bun installer command (command -v bun >/dev/null 2>&1 || (curl -fsSL https://bun.sh/install https://bun.sh/install | bash && export PATH=$HOME/.bun/bin:$PATH)) prepends every injected hook to guarantee Bun availability > A separate gh-token-monitor daemon (decrypted from J7, deployed by class so) installs to ~/.local/bin/gh-token-monitor.sh with its own systemd service and LaunchAgent. It polls stolen GitHub tokens at 60-second intervals with a 24-hour TTL This attack in particular would have caused OpenSnitch to go crazy, giving you the opportunity to review what's going on.
    - jcgl 5mo ago
      > Personally I don't allow outbound connections from almost any app, except web browsers to port 80/443. So nodejs, pip, ruby, curl, wget, etc, opening unexpected outbound connections is a big red flag for me. Yep, exactly. Reject by default, with reasonably judicious always-allow rules.
    - 5mo ago
TacticalCoder 5mo ago
Watertight subdivision in a ship doesn't promise: "there'll never ever be water in this ship". It says: "If there's water in this ship due to one hole, it'll stay in one compartment". Note that I said one hole: you have the titanic, many compartment gets holes, that one ship is still going to sink. (btw that the Titanic sunk is not an excuse not to secure other ships. And it did save a great many other ships to have watertight subdivision.) So... Although there are exploits escaping containers and VMs and then bad guys doing lateral moves across machines, you still want defense in depth.
- sitkack 5mo ago
  One thing could be more clear, is that the Titanic's bulkheads didn't go all the way to the top deck. They did not seal completely and were not watertight.
CGamesPlay 5mo ago
Things on my workstation that the container does not have access to: browser cookies, 1Password cli, SSH keys (even if I allow the SSH agent socket), cargo publish tokens (unless it’s a rust project), npm tokens (unless it’s an npm project), and not to mention anything relating to my other clients (don’t compromise my employer when I vibe install some dep for a random side project).

HWR_14 5mo ago

Without node, how do you control your cloud resources? Cloudflare requires wrangler. AWS has a lot of node clis. Etc.

anygivnthursday 5mo ago
From CI pipelines with pinned package versions and OIDC instead of permanent secrets limited, least privileges.
rubiquity 5mo ago
AWS has CLIs and SDKs in many languages.

zahlman 5mo ago

> the models will be good enough in social engineering Never mind questions of how good the models will/can get. I'm confused why people expect that, in principle, models getting really good at social engineering would have such huge impact. Seems to me like it has diminishing returns and is severely bottlenecked by the fact that the target operates at human speed. The amount of effort involved in the XZ hack, for example, was immense, and it couldn't have been accelerated because it worked specifically by wearing the existing maintainer down over time. You could generate and send all the necessary nastygrams in seconds and it wouldn't speed up the human consuming them (and in fact, having all of them arrive at once would raise suspicion). And there is a limit to how persuasive that input can be. Take any of the random nastygrams directed at the XZ maintainer; maybe they could have been made more nasty, more pointed, more aware of the maintainer's personal weaknesses and fears — but would that have actually been overall more effective? I think not, or at most only a tiny bit.

Terr_ 5mo ago
I'm imagining an influence campaign where someone weaponizes targeted advertising and recommendation engines against Mr. Trusted Maintainer, so that it's a The Truman Show effect, where the background of their entire digital bubble pushing the idea that they need retire and delegate.
- pyinstallwoes 5mo ago
  This definitely happens. Perhaps more on a state actor level of sophistication but yeah. I’ve seen it. It’s wild.