11 ms·

Google Antigravity exfiltrates data via indirect prompt injection attack

• nprateem 10mo ago
  I said months ago you'd be nuts to let these things loose on your machine. Quelle surprise.
• jjmaxwell4 10mo ago
  I know that Cursor and the related IDEs touch millions of secrets per day. Issues like this are going to continue to be pretty common.
  • iamsaitam 10mo ago
    If the secrets are in a .env file and you have them in your .gitignore they don't, as you should.
    • sixeyes 10mo ago
      did you miss the part where the agent immediately went around it? the .gitignore applies to the agent's own "read file" tool. not allowed? it will just run "cat .env" and be happy
• akshey-pr 10mo ago
  Damn, i paste links into cursor all the time. Wonder if the same applies, but definitely one more reason not to use antigravity
  • pennomi 10mo ago
    Cursor is also vulnerable to prompt injection through third-party content.
    • verdverm 10mo ago
      this is one reason to favor specialized agents and/or tool selection with guards (certain tools cannot appear together in a LLM request)
• mkagenius 10mo ago
  Sooner or later I believe, there will be models which can be deployed locally on your mac and are as good as say Sonnet 4.5. People should shift to completely local at that point. And use sandbox for executing code generated by llm. Edit: "completely local" meant not doing any network calls unless specifically approved. When llm calls are completely local you just need to monitor a few explicit network calls to be sure. Unlike gemini then you don't have to rely on certain list of whitelisted domains.
  • kami23 10mo ago
    I've been repeating something like 'keep thinking about how we would run this in the DC' at work. The cycles of pushing your compute outside the company and then bringing it back in once the next VP/Director/CTO starts because they need to be seen as doing something, and the thing that was supposed to make our lives easier is now very expensive... I've worked on multiple large migrations between DCs and cloud providers for this company and the best thing we've ever done is abstract our compute and service use to the lowest common denominator across the cloud providers we use...
  • KK7NIL 10mo ago
    If you read the article you'd notice that running an LLM locally would not fix this vulnerability.
    • yodon 10mo ago
      From the HN guidelines[0]: >Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that". [0]: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
      • KK7NIL 10mo ago
        That's fair, thanks for the heads up.
    • pennomi 10mo ago
• serial_dev 10mo ago
  > Gemini is not supposed to have access to .env files in this scenario (with the default setting ‘Allow Gitignore Access > Off’). However, we show that Gemini bypasses its own setting to get access and subsequently exfiltrate that data. They pinky promised they won’t use something, and the only reason we learned about it is because they leaked the stuff they shouldn’t even be able to see?
  • ArcHound 10mo ago
    When I read this I thought about a Dev frustrated with a restricted environment saying "Well, akschually.." So more of a Gemini initiated bypass of it's own instructions than malicious Google setup. Gemini can't see it, but it can instruct cat to output it and read the output. Hilarious.
    • withinboredom 10mo ago
      codex cli used to do this. "I can't run go test because of sandboxing rules" and then proceeds to set obscure environment variables and run it anyway. What's funny, is that it could just ask the user for permission to run "go test"
      • tetha 10mo ago
        A tired and very cynical part of me has to note: To the LLMs have reached the intelligence of an average solution consultant. Are they also frustrated if their entirely unsanctioned solution across 8 different wall bounces which randomly functions (just as stable as a house of cards on a dyke near the north sea in storm gusts) stops working?
    • empath75 10mo ago
      Cursor does this too.
  • bo1024 10mo ago
    As you see later, it uses cat to dump the contents of a file it’s not allowed to open itself.
• adezxc 10mo ago
  That's the bleeding edge you get with vibe coding
  • aruametello 10mo ago
    cutting edge perhaps?
    • zahlman 10mo ago
      "Bleeding edge" is an established English idiom, especially in technology: https://www.merriam-webster.com/dictionary/bleeding%20edge https://www.merriam-webster.com/dictionary/bleeding%20edge
• ares623 10mo ago
  [flagged]
• ArcHound 10mo ago
  Who would have thought that having access to the whole system can be used to bypass some artificial check. There are tools for that, sandboxing, chroots, etc... but that requires engineering and it slows GTM, so it's a no-go. No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. EDIT: they did, but left a site that enables arbitrary redirects in the default config. Fundamentally, with LLMs you can't separate instructions from data, which is the root cause for 99% of vulnerabilities. Security is hard man, excellent article, thoroughly enjoyed.
  • cowpig 10mo ago
    > No, local models won't help you here, unless you block them from the internet or setup a firewall for outbound traffic. This is the only way. There has to be a firewall between a model and the internet. Tools which hit both language models and the broader internet cannot have access to anything remotely sensitive. I don't think you can get around this fact.
    • ArcHound 10mo ago
      The sad thing is, that they've attempted to do so, but left a site enabling arbitrary redirects, which defeats the purpose of the firewall for an informed attacker.
    • miohtama 10mo ago
      How will the firewall for LLM look like? Because the problem is real, there will be a solution. Manually approve domains it can do HTTP requests to, like old school Windows firewalls?
      • ArcHound 10mo ago
        Yes, curated whitelist of domains sounds good to me. Of course, everything by Google they will still allow. My favourite firewall bypass to this day is Google translate, which will access arbitrary URL for you (more or less). I expect lots of fun with these.
• raincole 10mo ago
  I mean, agent coding is essentially copypasting code and shell commands from StackOverflow without reading them. Or installing a random npm package as your dependency. Should you do that? Maybe not, but people will keep doing that anyway as we've seen in the era of StackOverflow.
• bigbuppo 10mo ago
  Data Exfiltration as a Service is a growing market.
• liampulles 10mo ago
  Coding agents bring all the fun of junior developers, except that all the accountability for a fuckup rests with you. Great stuff, just awesome.
• jsmith99 10mo ago
  There's nothing specific to Gemini and Antigravity here. This is an issue for all agent coding tools with cli access. Personally I'm hesitant to allow mine (I use Cline personally) access to a web search MCP and I tend to give it only relatively trustworthy URLs.
  • ArcHound 10mo ago
    For me the story is that Antigravity tried to prevent this with a domain whitelist and file restrictions. They forgot about a service which enables arbitrary redirects, so the attackers used it. And LLM itself used the system shell to pro-actively bypass the file protection.
  • IshKebab 10mo ago
    I do think they deserve some of the blame for encouraging you to allow all commands automatically by default.
    • buu700 10mo ago
      YOLO-mode agents should be in a dedicated VM at minimum, if not a dedicated physical machine with a strict firewall. They should be treated as presumed malware that just happens to do something useful as a side effect. Vendors should really be encouraging this and providing tooling to facilitate it. There should be flashing red warnings in any agentic IDE/CLI whenever the user wants to use YOLO mode without a remote agent runner configured, and they should ideally even automate the process of installing and setting up the agent runner VM to connect to.
      • 0xbadcafebee 10mo ago
        But they literally called it 'yolo mode'. It's an idiot button. If they added protections by default, someone would just demand an option to disable all the protections, and all the idiots would use that.
        • buu700 10mo ago
• lbeurerkellner 10mo ago
  Interesting report. Though, I think many of the attack demos cheat a bit, by putting injections more or less directly in the prompt (here via a website at least). I know it is only one more step, but from a privilege perspective, having the user essentially tell the agent to do what the attackers are saying, is less realistic then let’s say a real drive-by attack, where the user has asked for something completely different. Still, good finding/article of course.
  • Capricorn2481 10mo ago
    > Though, I think many of the attack demos cheat a bit, by putting injections more or less directly in the prompt (here via a website at least) What difference does that make? The prompt is to read a website and the injection is on that website hidden in html. People aren't going to read the HTML of every website before they scrape it, so this is not an unrealistic vulnerability. Even worse, it ran arbitrary commands to get around its own restrictions. This just confirms if Antigravity tries to scrape a website with user generated content for any reason, whether the user provides the link or not, you have left your entire machine vulnerable.
• xnx 10mo ago
  OCR'ing the page instead of reading the 1 pixel font source would add another layer of mitigation. It should not be possible to send the machine a different set of instructions than a person would see.
• Epsom2025 10mo ago
  good
• zgk7iqea 10mo ago
  Don't cursor and vscode also have this problem?
  • verdverm 10mo ago
    Probably all of them do, depending on settings. Copilot / vscode will ask you to confirm link access before it will fetch it or you set the domain as trusted.
• wingmanjd 10mo ago
  I really liked Simon's Willison's [1] and Meta's [2] approach using the "Rule of Two". You can have no more than 2 of the following: - A) Process untrustworthy input - B) Have access to private data - C) Be able to change external state or communicate externally. It's not bullet-proof, but it has helped communicate to my management that these tools have inherent risk when they hit all three categories above (and any combo of them, imho). [EDIT] added "or communicate externally" to option C. [1] https://simonwillison.net/2025/Nov/2/new-prompt-injection-papers/ https://simonwillison.net/2025/Nov/2/new-prompt-injection-pa... [2] https://ai.meta.com/blog/practical-ai-agent-security/ https://ai.meta.com/blog/practical-ai-agent-security/
  • ArcHound 10mo ago
    I recall that. In this case, you have only A and B and yet, all of your secrets are in the hands of an attacker. It's great start, but not nearly enough. EDIT: right, when we bundle state with external Comms, we have all three indeed. I missed that too.
    • malisper 10mo ago
      Not exactly. Step E in the blog post: > Gemini exfiltrates the data via the browser subagent: Gemini invokes a browser subagent per the prompt injection, instructing the subagent to open the dangerous URL that contains the user's credentials. fulfills the requirements for being able to change external state
      • ArcHound 10mo ago
        I disagree. No state "owned" by LLM changed, it only sent a request to the internet like any other. EDIT: In other words, the LLM didn't change any state it has access to. To stretch this further - clicking on search results changes the internal state of Google. Would you consider this ability of LLM to be state-changing? Where would you draw the line?
        • wingmanjd 10mo ago
• godelski 10mo ago
  Does anyone else find it concerning how we're just shipping alpha code these days? I know it's really hard to find all bugs internally and you gotta ship, but it seems like we're just outsourcing all bug finding to people, making them vulnerable in the meantime. A "bug" like this seems like one that could have and should have been found internally. I mean it's Google, not some no-name startup. And companies like Microsoft are ready to ship this alpha software into the OS? Doesn't this kinda sound insane? I mean regardless of how you feel about AI, we can all agree that security is still a concern, right? We can still move fast while not pushing out alpha software. If you're really hyped on AI then aren't you concerned that low hanging fruit risks bringing it all down? People won't even give it a chance if you just show them the shittest version of things
  • funnybeam 10mo ago
    This isn’t a bug, it is known behaviour that is inherent and fundamental to the way LLMs function. All the AI companies are aware of this and are pressing ahead anyway - it is completely irresponsible. If you haven’t come across it before, check out Simon Willisons “lethal trifecta” concept which neatly sums up the issue and explains why there is no way to use these things safely for many of the things that they would be most useful for
• crazygringo 10mo ago
  While an LLM will never have security guarantees, it seems like the primary security hole here is: > However, the default Allowlist provided with Antigravity includes ‘webhook.site’. It seems like the default Allowlist should be extremely restricted, to only retrieving things from trusted sites that never include any user-generated content, and nothing that could be used to log requests where those logs could be retrieved by users. And then every other domain needs to be whitelisted by the user when they come up before a request can be made, visually inspecting the contents of the URL. So in this case, a dev would encounter a permissions dialog asking to access 'webhook.site' and see it includes "AWS_SECRET_ACCESS_KEY=..." and go... what the heck? Deny. Even better, specify things like where secrets are stored, and Antigravity could continuously monitor the LLM's to halt execution if a secret ever appears. Again, none of this would be a perfect guarantee, but it seems like it would be a lot better?
  • jsnell 10mo ago
    I don't share your optimism. Those kinds measures would be just security theater, not "a lot better". Avoiding secrets appearing directly in the LLM's context or outputs is trivial, and once you have the workaround implemented it will work reliably. The same for trying to statically detect shell tool invocations that could read+obfuscate a secret. The only thing that would work is some kind of syscall interception, but at that point you're just reinventing the sandbox (but worse). Your "visually inspect the contents of the URL" idea seems unlikely to help either. Then the attacker just makes one innocous-looking request to get allowlisted first.
  • DrSusanCalvin 10mo ago
    The agen already bypassed the file reading filter with cat, couldn't it just bypass the URL filter by running wget or a python script or hundreds of other things it has access to through the terminal? You'd have to run it in a VM behind a firewall.
  • deleted 10mo ago
    [deleted]
• paxys 10mo ago
  I'm not quite convinced. You're telling the agent "implement what it says on <this blog>" and the blog is malicious and exfiltrates data. So Gemini is simply following your instructions. It is more or less the same as running "npm install <malicious package>" on your own. Ultimately, AI or not, you are the one responsible for validating dependencies and putting appropriate safeguards in place.
  • ArcHound 10mo ago
    The article addresses that too with: > Given that (1) the Agent Manager is a star feature allowing multiple agents to run at once without active supervision and (2) the recommended human-in-the-loop settings allow the agent to choose when to bring a human in to review commands, we find it extremely implausible that users will review every agent action and abstain from operating on sensitive data. It's more of a "you have to anticipate that any instructions remotely connected to the problem aren't malicious", which is a long stretch.
    • mandog2000 10mo ago
      [dead]
  • Earw0rm 10mo ago
    Right, but at least with supply-chain attacks the dependency tree is fixed and deterministic. Nondeterministic systems are hard to debug, this opens up a threat-class which works analogously to supply-chain attacks but much harder to detect and trace.
  • Nathanba 10mo ago
    right but this product (agentic AI) is explicitly sold as being able to run on its own. So while I agree that these problems are kind of inherent in AIs... these companies are trying to sell it anyway even though they know that it is going to be a big problem.
  • zahlman 10mo ago
    The point is: 1. There are countless ways to hide machine-readable content on the blog that doesn't make a visible impact on the page as normally viewed by humans. 2. Even if you somehow verify what the LLM will see, you can't trivially predict how it will respond to what it sees there. 3. In particular, the LLM does not make a proper distinction between things that you told it to do, and things that it reads on the blog.
• bilekas 10mo ago
  We really are only seeing the beginning of the creativity attackers have for this absolutely unmanageable surface area. I ma hearing again and again by collegues that our jobs are gone, and some are definitely going to go, thankfully I'm in a position to not be too concerned with that aspect but seeing all of this agentic AI and automated deployment and trust that seems to be building in these generative models from a birds eye view is terrifying. Let alone the potential attack vector of GPU firmware itself given the exponential usage they're seeing. If I was a state well funded actor, I would be going there. Nobody seems to consider it though and so I have to sit back down at parties and be quiet.
  • MengerSponge 10mo ago
    Firms are waking up to the risk: https://techcrunch.com/2025/11/23/ai-is-too-risky-to-insure-say-people-whose-job-is-insuring-risk/ https://techcrunch.com/2025/11/23/ai-is-too-risky-to-insure-...
    • bilekas 10mo ago
      You know you're risky when AIG are not willing to back you. I'm old enough to remember the housing bubble and they were not exactly strict with their coverage.
  • Quothling 10mo ago
    I think it depends on where you work. I do quite a lot of work with agentic AI, but it's not like it's much of a risk factor when they have access to nothing. Which they won't have because we haven't even let humans have access to any form of secrets for decades. I'm not sure why people think it's a good idea, or necessary, to let agents run their pipelines, especially if you're storing secrets in envrionment files... I mean, one of the attacks in this article is getting the agent to ignore .gitignore... but what sort of git repository lets you ever push a .env file to begin with? Don't get me wrong, the next attack vector would be renaming the .env file to 2600.md or something but still. That being said. I think you should actually upscale your party doomsaying. Since the Russian invasion kicked EU into action, we've slowly been replacing all the OT we have with known firmware/hardware vulnerabilities (very quickly for a select few). I fully expect that these are used in conjunction with whatever funsies are being build into various AI models as well as all the other vectors for attacks.
• simonw 10mo ago
  Antigravity was also vulnerable to the classic Markdown image exfiltration bug, which was reported to them a few days ago and flagged as "intended behavior" I'm hoping they've changed their mind on that but I've not checked to see if they've fixed it yet. https://x.com/p1njc70r/status/1991231714027532526 https://x.com/p1njc70r/status/1991231714027532526
  • wunderwuzzi23 10mo ago
    It still is. plus there are many more issue. i documented some here: https://embracethered.com/blog/posts/2025/security-keeps-google-antigravity-grounded/ https://embracethered.com/blog/posts/2025/security-keeps-goo...
  • deleted 10mo ago
    [deleted]
• drmath 10mo ago
  One source of trouble here is that the agent's view of the web page is so different from the human's. We could reduce the incidence of these problems by making them more similar. Agents often have some DOM-to-markdown tool they use to read web pages. If you use the same tool (via a "reader mode") to view the web page, you'd be assured the thing you're telling the agent to read is the same thing you're reading. Cursor / Antigravity / etc. could have an integrated web browser to support this. That would make what the human sees closer to what the agent sees. We could also go the other way by having the agent's web browsing tool return web page screenshots instead of DOM / HTML / Markdown.
• jtokoph 10mo ago
  The prompt injection doesn’t even have to be in 1px font or blending color. The malicious site can just return different content based on the user-agent or other way of detecting the AI agent request.
  • pilingual 10mo ago
    AI trains people to be lazy, so it could be in plain sight buried in the instructions.
• ineedasername 10mo ago
  Are people not taking this as a default stance? Your mental model for this on security can’t be “it’s going to obey rules that are are enforced as conventions but not restrictions” Which is what you’re doing if you expect it to respect guidelines in a config. You need to treat it, in some respects, as someone you’re letting have an account on your computer so they can work off of it as well.
• dzonga 10mo ago
  the money security researchers & pentesters gonna get due to vulnerabilities from these a.i agents has gone up. likewise for the bad guys
• azeitona 10mo ago
  Software engineering became a pita with these tools intruding to do the work for your.
• Humorist2290 10mo ago
  One thing that especially interests me about these prompt-injection based attacks is their reproducibility. With some specific version of some firmware it is possible to give reproducible steps to identify the vulnerability, and by extension to demonstrate that it's actually fixed when those same steps fail to reproduce. But with these statistical models, a system card that injects 32 random bits at the beginning is enough to ruin any guarantee of reproducibility. Self-hosted models sure you can hash the weights or something, but with Gemini (/etc) Google (/et al) has a vested interest in preventing security researchers from reproducing their findings. Also rereading the article, I cannot put down the irony that it seems to use a very similar style sheet to Google Cloud Platform's documentation.
• pshirshov 10mo ago
  Run your shit in firejail. /thread
• wunderwuzzi23 10mo ago
  Cool stuff. Interestingly, I responsibly disclosed that same vulnerability to Google last week (even using the same domain bypass with webhook.site). For other (publicly) known issues in Antigravity, including remote command execution, see my blog post from today: https://embracethered.com/blog/posts/2025/security-keeps-google-antigravity-grounded/ https://embracethered.com/blog/posts/2025/security-keeps-goo...
• JyB 10mo ago
  How is that specific to antigravity? Seem like it could happen with a bunch of tools
  • thomas34298 10mo ago
    Codex can read any file on your PC without your explicit approval. Other agents like Claude Code would at least ask you or are sufficiently sandboxed.
    • throitallaway 10mo ago
      I'm not sure how much sandboxing can help here. Presumably you're giving the tool access to a repo directory, and that's where a juicy .env file can live. It will also have access to your environment variables. I suspect a lot of people permanently allow actions and classes of commands to be run by these tools rather than clicking "yes" a bunch of times during their workflows. Ride the vibes.
      • thomas34298 10mo ago
        That's the entire point of sandboxing, so none of what you listed would be accessible by default. Check out https://github.com/anthropic-experimental/sandbox-runtime https://github.com/anthropic-experimental/sandbox-runtime and https://github.com/Zouuup/landrun https://github.com/Zouuup/landrun as examples on how you could restrict agents for example.
• Nifty3929 10mo ago
  Proposed title change: Google Antigravity can be made to exfiltrate your own data
• simonw 10mo ago
  This kind of problem is present in most of the currently available crop of coding agents. Some of them have default settings that would prevent it (though good luck figuring that out for each agent in turn - I find those security features are woefully under-documented). And even for the ones that ARE secure by default... anyone who uses these things on a regular basis has likely found out how much more productive they are when you relax those settings and let them be more autonomous (at an enormous increase in personal risk)! Since it's so easy to have credentials stolen, I think the best approach is to assume credentials can be stolen and design them accordingly: - Never let a coding agent loose on a machine with credentials that can affect production environments: development/staging credentials only. - Set budget limits on the credentials that you expose to the agents, that way if someone steals them they can't do more than $X worth of damage. As an example: I do a lot of work with https://fly.io/ https://fly.io/ and I sometimes want Claude Code to help me figure out how best to deploy things via the Fly API. So I created a dedicated Fly "organization", separate from my production environment, set a spending limit on that organization and created an API key that could only interact with that organization and not my others.
• simonw 10mo ago
  More reports of similar vulnerabilities in Antigravity from Johann Rehberger: https://embracethered.com/blog/posts/2025/security-keeps-google-antigravity-grounded/ https://embracethered.com/blog/posts/2025/security-keeps-goo... He links to this page on the Google vulnerability reporting program: https://bughunters.google.com/learn/invalid-reports/google-products/4655949258227712/antigravity-known-issues https://bughunters.google.com/learn/invalid-reports/google-p... That page says that exfiltration attacks against the browser agent are "known issues" that are not eligible for reward (they are already working on fixes): > Antigravity agent has access to files. While it is cautious in accessing sensitive files, there’s no enforcement. In addition, the agent is able to create and render markdown content. Thus, the agent can be influenced to leak data from files on the user's computer in maliciously constructed URLs rendered in Markdown or by other means. And for code execution: > Working with untrusted data can affect how the agent behaves. When source code, or any other processed content, contains untrusted input, Antigravity's agent can be influenced to execute commands. [...] > Antigravity agent has permission to execute commands. While it is cautious when executing commands, it can be influenced to run malicious commands.
  • kccqzy 10mo ago
    As much as I hate to say it, the fact that the attacks are “known issues” seems well known in the industry among people who care about security and LLMs. Even as an occasional reader of your blog (thank you for maintaining such an informative blog!), I know about the lethal trifecta and the exfiltration risks since early ChatGPT and Bard. I have previously expressed my views on HN about removing one of the three lethal trifecta; it didn’t go anywhere. It just seems that at this phase, people are so excited about the new capabilities LLMs can unlock that they don’t care about security.
    • Helmut10001 10mo ago
      Then, the goal must be to guide users to run Antigravity in a sandbox, with only the data or information that it must access.
    • TeMPOraL 10mo ago
• deleted 10mo ago
  [deleted]
• p1necone 10mo ago
  I feel like I'm going insane reading how people talk about "vulnerabilities" like this. If you give an llm access to sensitive data, user input and the ability to make arbitrary http calls it should be blindingly obvious that it's insecure. I wouldn't even call this a vulnerability, this is just intentionally exposing things. If I had to pinpoint the "real" vulnerability here, it would be this bit, but the way it's just added as a sidenote seems to be downplaying it: "Note: Gemini is not supposed to have access to .env files in this scenario (with the default setting ‘Allow Gitignore Access > Off’). However, we show that Gemini bypasses its own setting to get access and subsequently exfiltrate that data."
  • simonw 10mo ago
    These aren't vulnerabilities in LLMs. They are vulnerabilities in software that we build on top of LLMs. It's important we understand them so we can either build software that doesn't expose this kind of vulnerability or, if we build it anyway, we can make the users of that software aware of the risks so they can act accordingly.
    • zahlman 10mo ago
      Right; the point is that it's the software that gives "access to sensitive data, user input and the ability to make arbitrary http calls" to the LLM. People don't think of this as a risk when they're building the software, either because they just don't think about security at all, or because they mentally model the LLM as unerringly subservient to the user — as if we'd magically solved the entire class of philosophical problems Asimov pointed out decades ago without even trying.
  • TacticalCoder 10mo ago
    [dead]
• j45 10mo ago
  This is slightly terrifying. All these years of cybersecurity build up and now there's these generic and vague wormholes right into it all.
• deleted 10mo ago
  [deleted]
• Habgdnv 10mo ago
  Ok, I am getting mad now. I don't understand something here. Should we open like 31337 different CVEs about every possible LLM on the market and tell them that we are super-ultra-security-researchers and we're shocked when we found out that <model name> will execute commands that it is given access to, based on the input text that is feed into the model? Why people keep doing these things? Ok, they have free time to do it and like to waste other's people time. Why is this article even on HN? How is this article in the front page? "Shocking news - LLMs will read code comments and act on them as if they were instructions".
  • Wolfenstein98k 10mo ago
    Isn't the problem here that third parties can use it as an attack vector?
    • Habgdnv 10mo ago
      The problem is a bit wider than that. One can frame it as "google gemini is vulterable" or "google's new VS code clone is vulnerable". The bigger picture is that the model predicts tokens (words) based on all the text it have. In a big codebase it becomes exponentially easier to mess the model's mind. At some point it will become confused what is his job. What is part of the "system prompt" and "code comments in the codebase" becomes blurry. Even the models with huge context windows get confused because they do not understand the difference between your instructions and "injected instructions" in a hidden text in the readme or in code comments. They see tokens and given enough malicious and cleverly injected tokens the model may and often will do stupid things. (The word "stupid" means unexpected by you) People are giving LLMs access to tools. LLMs will use them. No matter if it's Antigravity, Aider, Cursor, some MCP.
      • danudey 10mo ago
        I'm not sure what your argument is here. We shouldn't be making a fuss about all these prompt injection attacks because they're just inevitable so don't worry about it? Or we should stop being surprised that this happens because it happens all the time? Either way I would be extremely concerned about these use cases in any circumstance where the program is vulnerable and rapid, automatic or semi-automatic updates aren't available. My Ubuntu installation prompts me every day to install new updates, but if I want to update e.g. Kiro or Cursor or something it's a manual process - I have to see the pop-up, decide I want to update, go to the download page, etc. These tools are creating huge security concerns for anyone who uses them, pushing people to use them, and not providing a low-friction way for users to ensure they're running the latest versions. In an industry where the next prompt injection exploit is just a day or two away, rapid iteration would be key if rapid deployment were possible.
• brendoelfrendo 10mo ago
  We taught sand to think and thought we were clever, when in reality all this means is that now people can social engineer the sand.
• nextworddev 10mo ago
  Did Cursor pay this guy to write this FUD?
• rvz 10mo ago
  Never thought to see the standards for software development at Google to drop this low as not only they are embracing low quality software like Electron, the software was riddled with this embarrassing security issue. Absolute amateurs.
• throwaway173738 10mo ago
  This is kind of the LLM equivalent to “hello I’m the CEO please email me your password to the CI/CD system immediately so we can sell the company for $1000/share.”
• leo_e 10mo ago
  The most concerning part isn't the vulnerability itself, but Google classifying it as a "Known Issue" ineligible for rewards. It implies this is an architectural choice, not a bug. They are effectively admitting that you can't have an "agentic" IDE that is both useful and safe. They prioritized the feature set (reading files + internet access) over the sandbox. We are basically repeating the "ActiveX" mistakes of the 90s, but this time with LLMs driving the execution.
  • simonw 10mo ago
    That's a misinterpretation of what they mean by "known issue". Here's the full context from https://bughunters.google.com/learn/invalid-reports/google-products/4655949258227712/antigravity-known-issues https://bughunters.google.com/learn/invalid-reports/google-p... > For full transparency and to keep external security researchers hunting bugs in Google products informed, this article outlines some vulnerabilities in the new Antigravity product that we are currently aware of and are working to fix. Note the "are working to fix". It's classified as a "known issue" because you can't earn any bug bounty money for reporting it to them.
• Ethon 10mo ago
  Developers must rethink both agent permissions and allowlists
• defraxi 10mo ago
  [dead]
• celeryd 10mo ago
  Is it exfiltration if it's your own data within your own control?
• sixeyes 10mo ago
  i noticed this EXACT behavior of cat-ing .env in cursor too. completely flabbergasted. i saw it tried to read the .env to check that a token was present. couldn't due to policy ("delightful! someone thought this through.") but then immediately tried and succeeded in bypassing it.
• abir_taheer 10mo ago
  hi! we actually built a service to detect indirect prompt injections like this. I tested out the exact prompt used in this attack and we were able to successfully detect the indirect prompt injection. Feel free to reach out if you're trying to build safeguards into your ai system! centure.ai POST - https://api.centure.ai/v1/prompt-injection/text https://api.centure.ai/v1/prompt-injection/text Response: { "is_safe": false, "categories": [ { "code": "data_exfiltration", "confidence": "high" }, { "code": "external_actions", "confidence": "high" } ], "request_id": "api_u_t6cmwj4811e4f16c4fc505dd6eeb3882f5908114eca9d159f5649f", "api_key_id": "f7c2d506-d703-47ca-9118-7d7b0b9bde60", "request_units": 2, "service_tier": "standard" }