4 ms·
Hackers can read private AI assistant chats even though they're encrypted
- acureau 3y agoInteresting read, from the title I figured the article was going to be about "hackers" breaching accounts and reading past conversations. I don't think reading private chats is the way to put it, more like inferring topics of discussion with some accuracy
- lolinder 3y agoThe actual attack is described near the end of the article—ChatGPT and friends send tokens as fast as they are generated, and the encrypted data doesn't mask the size of the token. This allows the attacker to guess the size of each token as it comes in, which they feed into a specialized model that guesses the word sequence based on the token lengths. Mitigating this should be very simple—pad the token with null to some reasonable length before sending it, then clip it on the client side. This would result in slightly higher bandwidth usage, but not enough to be perceptible.
- williamcotton 3y agoThere’s something very enticing to a cracker to have the same data passed back each time as well, that conversation history with the chat bot.
- hedora 3y agoSSH already has mitigations for this, for what it's worth.
- yunohn 3y agoThis is actually a very interesting side channel attack for streaming data. They use an LLM to guess words from token length. Definitely, padding would help here. Nit: Title should say “guess” and not “read”.
- SushiHippie 3y agoRelated article on frontpage: Cloudflare mitigates AI side channel attack | https://news.ycombinator.com/item?id=39703255 https://news.ycombinator.com/item?id=39703255
- wkat4242 3y agoInteresting attack, but it's easily mitigated. Just batch the responses into sentences or something instead of words. Or padding, as others have suggested. The services will adapt quickly and the issue will be solved. It also requires a full packet capture of the target making it not very easy to execute.
- foogazi 3y ago> The services will adapt quickly and the issue will be solved. sure, now that they know about it