3 ms·
Super cool that this works. I'm surprised these companies re-use the same encryption key across models! I wonder if you can use these for attacks, like this pr
by x312 3mo ago
Super cool that this works. I'm surprised these companies re-use the same encryption key across models!
I wonder if you can use these for attacks, like this previous paper showing that if you know how a model reasons, you can "fake its thinking" to control it? https://news.ycombinator.com/item?id=48631888 https://news.ycombinator.com/item?id=48631888
- yubblegum 3mo agoSeriously, what does it take to encrypt per session? There are many ways to make it scalable and efficient so I am wondering if this is left like this to allow interested 3rd parties ahem unobtrusively peek what people are doing with the AI. (Thanks for the link. That’s an interesting idea!)
- dannyw 3mo agoThe provider has the hidden text anyway; this isn’t customer managed encryption.
- yubblegum 3mo agoSure, but if each session has a unique key then these need to be managed and stored and unauthorized access to these leaves tracks. So all that had to be 'compromised' is a single universally applicable key. Again, the question stands: session based encryption can be scalable and efficient. Why aren't they using it?
- paxys 3mo agoThe exploit here isn’t a leaked encryption key. It’s pretty likely that they are already using a unique key per conversation. The raw CoT eventually reaches the model, and you can convince the model to share it with you.
- theapadayo 3mo agoYeah encryption isn't the issue. The only way I see to fix this is if you stop the user from switching models mid-session, or strip out the thoughts when switching models. Either way you're degrading the user experience.
- yubblegum 3mo agoIf a different model is using encyrpted blocks of another model, then by definition it is no longer a session scoped bit of information. Since you can give it to any other session and another model, clearly it doesn't even have to be the same user. Therefore, there is only one (set) of universally available key(s) used by all models across all sessions.
- flexagoon 3mo ago> I'm surprised these companies re-use the same encryption key across models I assume switching the model in the middle of a conversation is intended behavior (very useful in coding agents, for example)