4 ms·

I really don't understand why server-side storage of the trace isn't a viable approach here, with only a unique key flowing to the client and back. Does it hav

by aix1 2mo ago

I really don't understand why server-side storage of the trace isn't a viable approach here, with only a unique key flowing to the client and back. Does it have something to do with how backend load-balancing works?

sandeepkd 2mo ago
Yes, this storage would be growing exponentially making the disk space and latency problems harder (add the disaster recovery/backups). I think the choice of using client side is not too bad if you ensure that its secured properly. Also the company can excuse itself from the liability of storing sensitive data on its servers, thats a big deal in itself to be compliant for enterprise audits 1. The down side is that it cannot be used across the clients even for the same user 2. Using the same encryption key was a bad choice here, a per user key would have solved this issue for sure.
- aix1 2mo ago
  Having thought about this a little more, it's clear that server-side storage is not compatible with Zero Data Retention (ZDR). However, in non-ZDR settings, it seems likely that the providers are capturing all that data anyway? > a per user key would have solved this issue for sure It would have helped with PII leakage, but not with plain-text trace extraction attacks, right?
  - sandeepkd 2mo ago
    Per user encryption key ties it with the user session (assuming you do authentication properly), no one else can access it. User being able to see the information is not really an attack vector in this case. The compliance rules at times are outdated and people skirt around them by following the worded rule instead of the intent.
amluto 2mo ago
Makes no difference. There is a policy as to whether to allow use of a reasoning trace in a given context. Whether that trace originates from authenticated ciphertext or a backend database is basically irrelevant.
- aix1 2mo ago
  Good point, thanks.
dannyw 2mo ago
That’s incompatible with zero data retention and so you’ll lose a lot of enterprise customers.
cryptonector 2mo ago
Encrypted state cookies solve real problems (server-side storage, latency, scaling) and are not the problem. The problem is insufficient binding of some of a session's encrypted state cookies and others -- insufficient binding of some session state to other session state. Here we have HTTP encrypted state cookies for identifying authenticate user IDs and maybe for identifying sessions / chats, while the reasoning traces are also encrypted state cookies but not HTTP cookies, and the latter are somehow not sufficiently bound to the former. The fix is to either have per-user or per-session keys for encrypting reasoning traces, or write the user ID / account ID and maybe also session ID into the plaintext of the reasoning trace _then check that that matches the ones in the HTTP cookies when decrypting the traces_.

by aix1 2mo ago

sandeepkd 2mo ago

Yes, this storage would be growing exponentially making the disk space and latency problems harder (add the disaster recovery/backups). I think the choice of using client side is not too bad if you ensure that its secured properly. Also the company can excuse itself from the liability of storing sensitive data on its servers, thats a big deal in itself to be compliant for enterprise audits 1. The down side is that it cannot be used across the clients even for the same user 2. Using the same encryption key was a bad choice here, a per user key would have solved this issue for sure.

aix1 2mo ago
Having thought about this a little more, it's clear that server-side storage is not compatible with Zero Data Retention (ZDR). However, in non-ZDR settings, it seems likely that the providers are capturing all that data anyway? > a per user key would have solved this issue for sure It would have helped with PII leakage, but not with plain-text trace extraction attacks, right?
- sandeepkd 2mo ago
  Per user encryption key ties it with the user session (assuming you do authentication properly), no one else can access it. User being able to see the information is not really an attack vector in this case. The compliance rules at times are outdated and people skirt around them by following the worded rule instead of the intent.

amluto 2mo ago

Makes no difference. There is a policy as to whether to allow use of a reasoning trace in a given context. Whether that trace originates from authenticated ciphertext or a backend database is basically irrelevant.

aix1 2mo ago
Good point, thanks.

dannyw 2mo ago

That’s incompatible with zero data retention and so you’ll lose a lot of enterprise customers.

cryptonector 2mo ago

Encrypted state cookies solve real problems (server-side storage, latency, scaling) and are not the problem. The problem is insufficient binding of some of a session's encrypted state cookies and others -- insufficient binding of some session state to other session state. Here we have HTTP encrypted state cookies for identifying authenticate user IDs and maybe for identifying sessions / chats, while the reasoning traces are also encrypted state cookies but not HTTP cookies, and the latter are somehow not sufficiently bound to the former. The fix is to either have per-user or per-session keys for encrypting reasoning traces, or write the user ID / account ID and maybe also session ID into the plaintext of the reasoning trace _then check that that matches the ones in the HTTP cookies when decrypting the traces_.