3 ms·
I assume they’re encrypted/DRM’ed when deployed on inference hardware, so only core researchers/sec admins would potentially have some access to unprotected wei
by 05 4mo ago
I assume they’re encrypted/DRM’ed when deployed on inference hardware, so only core researchers/sec admins would potentially have some access to unprotected weights, and they are far too well paid to risk it leaking the model
- jltsiren 4mo agoIncentives matter on the average, but people are too unpredictable for categorical statements like that. They can always have other reasons beyond personal gain to leak secrets. There was no shortage of spies and defectors leaking American nuclear secrets to the USSR during the Cold War.
- Retr0id 4mo agoI wouldn't be surprised if they encrypt them at rest, but at some point the weights have to be loaded into vram.
- 05 4mo agoNewer NVidia cards (H100 and up) support both in-memory model encryption and ‘trusted’ execution environment/remote attestation, not sure how widely used in frontier model deployments, but at least vendor claimed perf overhead is ‘3%’ [0] [0] https://www.spheron.network/blog/confidential-gpu-computing-nvidia-tee-encrypted-vram/#:~:text=Performance%20Overhead:%20CC%20Mode%20vs%20Standard https://www.spheron.network/blog/confidential-gpu-computing-...