3 ms·
I wouldn't be surprised if they encrypt them at rest, but at some point the weights have to be loaded into vram.
by Retr0id 5mo ago
I wouldn't be surprised if they encrypt them at rest, but at some point the weights have to be loaded into vram.
- 05 5mo agoNewer NVidia cards (H100 and up) support both in-memory model encryption and ‘trusted’ execution environment/remote attestation, not sure how widely used in frontier model deployments, but at least vendor claimed perf overhead is ‘3%’ [0] [0] https://www.spheron.network/blog/confidential-gpu-computing-nvidia-tee-encrypted-vram/#:~:text=Performance%20Overhead:%20CC%20Mode%20vs%20Standard https://www.spheron.network/blog/confidential-gpu-computing-...