3 ms·
Open source doesn't matter if someone else is running it, right? They can change it? As long as the prompt is not encrypted at some point, and I don't think LL
by XCSme 2mo ago
Open source doesn't matter if someone else is running it, right? They can change it?
As long as the prompt is not encrypted at some point, and I don't think LLMs can run on encrypted prompts, then it can be read.
- ljlolel 2mo agoWith confidential compute / TEEs you can guarantee that the code is running, it's verifiable with remote attestation
- XCSme 2mo agoSo where does the guarantee stop? At the GPU driver level? Firmware level? What if the GPU has a custom bios flash that somehow logs the unencrypted prompts?
- ljlolel 2mo agogoes all the way to keys owned by Nvidia and Intel
- inigyou 2mo agoSGX has been cracked
- ljlolel 2mo agodepends on your threat model