4 ms·
If you are curious about what use case there is of it: imagine that instead of JPEG compression, we are running an LLM. You send your prompt encrypted, server h
by xyzzyz 2y ago
If you are curious about what use case there is of it: imagine that instead of JPEG compression, we are running an LLM. You send your prompt encrypted, server homomorphically executes the model and sends the encrypted output, which you then decrypt. As a result, you get LLM output, but the server operators never learn anything about the contents of the query or response.
- HeatrayEnjoyer 2y agoI wonder if FHE would ever be practical for an LLM. FHE necessitates calculating over the entire data, an extreme requirement for many workloads but routine for LLM inference.
- totetsu 2y agoBut.. if the LLM uses something like a MOE architecture, then surely some information about the prompt would leak just by observing what parts of the model are pulled into memory?
- palata 2y agoI think if you want full homomorphic encryption, you have to go for the worst case. So you always pull all the model? That's usually why FHE is not practical.