4 ms·
> FHE gives you NO guarantee about the code that is running on the encrypted data. I can run a leaky AI model or a SELECT * on encrypted data and still get the
by bawolff 4y ago
> FHE gives you NO guarantee about the code that is running on the encrypted data. I can run a leaky AI model or a SELECT * on encrypted data and still get the output. What I can do (and that's assuming there is open-sourced, auditable code) is to make sure that anyone with hypervisor access on that machine cannot dump my data out during processing.
I might be misunderstanding but i think this is misleading. Any code can be run, but the person running the code cannot see the results (or any side effects), so they cannot leak data.
- v4dok 4y agoDepends on what do you mean by "person running the code" If by person you mean the admin of the machine, then yes. If by person you mean the developer of the FHE-based application, then "maybe" If by person you mean the analyst who would in the end order an AI python model to be executed through the FHE-based software on a machine. Then no, that person will in the end get back human-readable results. Be that a model, or a table from an SQL DB running in FHE
- vesinisa 4y agoThe way I have understood FHE is that any algorithm that would operate on the data would, by definition, be unable to produce any result that was intelligible to anyone except the person holding the original key. At no point during the execution of an FHE algorithm is the data decrypted. The amazing thing is exactly that the code running on the data does not understand the data it is consuming nor the data that it is producing. Maybe someone who has actually studied homomorphic encryption can chime in.
- v4dok 4y agoThats true. But it will still produce some data, and that data will be viewed by someone eventually who owns the key to decrypt it. FHE tells you nothing about what this product should be. It could as well be a full copy of the original data. For example: I run an ML model using FHE on some data I shouldn't have access to in plaintext. The expected outcome of this workflow is a trained ML model on that data. FHE tells me nothing about the quality of this model. It could as well be an overfit model that spits out all the sensitive data.
- jxf 4y agoWouldn't doing the same thing without FHE also result in the same problem?
- v4dok 4y agoYeah, and many more. But I've seen multiple people argue that using FHE will magically solve all their privacy problems and its far from true. FHE (and similar technologies) solve a piece of that "puzzle" and most providers somehow gloss it over.
- deleted 4y ago[deleted]
- vesinisa 4y agoSorry but I still fail to see how that would be a problem, since the output of the program (e.g. the ML model parameters) would themselves not be intelligible to you. To make _any_ (non-cryptanalytical) inference on the plaintext of the homomorphically encrypted data _necessarily_ requires that the attacker at some points can access or execute some classical code on the plaintext. This would obviously violate the "fully" part of FHE. Edit: Okay so I might now understand you refer to a scenario where the user submits their data in homomorphic form to the cloud, where an AI model is trained on it. The AI model parameters are later returned to the user's device, which then decrypts them with the user's key and executes a classical model with those parameters, and then resubmits the user's data after processing with the said ML model (unencrypted) back to the cloud. It's true the user usually has no way of auditing the code / model that runs on their device, but isn't that rather easily alleviated by opening up the APIs for communicating with the cloud part of the service?
- v4dok 4y agoClose. The first part is fair. A more real-life example. I am a pharma company and I want to execute a query on some hospital data. The hospital doesn't want to give me the data in plaintext but they are fine with me getting some aggregate insights from their data that are not PII. Now lets assume I decide to do that using FHE. I can now compute my query on the encrypted hospital data and I never see the plaintext data. What do we "win" in this scenario? We can do this computation wherever we want because no matter where the computation is done, the data will be encrypted, so no risk for the infra provider to see that data. What we don't "automatically win" in this scenario? 1. Guarantees that indeed I am running an SQL query on that data and not something else along with it -> That is only possible to guarantee if the FHE software is properly audited (same with any software tbh, but easier with FHE and similar techs because of the integrity guarantees due to encryption). 2. Guarantees that the SQL query I made will not leak patient data in the end (through linking additional data, or diff attacks) (same with any other SQL query) People who are deep into these technologies will say "yes of course" thats not an FHE problem. And that is true. But every FHE vendor I've seen blur that difference by not specifying what kind of attacks they protect against when they talk about "protecting privacy". Heck, most of them they don't even talk about the attestation process and how their clients can make sure that they can trust the software running in encrypted form. Yes, these hold true for all software, but the point (for me) of encryption in-use is to make sure we hold software to a higher trust standard than today, not just replace a trusted party with another one.
- bawolff 4y agoWhat i mean, is the only person who can learn anything about the data, is the entity who posesses the decryption key. In any sane deployment of FHE the key holder is the person who owns the data, not the app developer and not the person "running" the program.