5 ms·
So, you're training a statistical model - teaching it to recognize certain patterns - on data that is somehow wholly without patterns? Even with a certain amoun
by falcolas 7y ago
So, you're training a statistical model - teaching it to recognize certain patterns - on data that is somehow wholly without patterns? Even with a certain amount of noise in the individual data points, if you're given enough data to train a statistical model to identify traits in the ciphertext you also probably have enough data to break the encryption.
Also, something can be statistically random and still have patterns (see PRNGs, which are statically random (you can't identify the next value from previous values), but there's still a pattern if you know the algorithm and seed).
I'll admit, the promise of homomorphic encryption is pretty amazing, but this particular combination of data and ML seems like a fairly obvious way to leak data. I believe there's a reason that homomorphic encryption has not been broadly accepted as an allowed standard.
EDIT: So, I think I'm missing a practical example. I have a model which I want to train on data homomorphically encrypted with key X, and that model is a very simple "is this a cat". I'm given a whole set of data encrypted with key X that's tagged with "cat" and "not a cat".
Once the model is trained, I can run this on any data encrypted with key X and find out if the data contains a cat (with some degree of accuracy). I have no way of telling information outside the tags provided on the training data, but it still gives me, a person without the encryption key, the ability to identify any feature that's tagged in the training set on any un-tagged production set.
Having seen a large quantity of ML training sets, the tag sets are rarely so limited. There's also often "elephant", "ball", and "dog" tags, even if I'm only being asked to train on cats.
- thefifthsetpin 7y ago> homomorphic encryption has not been broadly accepted as an allowed standard. I get that a lack of broad acceptance is a hard thing to source, but do you have one?
- falcolas 7y agohttps://csrc.nist.gov/publications/detail/sp/800-111/final https://csrc.nist.gov/publications/detail/sp/800-111/final is a good starting point (and used by HIPPA, as an aside). It leads to block cipher standards, AES standards, and so on. I think it's not unreasonable to say that if it should ever appear as a recommended cipher by NIST, then it can be considered to be broadly accepted.
- madrafi 7y agoLast year a group of companies and institutions created a working standard. https://homomorphicencryption.org/standard/ https://homomorphicencryption.org/standard/
- KenoFischer 7y ago> Once the model is trained, I can run this on any data encrypted with key X and find out if the data contains a cat (with some degree of accuracy). I have no way of telling information outside the tags provided on the training data, but it still gives me, a person without the encryption key, the ability to identify any feature that's tagged in the training set on any un-tagged production set. I think you're missing the fact that the predictions come back encrypted, so you don't learn anything unless you know the key. Also, in this particular example the model was trained on unencrypted data, but as discussed below, you can do either.
- falcolas 7y agoPredictions are a tag, and a separate confidence value. A fairly finite set. Cat. Not a Cat. Large quantities of small finite sets are anathema to encryption.
- KenoFischer 7y agoI'm still only guessing at the objection, but if it helps, the encryptions are randomized, so two different encryptions of the same value do not have the same ciphertext.
- heavenlyblue 7y agoNot all encryption algorithms are susceptible to known-cyphertext attacks.
- heavenlyblue 7y agoIf you map a binary variable into a, let's say, 128-bit number (non-linearly) where a mapping is unique for every encryption key used, there's nothing making it an anathema for encryption. I am not sure whether this is achievable practically though.
- ReidZB 7y ago> I believe there's a reason that homomorphic encryption has not been broadly accepted as an allowed standard. Well, yes... Practical homomorphic encryption is cutting-edge research, and standards bodies like NIST aren't going to deal with an area like this until it's much closer to "solved" (by which I mean much more efficient, with more practical applications, widely used and scrutinized schemes, etc.)