7 ms·

Will this be useful for machine learning in the same way as this ? https://medium.com/numerai/encrypted-data-for-efficient-markets-fffbe9743ba8 https://medium.

Will this be useful for machine learning in the same way as this ? https://medium.com/numerai/encrypted-data-for-efficient-markets-fffbe9743ba8 https://medium.com/numerai/encrypted-data-for-efficient-mark...

amenghra 9y ago
This is very interesting from an academic/theory point of view. There currently aren't a lot practical use cases where we can afford a performance loss of ~100,000,000x (your homomorphic crypto algorithm is going to run on the order of ~Hz on a ~Ghz CPU).
- tgtweak 9y ago
  There are applications for it, despite the speed penalty.
  - dsacco 9y ago
    What are those applications?
    - tgtweak 9y ago
      Any application where you need an untrusted third party to do operations on encrypted data, without them getting access to the data itself.
      - dsacco 9y ago
        Right but that's nearly tautological...I meant specific applications that can tolerate the massive performance sacrifice.
proofofstake 9y ago
Yes, but not to the same degree. Numerai uses structure-preserving encryption / neural encryption. This allows people to use any existing machine learning algorithm on the data. For fully homomorphic encryption you would need specialized algorithms. These are way more difficult to design. They also run slower.
- lkowalcz 9y ago
  Worth pointing out that Numerai actually doesn't use encryption in any standard sense (including structure-preserving encryption), but instead seems to be using some heuristic method of obfuscating their data. Their (closed-source) method of obfuscating their data apparently does have the property that it preserves the structure of the data, but calling it "structure-preserving encryption" is misleading imo since it risks confusing it with standard notions of encryption and structure-preserving encryption which have much stronger security guarantees. (Their marketing seems to encourage this conflation by, for example, citing academic advances in standard notions of homomorphic encryption and SPE and implying that these advances have enabled Numerai's technology) https://medium.com/numerai/encrypted-data-for-efficient-markets-fffbe9743ba8 https://medium.com/numerai/encrypted-data-for-efficient-mark...
  - sandGorgon 9y ago
    Interesting. Any insight into what specific algorithms these are ? I would love to play around with my own data this way. It wouldn't be a simple hash or something, would it ?
    - lkowalcz 9y ago
      I suspect they might just be anonymizing their features (removing any labels), then normalizing them to [0,1].
  - proofofstake 9y ago
    They switched from structure-preserving encryption to neural encryption (using a neural net's layer activations). > Just a few months ago this package was released by Louis Aslett at Oxford http://www.louisaslett.com/HomomorphicEncryption/ http://www.louisaslett.com/HomomorphicEncryption/. Louis helped me use his package to do Fan and Vercauteren homomorphic encryption on my dataset. Because the ciphertexts are polynomials it's not too easy for an average data scientist to use the data. That's why I came up with more chill ways of encrypting Numerai's data that the article mentions like order-preserving encryption. There's a security vs easy of use trade off, for sure. But homomorphic encryption is a real thing. https://www.reddit.com/r/MachineLearning/comments/3zvuge/encrypted_data_for_efficient_markets_an_mnist_for/cyprq84/ https://www.reddit.com/r/MachineLearning/comments/3zvuge/enc... Louis Aslett authored https://arxiv.org/abs/1508.06574 https://arxiv.org/abs/1508.06574
maffydub 9y ago
Open Mined (http://openmined.org/ http://openmined.org/) are looking at this from the other angle - sharing the encrypted neural network with their users so that they can train it without sharing their data at all - the encrypted gradients are computed by the user and collated and decrypted by the company that wants to train the neural network. (Not affiliated in any way, but went to a really interesting talk on this a few weeks ago as part of the London Machine Learning meetup group - https://www.meetup.com/London-Machine-Learning-Meetup/events/241613150/. https://www.meetup.com/London-Machine-Learning-Meetup/events...)
- sandGorgon 9y ago
  oh - its very similar to numerai. Pretty cool. I think this is their library - https://github.com/OpenMined/R-Homomorphic-Encryption-Package https://github.com/OpenMined/R-Homomorphic-Encryption-Packag... (they havent ported this to python, so are having to use R-bridge to use it in jupyter. I wonder why).

amenghra 9y ago

This is very interesting from an academic/theory point of view. There currently aren't a lot practical use cases where we can afford a performance loss of ~100,000,000x (your homomorphic crypto algorithm is going to run on the order of ~Hz on a ~Ghz CPU).

tgtweak 9y ago
There are applications for it, despite the speed penalty.
- dsacco 9y ago
  What are those applications?
  - tgtweak 9y ago
    Any application where you need an untrusted third party to do operations on encrypted data, without them getting access to the data itself.
    - dsacco 9y ago
      Right but that's nearly tautological...I meant specific applications that can tolerate the massive performance sacrifice.

proofofstake 9y ago

Yes, but not to the same degree. Numerai uses structure-preserving encryption / neural encryption. This allows people to use any existing machine learning algorithm on the data. For fully homomorphic encryption you would need specialized algorithms. These are way more difficult to design. They also run slower.

lkowalcz 9y ago
Worth pointing out that Numerai actually doesn't use encryption in any standard sense (including structure-preserving encryption), but instead seems to be using some heuristic method of obfuscating their data. Their (closed-source) method of obfuscating their data apparently does have the property that it preserves the structure of the data, but calling it "structure-preserving encryption" is misleading imo since it risks confusing it with standard notions of encryption and structure-preserving encryption which have much stronger security guarantees. (Their marketing seems to encourage this conflation by, for example, citing academic advances in standard notions of homomorphic encryption and SPE and implying that these advances have enabled Numerai's technology) https://medium.com/numerai/encrypted-data-for-efficient-markets-fffbe9743ba8 https://medium.com/numerai/encrypted-data-for-efficient-mark...
- sandGorgon 9y ago
  Interesting. Any insight into what specific algorithms these are ? I would love to play around with my own data this way. It wouldn't be a simple hash or something, would it ?
  - lkowalcz 9y ago
    I suspect they might just be anonymizing their features (removing any labels), then normalizing them to [0,1].
- proofofstake 9y ago
  They switched from structure-preserving encryption to neural encryption (using a neural net's layer activations). > Just a few months ago this package was released by Louis Aslett at Oxford http://www.louisaslett.com/HomomorphicEncryption/ http://www.louisaslett.com/HomomorphicEncryption/. Louis helped me use his package to do Fan and Vercauteren homomorphic encryption on my dataset. Because the ciphertexts are polynomials it's not too easy for an average data scientist to use the data. That's why I came up with more chill ways of encrypting Numerai's data that the article mentions like order-preserving encryption. There's a security vs easy of use trade off, for sure. But homomorphic encryption is a real thing. https://www.reddit.com/r/MachineLearning/comments/3zvuge/encrypted_data_for_efficient_markets_an_mnist_for/cyprq84/ https://www.reddit.com/r/MachineLearning/comments/3zvuge/enc... Louis Aslett authored https://arxiv.org/abs/1508.06574 https://arxiv.org/abs/1508.06574

maffydub 9y ago

Open Mined (http://openmined.org/ http://openmined.org/) are looking at this from the other angle - sharing the encrypted neural network with their users so that they can train it without sharing their data at all - the encrypted gradients are computed by the user and collated and decrypted by the company that wants to train the neural network. (Not affiliated in any way, but went to a really interesting talk on this a few weeks ago as part of the London Machine Learning meetup group - https://www.meetup.com/London-Machine-Learning-Meetup/events/241613150/. https://www.meetup.com/London-Machine-Learning-Meetup/events...)

sandGorgon 9y ago
oh - its very similar to numerai. Pretty cool. I think this is their library - https://github.com/OpenMined/R-Homomorphic-Encryption-Package https://github.com/OpenMined/R-Homomorphic-Encryption-Packag... (they havent ported this to python, so are having to use R-bridge to use it in jupyter. I wonder why).