10 ms·

Simple Encrypted Arithmetic Library goes open source

215 points by dsr12 8y ago

gadders 8y ago
In case anyone else wondered: https://en.wikipedia.org/wiki/Homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption "Homomorphic encryption is a form of encryption that allows computation on ciphertexts, generating an encrypted result which, when decrypted, matches the result of the operations as if they had been performed on the plaintext."
newfocogi 8y ago
I am not an encryption expert, but to the casual techy follower, homomorphic encryption seems like it could be huge for the union of machine learning and digital privacy. Are there any experts on HN who could comment on how mature this technology is? Or does anyone know of any interesting projects that use this? I know Numerai is using something of this sort, but it is hard for me to tell how legitimate it is.
- rtkwe 8y ago
  Another big advantage I'd hope to see is it enabling more interesting volunteer computing efforts or maybe a truly distributed web infrastructure.
- osaariki 8y ago
  I work at Microsoft Research with the SEAL team on applications for Fully Homomorphic Encryption (FHE). Batched integer FHE schemes are great for machine learning. The regular data access patterns in neural networks make using the batching easy and the rather "wide" computations lead to good encryption parameters. Also a new scheme, CKKS, gives the ability to divide by certain plaintext constants, which helps control scaling in fixed point representations. This comes in exchange for results being approximate, which is not a problem in machine learning. Currently what FHE needs is better toolchain support. Selecting encodings and encryption parameters is hard, but a lot can be done with good language support and tooling. We're presenting some of our work on automatically selecting good data layouts for NNs on FHE at the PPML workshop in NIPS next weekend [1]. [1] https://ppml-workshop.github.io/ppml/ https://ppml-workshop.github.io/ppml/
- dboreham 8y ago
  One issue to be aware of : it is typically very (very) slow.
- garganzol 8y ago
  Homomorphic encryption (full and partial) opens the road to some unforeseen forms of obfuscation. This basically means that we'll see the status quo change in this area quite soon. While the full homomorphic encryption schemes tend to be at the slower side for now, partially homomorphic systems are fast.
ainiriand 8y ago
Never roll out your own crypto.
- dpark 8y ago
  When people say "never roll your own crypto", they mean you, the amateur who doesn't have the experience, expertise, or scale to get it right. They don't mean that literally no one should ever write crypto code.
  - WorldMaker 8y ago
    Yes, if there are experts today on homomorphic encryption, one would assume that Microsoft's department of Cryptographic Research plausibly includes some (if not all) of them. Their publications are online, if one wants to see what peer reviewed works they've published: https://www.microsoft.com/en-us/research/group/cryptography-research/ https://www.microsoft.com/en-us/research/group/cryptography-...
- Bhilai 8y ago
  One would assume that "Cryptography Research Group" at Microsoft would include some word class cryptographers and I would think its okay for Cryptographers to roll their own Crypto or there wont be any progress. Their work is open source so their peers in industry and academia can actually review.
  - paulie_a 8y ago
    Not that I distrust this particular code, Microsoft has had some serious blunders in the past. Like chopping off any password over 8 characters.
- ainiriand 8y ago
  I always forget that I should not use sarcasm on HN. Apologies.
ancarda 8y ago
I know very little about C, C++, or Rust, so I want to ask this to understand why C++ was picked for this. My limited understanding is that people were tolerating C++ libraries like OpenSSL, but that new crypto or security code would likely be written in Rust for safety. The blog doesn't say much (to me) about the choice: In addition to having no external dependencies, Microsoft SEAL is written in standard C++, making it easy to compile in many different environments. Is Rust unsuitable for this project? If anyone here worked on SEAL, I'd love to learn more about this -- did the SEAL team consider using Rust?
- wrs 8y ago
  Rust is great, but its adoption is minuscule compared to C++. Given the point of this release is to encourage wide scale adoption of certain algorithms, C++ is the realistic choice at this time.
- dpark 8y ago
  > My limited understanding is that ... that new crypto or security code would likely be written in Rust for safety. Where did you get that impression? The creators of Rust would undoubtedly love that to be the case, but Rust is still essentially in its infancy. To my knowledge there are almost no major projects released that have been built in Rust. It's ranked below such heavy hitters as Ada and Prolog for popularity on the TIOBE index[1]. It's pretty unlikely that Rust will become the standard language for everything crypo- or security-related anytime soon (or ever, but it could happen). [1] https://www.tiobe.com/tiobe-index/ https://www.tiobe.com/tiobe-index/
  - steveklabnik 8y ago
    > there are almost no major projects released that have been built in Rust. https://news.ycombinator.com/item?id=18545373 https://news.ycombinator.com/item?id=18545373 This includes Microsoft, which I forgot in that post, who uses it in production for their IOT product, and as part of Visual Studio Code. That said, for crypto, there are some less than ideal things. That’s not stopping projects from working on it.
wwarner 8y ago
Helpful! This kind of encryption is required to make relational data secure. https://arxiv.org/pdf/1512.03498.pdf https://arxiv.org/pdf/1512.03498.pdf
craftyguy 8y ago
Perhaps someone can explain why this can be trusted? microsoft has a long history of cooperating with governments across a broad range of oppressiveness (e.g. US, China, etc). I wasn't able to quickly/easily find any peer reviews of this implementation. Without any more information, how do we know this isn't backdoored in some way?
- pwaivers 8y ago
  I believe releasing it to open source makes the code more trustworthy. Now, anyone can go in and verify how it works. Any backdoor would be visible in the code.
  - craftyguy 8y ago
    > I believe releasing it to open source makes the code more trustworthy I don't, and I think it's dangerous to assume that. Many open source projects have systems in place for publicly reviewing and approving code, but microsoft does not have a history of conducting public code reviews for their projects. I would never assume that publicly viewable code is being reviewed by others who know what they are doing if it's not obvious that they are. > Now, anyone can go in and verify how it works. Any backdoor would be visible in the code I would only trust those with a high degree of mathematical knowledge, specifically around the subject of cryptography to be able review the code in any meaningful way. The rest of us can verify that it builds successfully and, well, that's about it.
    - jf- 8y ago
      Be that as it may, if you’re trying to hide something releasing it as open source is a pretty terrible way to do it.
      - craftyguy 8y ago
        Not if it's highly unlikely anyone with enough knowledge to know otherwise is going to review it. The upside is that you get folks like the one above who automatically assume open source software is safe, and use it.
polskibus 8y ago
What are the performance costs of using such library vs raw data operations?
- garganzol 8y ago
  To give you an estimate: an AES encryption operation takes about 2 seconds to compute in homomorphic encrypted domain on a generic machine. Non-encrypted AES operation is approximately 10.000.000 times faster.
  - polskibus 8y ago
    I understand that initial encryption costs a lot, that can be one-off though. My question was more about the computation on data - whether the same computation (using this library?) on homomorphically encrypted data is performing as well as on unencrypted data? Is there a penalty to pay on every operation after initial encryption?
tuxxy 8y ago
To give some context, this is moving away from Microsoft's "open source" license that basically requires you to give Microsoft access to whatever you make with their libraries. Moving to MIT is great for this. As someone working on FHE, we'll finally be able to see some interesting applications with this library other than Microsoft's research. Great job to the SEAL team!
- umvi 8y ago
  What does FHE stand for? All my searches are leading to religious articles about spending time with your family
  - CiPHPerCoder 8y ago
    Fully Homomorphic Encryption
  - emadelwany 8y ago
    I believe they're referring to Fully Homomorphic Encryption [1] [1] https://en.wikipedia.org/wiki/Homomorphic_encryption#Fully_homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption#Fully_h...
ocdtrekkie 8y ago
Isn't the "SEAL" acronym a bit overloaded in the realm of cryptography? https://en.wikipedia.org/wiki/SEAL_(cipher) https://en.wikipedia.org/wiki/SEAL_(cipher)
- marshray 8y ago
  I'd never heard of it, a web search turns up only a handful of mentions, and it's not even the only thing in crypto to be called 'seal'.
  - ocdtrekkie 8y ago
    I figured it wasn't the only one. People who have gone through Cisco material have an outsize exposure to this particular flavor of SEAL because it's one of the few presented alternatives to AES in their material: https://security.stackexchange.com/questions/141879/cisco-says-software-optimized-encryption-algorithm-seal-is-more-secure-than-ae https://security.stackexchange.com/questions/141879/cisco-sa...
- mtgx 8y ago
  Now they can say this is not your father's "military-grade encryption". This is SEAL-grade encryption!
patwolf 8y ago
I recall several years ago hearing about other companies, particularly IBM, patenting technologies related to homomorphic encryption. I wonder if there is any risk to some of this infringing on a patent and what that means for users of OSS when that happens.
CephalopodMD 8y ago
Has anyone tried to use this to create a distributed cloud? If storage and certain types of computation could be encrypted, anyone could sell their spare computation power without trust issues.
- UncleMeat 8y ago
  Somewhat Homomorphic Encryption unfortunately only works for a pretty limited set of workloads and FHE is still mind bogglingly slow so a general purpose cloud for arbitrary computation isn't going to be a thing any time soon.
  - CiPHPerCoder 8y ago
    Also, by itself, homomorphic encryption doesn't protect against chosen-ciphertext attacks, which makes it very unattractive for 99.9% of real world crypto use-cases. https://tonyarcieri.com/all-the-crypto-code-youve-ever-written-is-probably-broken https://tonyarcieri.com/all-the-crypto-code-youve-ever-writt... In my experience, a lot of teams also reach for FHE to solve problems that are easily and securely solved without it (i.e. searchable encryption): https://github.com/paragonie/ciphersweet https://github.com/paragonie/ciphersweet That being said, there is the 0.01% problem space where FHE makes perfect sense (mostly in the realm of encrypted databases and machine learning). For those systems, if the database can ever be considered adversarial (your threat model may rule this out, most applications' do not), a simple construction involving HMAC, a secure digital signature algorithm, and an append-only ledger can protect the application that reads from the database against chosen-ciphertext attacks. https://paragonie.com/blog/2017/12/assuring-ciphertext-integrity-for-homomorphic-cryptosystems https://paragonie.com/blog/2017/12/assuring-ciphertext-integ...
    - Ar-Curunir 8y ago
      Why does lack of CCA security make FHE unattractive? The entire point of FHE is to allow malleability of ciphertexts.
      - 8y ago
tonetheman 8y ago
Not simple. But super cool stuff

215 points by dsr12 8y ago

In case anyone else wondered: https://en.wikipedia.org/wiki/Homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption "Homomorphic encryption is a form of encryption that allows computation on ciphertexts, generating an encrypted result which, when decrypted, matches the result of the operations as if they had been performed on the plaintext."

newfocogi 8y ago

I am not an encryption expert, but to the casual techy follower, homomorphic encryption seems like it could be huge for the union of machine learning and digital privacy. Are there any experts on HN who could comment on how mature this technology is? Or does anyone know of any interesting projects that use this? I know Numerai is using something of this sort, but it is hard for me to tell how legitimate it is.

rtkwe 8y ago
Another big advantage I'd hope to see is it enabling more interesting volunteer computing efforts or maybe a truly distributed web infrastructure.
osaariki 8y ago
I work at Microsoft Research with the SEAL team on applications for Fully Homomorphic Encryption (FHE). Batched integer FHE schemes are great for machine learning. The regular data access patterns in neural networks make using the batching easy and the rather "wide" computations lead to good encryption parameters. Also a new scheme, CKKS, gives the ability to divide by certain plaintext constants, which helps control scaling in fixed point representations. This comes in exchange for results being approximate, which is not a problem in machine learning. Currently what FHE needs is better toolchain support. Selecting encodings and encryption parameters is hard, but a lot can be done with good language support and tooling. We're presenting some of our work on automatically selecting good data layouts for NNs on FHE at the PPML workshop in NIPS next weekend [1]. [1] https://ppml-workshop.github.io/ppml/ https://ppml-workshop.github.io/ppml/
dboreham 8y ago
One issue to be aware of : it is typically very (very) slow.
garganzol 8y ago
Homomorphic encryption (full and partial) opens the road to some unforeseen forms of obfuscation. This basically means that we'll see the status quo change in this area quite soon. While the full homomorphic encryption schemes tend to be at the slower side for now, partially homomorphic systems are fast.

ainiriand 8y ago

Never roll out your own crypto.

dpark 8y ago
When people say "never roll your own crypto", they mean you, the amateur who doesn't have the experience, expertise, or scale to get it right. They don't mean that literally no one should ever write crypto code.
- WorldMaker 8y ago
  Yes, if there are experts today on homomorphic encryption, one would assume that Microsoft's department of Cryptographic Research plausibly includes some (if not all) of them. Their publications are online, if one wants to see what peer reviewed works they've published: https://www.microsoft.com/en-us/research/group/cryptography-research/ https://www.microsoft.com/en-us/research/group/cryptography-...
Bhilai 8y ago
One would assume that "Cryptography Research Group" at Microsoft would include some word class cryptographers and I would think its okay for Cryptographers to roll their own Crypto or there wont be any progress. Their work is open source so their peers in industry and academia can actually review.
- paulie_a 8y ago
  Not that I distrust this particular code, Microsoft has had some serious blunders in the past. Like chopping off any password over 8 characters.
ainiriand 8y ago
I always forget that I should not use sarcasm on HN. Apologies.

ancarda 8y ago

I know very little about C, C++, or Rust, so I want to ask this to understand why C++ was picked for this. My limited understanding is that people were tolerating C++ libraries like OpenSSL, but that new crypto or security code would likely be written in Rust for safety. The blog doesn't say much (to me) about the choice: In addition to having no external dependencies, Microsoft SEAL is written in standard C++, making it easy to compile in many different environments. Is Rust unsuitable for this project? If anyone here worked on SEAL, I'd love to learn more about this -- did the SEAL team consider using Rust?

wrs 8y ago
Rust is great, but its adoption is minuscule compared to C++. Given the point of this release is to encourage wide scale adoption of certain algorithms, C++ is the realistic choice at this time.
dpark 8y ago
> My limited understanding is that ... that new crypto or security code would likely be written in Rust for safety. Where did you get that impression? The creators of Rust would undoubtedly love that to be the case, but Rust is still essentially in its infancy. To my knowledge there are almost no major projects released that have been built in Rust. It's ranked below such heavy hitters as Ada and Prolog for popularity on the TIOBE index[1]. It's pretty unlikely that Rust will become the standard language for everything crypo- or security-related anytime soon (or ever, but it could happen). [1] https://www.tiobe.com/tiobe-index/ https://www.tiobe.com/tiobe-index/
- steveklabnik 8y ago
  > there are almost no major projects released that have been built in Rust. https://news.ycombinator.com/item?id=18545373 https://news.ycombinator.com/item?id=18545373 This includes Microsoft, which I forgot in that post, who uses it in production for their IOT product, and as part of Visual Studio Code. That said, for crypto, there are some less than ideal things. That’s not stopping projects from working on it.

wwarner 8y ago

Helpful! This kind of encryption is required to make relational data secure. https://arxiv.org/pdf/1512.03498.pdf https://arxiv.org/pdf/1512.03498.pdf

craftyguy 8y ago

Perhaps someone can explain why this can be trusted? microsoft has a long history of cooperating with governments across a broad range of oppressiveness (e.g. US, China, etc). I wasn't able to quickly/easily find any peer reviews of this implementation. Without any more information, how do we know this isn't backdoored in some way?

pwaivers 8y ago
I believe releasing it to open source makes the code more trustworthy. Now, anyone can go in and verify how it works. Any backdoor would be visible in the code.
- craftyguy 8y ago
  > I believe releasing it to open source makes the code more trustworthy I don't, and I think it's dangerous to assume that. Many open source projects have systems in place for publicly reviewing and approving code, but microsoft does not have a history of conducting public code reviews for their projects. I would never assume that publicly viewable code is being reviewed by others who know what they are doing if it's not obvious that they are. > Now, anyone can go in and verify how it works. Any backdoor would be visible in the code I would only trust those with a high degree of mathematical knowledge, specifically around the subject of cryptography to be able review the code in any meaningful way. The rest of us can verify that it builds successfully and, well, that's about it.
  - jf- 8y ago
    Be that as it may, if you’re trying to hide something releasing it as open source is a pretty terrible way to do it.
    - craftyguy 8y ago
      Not if it's highly unlikely anyone with enough knowledge to know otherwise is going to review it. The upside is that you get folks like the one above who automatically assume open source software is safe, and use it.

polskibus 8y ago

What are the performance costs of using such library vs raw data operations?

garganzol 8y ago
To give you an estimate: an AES encryption operation takes about 2 seconds to compute in homomorphic encrypted domain on a generic machine. Non-encrypted AES operation is approximately 10.000.000 times faster.
- polskibus 8y ago
  I understand that initial encryption costs a lot, that can be one-off though. My question was more about the computation on data - whether the same computation (using this library?) on homomorphically encrypted data is performing as well as on unencrypted data? Is there a penalty to pay on every operation after initial encryption?

tuxxy 8y ago

To give some context, this is moving away from Microsoft's "open source" license that basically requires you to give Microsoft access to whatever you make with their libraries. Moving to MIT is great for this. As someone working on FHE, we'll finally be able to see some interesting applications with this library other than Microsoft's research. Great job to the SEAL team!

umvi 8y ago
What does FHE stand for? All my searches are leading to religious articles about spending time with your family
- CiPHPerCoder 8y ago
  Fully Homomorphic Encryption
- emadelwany 8y ago
  I believe they're referring to Fully Homomorphic Encryption [1] [1] https://en.wikipedia.org/wiki/Homomorphic_encryption#Fully_homomorphic_encryption https://en.wikipedia.org/wiki/Homomorphic_encryption#Fully_h...

ocdtrekkie 8y ago

Isn't the "SEAL" acronym a bit overloaded in the realm of cryptography? https://en.wikipedia.org/wiki/SEAL_(cipher) https://en.wikipedia.org/wiki/SEAL_(cipher)

marshray 8y ago
I'd never heard of it, a web search turns up only a handful of mentions, and it's not even the only thing in crypto to be called 'seal'.
- ocdtrekkie 8y ago
  I figured it wasn't the only one. People who have gone through Cisco material have an outsize exposure to this particular flavor of SEAL because it's one of the few presented alternatives to AES in their material: https://security.stackexchange.com/questions/141879/cisco-says-software-optimized-encryption-algorithm-seal-is-more-secure-than-ae https://security.stackexchange.com/questions/141879/cisco-sa...
mtgx 8y ago
Now they can say this is not your father's "military-grade encryption". This is SEAL-grade encryption!

patwolf 8y ago

I recall several years ago hearing about other companies, particularly IBM, patenting technologies related to homomorphic encryption. I wonder if there is any risk to some of this infringing on a patent and what that means for users of OSS when that happens.

CephalopodMD 8y ago

Has anyone tried to use this to create a distributed cloud? If storage and certain types of computation could be encrypted, anyone could sell their spare computation power without trust issues.

UncleMeat 8y ago
Somewhat Homomorphic Encryption unfortunately only works for a pretty limited set of workloads and FHE is still mind bogglingly slow so a general purpose cloud for arbitrary computation isn't going to be a thing any time soon.
- CiPHPerCoder 8y ago
  Also, by itself, homomorphic encryption doesn't protect against chosen-ciphertext attacks, which makes it very unattractive for 99.9% of real world crypto use-cases. https://tonyarcieri.com/all-the-crypto-code-youve-ever-written-is-probably-broken https://tonyarcieri.com/all-the-crypto-code-youve-ever-writt... In my experience, a lot of teams also reach for FHE to solve problems that are easily and securely solved without it (i.e. searchable encryption): https://github.com/paragonie/ciphersweet https://github.com/paragonie/ciphersweet That being said, there is the 0.01% problem space where FHE makes perfect sense (mostly in the realm of encrypted databases and machine learning). For those systems, if the database can ever be considered adversarial (your threat model may rule this out, most applications' do not), a simple construction involving HMAC, a secure digital signature algorithm, and an append-only ledger can protect the application that reads from the database against chosen-ciphertext attacks. https://paragonie.com/blog/2017/12/assuring-ciphertext-integrity-for-homomorphic-cryptosystems https://paragonie.com/blog/2017/12/assuring-ciphertext-integ...
  - Ar-Curunir 8y ago
    Why does lack of CCA security make FHE unattractive? The entire point of FHE is to allow malleability of ciphertexts.
    - 8y ago

tonetheman 8y ago

Not simple. But super cool stuff