6 ms·

The distinction between cryptographic issues and coding-level issues seems valuable. I think "You Shouldn't Roll Your Own Crypto" is really three separate piec

The distinction between cryptographic issues and coding-level issues seems valuable. I think "You Shouldn't Roll Your Own Crypto" is really three separate pieces of advice, each of which deserves justifying independently. Roughly: - You shouldn't try to invent your own cryptographic primitives. - You shouldn't try to design your own equivalent to (say) SSL. - You shouldn't try to implement something like SSL, or parse ASN.1, yourself. I don't think many people are likely to fall foul of the first. It seems to me the second is where the real devils lurk: dealing with padding and cipher modes and all that kind of stuff. But it's not at all clear to me that the third has been historically justified. I think if someone in the early 2000s had thought "I don't like the look of OpenSSL and I don't think C is a good choice of language. I'm going to write my own implementation of the parts I need in (say) Ada" they might well not have regretted that choice.

unilynx 5y ago
Given the many historic openssl bugs related to certificate parsing (ie x509 and asn1) I’m not sure you should even trust expert crypto programmers to deal with that
- cormacrelf 5y ago
  Some folks are trying to kill ASN.1. It is probably an awkward moment for that, when people are looking at binary encodings more because JSON is boring and doesn't stand out on a CV. It's got all the hallmarks of old tech that looks like it's survived on its merits, but it really isn't like that. > "ASN.1 is a serialization format which enables C programmers to turn binary data into remote code execution vulnerabilities" -- https://twitter.com/bascule/status/1086778481864658945 https://twitter.com/bascule/status/1086778481864658945 Also https://twitter.com/TwitchiH/status/1382649508371308547 https://twitter.com/TwitchiH/status/1382649508371308547, discussing it in context of the San Bernadino iPhone being broken with an ASN.1 vuln.
  - lifthrasiir 5y ago
    The difference between ASN.1 (BER/DER) and modern binary encodings is that people have tried to parse the former by hand but not the latter (at least to my experience). BER/DER itself is a fine encoding---sure, it has a legacy including six string types but we can deal with them---and in principle a simple library with bound checks should have been more than enough. ASN.1 is however structured in a way that such a simple library looks like incomplete; it looks like that a correct way to parse BER/DER is an ASN.1 compiler, which tends to be bulky and annoying to integrate. As a result for a long time most ASN.1 implementations were either a "complete" ASN.1 compiler or a hand-crafted parser, with no middle ground of a library that is simple enough and prevents you from memory unsafety.
  - memling 5y ago
    > Some folks are trying to kill ASN.1. It is probably an awkward moment for that, when people are looking at binary encodings more because JSON is boring and doesn't stand out on a CV. It's got all the hallmarks of old tech that looks like it's survived on its merits, but it really isn't like that. I think it's rather more than this. ASN.1 is deeply embedded in daily life: 3G/4G/LTE/5G, V2X, and many other protocols are specified in ASN.1. The inertia has as much to do with backwards compatibility as anything else. If you need to read call data records, you have to parse BER; if you want to deal with vehicle to vehicle communications you have to parse OER. 3G protocols are generally PER. Crypto is DER. Are there alternatives that are better in some respects? Sure. Are they likely to supplant existing telecommunications standards? No. Should move away from ASN.1 for crypto purposes? Probably, though I'm not experienced enough to have a strong opinion. I'd be interested to see whether we could install a new standard in a suitable time frame, and indeed what form that would take. (In the two links you cited, I didn't see anything, but I'm actually really interested!) I think ASN.1 is probably here to stay for a long time, and not merely as a resume polishing agent.
  - woodruffw 5y ago
    > because JSON is boring and doesn't stand out on a CV. It's a little bit more than that: cryptographic container formats usually need to be stable on traversal/canonicalized so that things like hashes remain consistent. JSON alone doesn't have a canonicalized format (there are various suggested canonical forms), and leaves details about key iteration, duplication, etc. ambiguous in ways that are potential sources of vulnerability (what happens if a particular protocol allows users to introduce untrusted attributes to a JSON object and doesn't check for name clashes?). Edit: not to mention integer width/representation of large integers! That's a big one for cryptographic protocols.
    - memling 5y ago
      > It's a little bit more than that: cryptographic container formats usually need to be stable on traversal/canonicalized so that things like hashes remain consistent. Yeah, this, too: there's got to be a single way of representing the same message. (BER, in contrast to DER, has multiple means of writing the same message. BOOLEAN types, for example, indicate FALSE by 0x00, but TRUE is any non-zero value. I forget how DER resolves this. DER disallows the use of indefinite lengths, while BER admits both definite and indefinite lengths. Etc., etc.) I'm sympathetic to the challenges people face with ASN.1, having worked with it rather extensively in my career. I don't recommend it to people who have serialization challenges usually because something like protobuf or flatbuffers seems like a more reasonable choice in this day and age...but at the same time ASN.1 has a history of mistakes and innovations for solving some challenging problems in representation and encoding that should inform decision-making moving forward.
  - jnwatson 5y ago
    ASN.1's primary problem is that it has been around a long time. Long before folks understood the security implications of a buffer overflow, we've been parsing ASN.1. Most of the vulnerabilities stem from the basic problem of trusting length fields. Any binary encoding is going to have similar issues.
ThePhysicist 5y ago
Often you don't have a choice but to write your own parsers for serialization formats, as inter-language & inter-tool interoperability is still not very good, except maybe for TLS-related stuff. There's e.g. no predefined way to verify an ECDSA signature created in JS using Golang (or vice versa), so you have to parse the R & S values by hand. Luckily they are just concatenated big-endian numbers.
- lu4p 5y ago
  I'm pretty sure there is https://pkg.go.dev/crypto/ecdsa#Verify https://pkg.go.dev/crypto/ecdsa#Verify
  - ThePhysicist 5y ago
    That's exactly what I referred to: The function takes two BigInt's, which you have to unpack yourself from the byte-serialization created by JS. There's no JS-compatible ECDH key derivation function in Golang either, so you have to implement the multiplication yourself (which again isn't difficult).
    - dcow 5y ago
      My impression is that crypto people, being mildly allergic to asn.1, decided not to use it to specify standard serialization formats in our new world of EC crypto. So here we are without asn.1 and without a replacement. I generally think if you’re going to remove something because you think it sucks it’s on you to replace it with an equivalent or clearly justify why it was vestigial if you don't have one. In this case you really don't need a DER calibre encoding either, you just need a standard in the first place. It’s just that the standard would historically likely use asn.1. But that doesn't have to be the case. Maybe the closest thing is the ECDSA point form specified in https://www.secg.org/sec1-v2.pdf#page16 https://www.secg.org/sec1-v2.pdf#page16. The the curve domain parameters in sec2 are, you guessed it, specified in asn.1.
      - johnisgood 5y ago
janeroe 5y ago
> - You shouldn't try to invent your own cryptographic primitives. > - You shouldn't try to design your own equivalent to (say) SSL. > - You shouldn't try to implement something like SSL, or parse ASN.1, yourself. Why not? You should do all those things, how else can one get a deeper insight into how things work. It's just that you probably don't want to use that in production environment unless you really know what you're doing.
- bostik 5y ago
  You make a good point, but I feel it's getting downvoted thanks to the implied tone. > [...] get a deeper insight into how things work. It's just that you probably don't want to use that in production [...] That is the gist of it. If you are really into applied cryptography, then rolling your own crypto - and I can't stress this enough - for PURELY ACADEMIC purposes is fine. As long as you are honest with yourself, writing some applied crypto code is a good way to learn some of the horrible ways things can go wrong. Know deep in your heart that no matter how well you thought you did, the code you wrote is broken. Badly. The "oh... oh shit..." realisation when looking back at the code will be enlightening. Just please, for the love of all that is good, do not publish the code you wrote. Otherwise, thanks to the law of large numbers, someone, somewhere will grab your code, embed it in their production systems and the next thing you know, a toy library you wrote is now responsible for yet another IoT disaster.
  - adgjlsfhk1 5y ago
    Or, here me out, roll your own crypto, and put it on a webservers that GPT-2 generated messages back and forth. That should give an intern at the NSA/KGB a really annoying summer.
- tialaramex 5y ago
  > how else can one get a deeper insight into how things work Learn from other people's mistakes. You cannot possibly afford to make all those mistakes yourself.
johnisgood 5y ago
> - You shouldn't try to implement something like SSL, or parse ASN.1, yourself. What would you recommend for compiling and decoding/encoding ASN.1 using Java? I tried OSS NOKALVA's ASN.1 for Java but constraint checking is non-existent or very limited. I am forced to implement it myself as it does not exist from what I gathered. If you know one that does constraint checking, that would be amazing. A quick search gave me this: https://github.com/yafred/asn1-tool/issues/13 https://github.com/yafred/asn1-tool/issues/13, so perhaps it does not seem so infeasible. > I think if someone in the early 2000s had thought "I don't like the look of OpenSSL and I don't think C is a good choice of language. I'm going to write my own implementation of the parts I need in (say) Ada" https://github.com/Componolit/libsparkcrypto https://github.com/Componolit/libsparkcrypto might be of interest to some. It is written in Ada/SPARK.
- memling 5y ago
  > I tried OSS NOKALVA's ASN.1 for Java but constraint checking is non-existent or very limited. A quick look at their documentation suggests the use of the -constraints option.[1] Constraints are such a basic part of the ASN.1 specifications I can't imagine any vendor not implementing it. I know for a fact that Objective Systems' compiler does.[2] [1]: https://www.oss.com/asn1/products/documentation/asn1_java_8.3/asn1java-compiler-options-reference.html#constraints https://www.oss.com/asn1/products/documentation/asn1_java_8.... [2]: https://www.obj-sys.com/products/asn1c/index.php https://www.obj-sys.com/products/asn1c/index.php
  - johnisgood 5y ago
    Thank you, I think I missed `-constraints`, then! I will give it a go and check out the generated file. I expected it to be turned on automatically. My bad! That said, I will check if it does the constraint checking properly. I hope it does! I know, Erlang's ASN.1 compiler does it too, automatically. It is pretty awesome. --- Update: `-constraints` does work (`validConstraint = temp1 >= 10 && temp1 <= 20`) I will try out more complex ones. :)
tptacek 5y ago
If someone in the 2000s had thought "I don't like C and I don't like OpenSSL so I'm going to write my own Ada SSL/TLS", we would have yet another implementation of Bleichenbacher's RSA error oracle. It is not actually that easy to put a safe TLS implementation together; you need to study the TLS protocol and implementation vulnerabilities, and they're not all assembled in one nice place.
- mjw1007 5y ago
  Non-rhetorical question: do you think that would have been likely to cause the hypothetical implementor more or less pain, over the next decade or so, than OpenSSL's buffer overflows?
  - tptacek 5y ago
    It depends on the window of time you choose, but if the window is "in the last 10 years", then I'd say cryptographic mistakes are likely to cause more pain than OpenSSL memory corruption vulnerabilities. Any other C library, with less scrutiny, and it'd be the other way around.

unilynx 5y ago

Given the many historic openssl bugs related to certificate parsing (ie x509 and asn1) I’m not sure you should even trust expert crypto programmers to deal with that

cormacrelf 5y ago
Some folks are trying to kill ASN.1. It is probably an awkward moment for that, when people are looking at binary encodings more because JSON is boring and doesn't stand out on a CV. It's got all the hallmarks of old tech that looks like it's survived on its merits, but it really isn't like that. > "ASN.1 is a serialization format which enables C programmers to turn binary data into remote code execution vulnerabilities" -- https://twitter.com/bascule/status/1086778481864658945 https://twitter.com/bascule/status/1086778481864658945 Also https://twitter.com/TwitchiH/status/1382649508371308547 https://twitter.com/TwitchiH/status/1382649508371308547, discussing it in context of the San Bernadino iPhone being broken with an ASN.1 vuln.
- lifthrasiir 5y ago
  The difference between ASN.1 (BER/DER) and modern binary encodings is that people have tried to parse the former by hand but not the latter (at least to my experience). BER/DER itself is a fine encoding---sure, it has a legacy including six string types but we can deal with them---and in principle a simple library with bound checks should have been more than enough. ASN.1 is however structured in a way that such a simple library looks like incomplete; it looks like that a correct way to parse BER/DER is an ASN.1 compiler, which tends to be bulky and annoying to integrate. As a result for a long time most ASN.1 implementations were either a "complete" ASN.1 compiler or a hand-crafted parser, with no middle ground of a library that is simple enough and prevents you from memory unsafety.
- memling 5y ago
  > Some folks are trying to kill ASN.1. It is probably an awkward moment for that, when people are looking at binary encodings more because JSON is boring and doesn't stand out on a CV. It's got all the hallmarks of old tech that looks like it's survived on its merits, but it really isn't like that. I think it's rather more than this. ASN.1 is deeply embedded in daily life: 3G/4G/LTE/5G, V2X, and many other protocols are specified in ASN.1. The inertia has as much to do with backwards compatibility as anything else. If you need to read call data records, you have to parse BER; if you want to deal with vehicle to vehicle communications you have to parse OER. 3G protocols are generally PER. Crypto is DER. Are there alternatives that are better in some respects? Sure. Are they likely to supplant existing telecommunications standards? No. Should move away from ASN.1 for crypto purposes? Probably, though I'm not experienced enough to have a strong opinion. I'd be interested to see whether we could install a new standard in a suitable time frame, and indeed what form that would take. (In the two links you cited, I didn't see anything, but I'm actually really interested!) I think ASN.1 is probably here to stay for a long time, and not merely as a resume polishing agent.
- woodruffw 5y ago
  > because JSON is boring and doesn't stand out on a CV. It's a little bit more than that: cryptographic container formats usually need to be stable on traversal/canonicalized so that things like hashes remain consistent. JSON alone doesn't have a canonicalized format (there are various suggested canonical forms), and leaves details about key iteration, duplication, etc. ambiguous in ways that are potential sources of vulnerability (what happens if a particular protocol allows users to introduce untrusted attributes to a JSON object and doesn't check for name clashes?). Edit: not to mention integer width/representation of large integers! That's a big one for cryptographic protocols.
  - memling 5y ago
    > It's a little bit more than that: cryptographic container formats usually need to be stable on traversal/canonicalized so that things like hashes remain consistent. Yeah, this, too: there's got to be a single way of representing the same message. (BER, in contrast to DER, has multiple means of writing the same message. BOOLEAN types, for example, indicate FALSE by 0x00, but TRUE is any non-zero value. I forget how DER resolves this. DER disallows the use of indefinite lengths, while BER admits both definite and indefinite lengths. Etc., etc.) I'm sympathetic to the challenges people face with ASN.1, having worked with it rather extensively in my career. I don't recommend it to people who have serialization challenges usually because something like protobuf or flatbuffers seems like a more reasonable choice in this day and age...but at the same time ASN.1 has a history of mistakes and innovations for solving some challenging problems in representation and encoding that should inform decision-making moving forward.
- jnwatson 5y ago
  ASN.1's primary problem is that it has been around a long time. Long before folks understood the security implications of a buffer overflow, we've been parsing ASN.1. Most of the vulnerabilities stem from the basic problem of trusting length fields. Any binary encoding is going to have similar issues.

ThePhysicist 5y ago

Often you don't have a choice but to write your own parsers for serialization formats, as inter-language & inter-tool interoperability is still not very good, except maybe for TLS-related stuff. There's e.g. no predefined way to verify an ECDSA signature created in JS using Golang (or vice versa), so you have to parse the R & S values by hand. Luckily they are just concatenated big-endian numbers.

lu4p 5y ago
I'm pretty sure there is https://pkg.go.dev/crypto/ecdsa#Verify https://pkg.go.dev/crypto/ecdsa#Verify
- ThePhysicist 5y ago
  That's exactly what I referred to: The function takes two BigInt's, which you have to unpack yourself from the byte-serialization created by JS. There's no JS-compatible ECDH key derivation function in Golang either, so you have to implement the multiplication yourself (which again isn't difficult).
  - dcow 5y ago
    My impression is that crypto people, being mildly allergic to asn.1, decided not to use it to specify standard serialization formats in our new world of EC crypto. So here we are without asn.1 and without a replacement. I generally think if you’re going to remove something because you think it sucks it’s on you to replace it with an equivalent or clearly justify why it was vestigial if you don't have one. In this case you really don't need a DER calibre encoding either, you just need a standard in the first place. It’s just that the standard would historically likely use asn.1. But that doesn't have to be the case. Maybe the closest thing is the ECDSA point form specified in https://www.secg.org/sec1-v2.pdf#page16 https://www.secg.org/sec1-v2.pdf#page16. The the curve domain parameters in sec2 are, you guessed it, specified in asn.1.
    - johnisgood 5y ago

janeroe 5y ago

> - You shouldn't try to invent your own cryptographic primitives. > - You shouldn't try to design your own equivalent to (say) SSL. > - You shouldn't try to implement something like SSL, or parse ASN.1, yourself. Why not? You should do all those things, how else can one get a deeper insight into how things work. It's just that you probably don't want to use that in production environment unless you really know what you're doing.

bostik 5y ago
You make a good point, but I feel it's getting downvoted thanks to the implied tone. > [...] get a deeper insight into how things work. It's just that you probably don't want to use that in production [...] That is the gist of it. If you are really into applied cryptography, then rolling your own crypto - and I can't stress this enough - for PURELY ACADEMIC purposes is fine. As long as you are honest with yourself, writing some applied crypto code is a good way to learn some of the horrible ways things can go wrong. Know deep in your heart that no matter how well you thought you did, the code you wrote is broken. Badly. The "oh... oh shit..." realisation when looking back at the code will be enlightening. Just please, for the love of all that is good, do not publish the code you wrote. Otherwise, thanks to the law of large numbers, someone, somewhere will grab your code, embed it in their production systems and the next thing you know, a toy library you wrote is now responsible for yet another IoT disaster.
- adgjlsfhk1 5y ago
  Or, here me out, roll your own crypto, and put it on a webservers that GPT-2 generated messages back and forth. That should give an intern at the NSA/KGB a really annoying summer.
tialaramex 5y ago
> how else can one get a deeper insight into how things work Learn from other people's mistakes. You cannot possibly afford to make all those mistakes yourself.

johnisgood 5y ago

> - You shouldn't try to implement something like SSL, or parse ASN.1, yourself. What would you recommend for compiling and decoding/encoding ASN.1 using Java? I tried OSS NOKALVA's ASN.1 for Java but constraint checking is non-existent or very limited. I am forced to implement it myself as it does not exist from what I gathered. If you know one that does constraint checking, that would be amazing. A quick search gave me this: https://github.com/yafred/asn1-tool/issues/13 https://github.com/yafred/asn1-tool/issues/13, so perhaps it does not seem so infeasible. > I think if someone in the early 2000s had thought "I don't like the look of OpenSSL and I don't think C is a good choice of language. I'm going to write my own implementation of the parts I need in (say) Ada" https://github.com/Componolit/libsparkcrypto https://github.com/Componolit/libsparkcrypto might be of interest to some. It is written in Ada/SPARK.

memling 5y ago
> I tried OSS NOKALVA's ASN.1 for Java but constraint checking is non-existent or very limited. A quick look at their documentation suggests the use of the -constraints option.[1] Constraints are such a basic part of the ASN.1 specifications I can't imagine any vendor not implementing it. I know for a fact that Objective Systems' compiler does.[2] [1]: https://www.oss.com/asn1/products/documentation/asn1_java_8.3/asn1java-compiler-options-reference.html#constraints https://www.oss.com/asn1/products/documentation/asn1_java_8.... [2]: https://www.obj-sys.com/products/asn1c/index.php https://www.obj-sys.com/products/asn1c/index.php
- johnisgood 5y ago
  Thank you, I think I missed `-constraints`, then! I will give it a go and check out the generated file. I expected it to be turned on automatically. My bad! That said, I will check if it does the constraint checking properly. I hope it does! I know, Erlang's ASN.1 compiler does it too, automatically. It is pretty awesome. --- Update: `-constraints` does work (`validConstraint = temp1 >= 10 && temp1 <= 20`) I will try out more complex ones. :)

tptacek 5y ago

If someone in the 2000s had thought "I don't like C and I don't like OpenSSL so I'm going to write my own Ada SSL/TLS", we would have yet another implementation of Bleichenbacher's RSA error oracle. It is not actually that easy to put a safe TLS implementation together; you need to study the TLS protocol and implementation vulnerabilities, and they're not all assembled in one nice place.

mjw1007 5y ago
Non-rhetorical question: do you think that would have been likely to cause the hypothetical implementor more or less pain, over the next decade or so, than OpenSSL's buffer overflows?
- tptacek 5y ago
  It depends on the window of time you choose, but if the window is "in the last 10 years", then I'd say cryptographic mistakes are likely to cause more pain than OpenSSL memory corruption vulnerabilities. Any other C library, with less scrutiny, and it'd be the other way around.