7 ms·

OpenBSD forks, prunes, fixes OpenSSL

62 points by zorked 13y ago

RRRA 13y ago
Cryptographic algorithm usually are pretty straight forward when you look at them. Beside good verifiable coding practice to have a quality implementation, it seems that the issue is with the many side channels attack that needs to be obfuscated and make the whole thing a lot harder to maintain and verify. So I'm guessing the issue is probably more about the very badly engineered protocols with blurry designs and too many bells and whistles rather than with the basic cryptographic building blocks?
- marshray 13y ago
  Cryptographic algorithm usually are pretty straight forward when you look at them. Take a look at the hand-optimized for speed immplementations in assembly sometime.
  - userbinator 13y ago
    They're still just doing a bunch of math; the complexity is in I/O and protocol interactions, not the algorithms themselves.
    - Tloewald 13y ago
      In my experience, people tend to wildly over- or under- estimate the complexity of anything they don't understand.
    - marshray 13y ago
      Here you go. http://www.opensource.apple.com/source/OpenSSL098/OpenSSL098-27/src/crypto/aes/asm/ http://www.opensource.apple.com/source/OpenSSL098/OpenSSL098... Note: it's Perl.
    - pbsd 13y ago
      The AES is a good example of a cryptographic algorithm that is rather not straightforward to implement in an efficient and safe manner. AES is defined on a 4x4 byte state, and every operation (even the sbox) is defined over GF(2^8). However, fast implementations resemble very little of that definition, and end up being a large number of xor and (large) table lookups. This is a much faster way to do it, on general-purpose machines, but it often leaves you open to cache-timing attacks. So to come up with a fast and secure AES implementation you need to transpose the state in a way that resembles processing 8 blocks in parallel, and emulate hardware-level bitwise arithmetic on software. This is a so-called bitsliced implementation. By this point the code is much more complex and tricky than the original specification, and it will only actually be fast for parallel modes of operation such as CTR, leaving CBC hanging. Thankfully processors now implement AES round instructions, which simplifies secure implementations, but makes portable implementations difficult. Cryptographers have learned from this state of affairs, though. New algorithms being created (like Keccak, aka SHA-3) now try harder to make the straightforward implementation secure and reasonably fast by default. Public-key algorithms, on the other hand, are always tricky to implement, even if the arithmetic itself is not very complicated. One of the appeals of curve25519 (and other so-called "safe curves") is precisely that it is rather easy (but still not trivial) to implement them in a safe manner.
- anaphor 13y ago
  Protecting against sidechannels doesn't mean the code has to be obfuscated. You do need to reimplement some things that are usually pretty simple (like string comparison) but it does not mean you can't abstract that away into a library like anything else.
- dsl 13y ago
  This is a quote from a OpenBSD developer that sums up the work they are doing nicely: "Of necessity, some crypto code will be a bit scary to look at. anything that has to do regular jobs, like building a pathname, in a security library, should be a thing of beauty, not something that makes you want to throw up in your mouth."
leccine 13y ago
OpenSSL is beyond repair. We need a new clean start, sane design and rock solid implementation with principles and following best practices.
- AceJohnny2 13y ago
  Like NaCl? http://nacl.cr.yp.to/ http://nacl.cr.yp.to/
  - tptacek 13y ago
    NaCl addresses a different problem than OpenSSL. NaCl is great if you have no legacy or compatibility requirements, and it should be the only crypto library most web and mobile developers ever come within a mile of. But it's not a workable replacement for OpenSSL.
    - Tomte 13y ago
      What about this from the libsodium web site? "NaCl ships with constructions that were just prototypes, and that shouldn’t be used any more." What does "shouldn't be used any more" mean? Certainly not security, or djb would have released an update. Compatibility with other products? Something else?
      - tptacek 13y ago
        No, IIRC Bernstein improved his public key signing system, and libsodium uses the new version. Neither construction is useful for standard TLS.
  - uuid_to_string 13y ago
    For per packet encryption. For authentication, there is CurveCP. People seem confident with OpenSSH's authentication mechanism. Why not use that? At some point one has to trust that the IP address one is sending/retrieving data to/from is the correct one. That's easier said than done if some host wants to keep changing its IP address every few days. The SSL PKI scheme (the SSL approach to authentication), as implemented for public websites, is not much of a confidence-builder, IMO. Opinions may differ. If websites maintained consistent IP addresses and we could authenticate these machines using OpenSSH keys, I would be more willing to believe we could verify their "authenticity".
bla2 13y ago
It's cool they're doing this cleanup, but overly self-righteous commit messages (http://opensslrampage.org/ http://opensslrampage.org/) on patches like http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/crypto/asn1/a_time.c.diff?r1=1.15;r2=1.16 http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... rub me the wrong way (I'm sure comparing a string pointer to '5' / decimal 53 is _exactly_ what the old, ridiculed code meant to do…)
- bla2 13y ago
  (To their credit, they did fix this bug 2 hours later: http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/crypto/asn1/a_time.c.diff?r1=1.16;r2=1.17 http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/crypto/asn1/a_time.c http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... )
- Gigablah 13y ago
  Well, fixing legacy code is generally regarded as crap work and developers need to vent. Still, they're lucky that they're not implementing the actual crypto code, otherwise they'll face much worse public beatdowns.
- jevinskie 13y ago
  That is pretty bad. I can't see how that wouldn't give a new compiler warning that would tell the dev that they probably shouldn't commit that patch.
  - xarball 13y ago
    You may need a few compiler flags turned off to compile OpenSSL ;) That said, many opensource projects ignore the vast majority of compilation warnings. Many projects do the same.
- Touche 13y ago
  This is why I would never use OpenBSD (but yeah, I use OpenSSH). Can't use an OS for people with this type of attitude.
Tloewald 13y ago
I have to say -- and I am not a system programmer -- this shit is pretty hilarious. E.g. Multiple unnecessary and broken implementations of common library functions (printf!).
tedivm 13y ago
> Some of that is indentation, because we are trying to make the code more comprehensible. 99.99% of the community does not care for VMS support, and 98% do not care for Windows support. They care for POSIX support, so that the Unix and Unix derivatives can run. Do they not realize that clients also need SSL support? Servers that don't have anyone to talk to them are kind of useless, so at best this is half of solution (unless it really is the year of the linux desktop, of course). There are a ton of applications out there that run on Windows and use OpenSSL for handling SSL connections. That being said this may be a great replacement for servers, and there really isn't a requirement for there to be "one library to rule them all". This may be a great step towards diversifying the encryption-library ecosystem a bit.
- talideon 13y ago
  They're building it first and foremost for OpenBSD. If anybody's interested in using their fork, they'll create a portable version, as they do for OpenSSH, OpenSMTPD, &c. The act of stripping the library down is worthwhile though: OpenSSL has accumulated a lot of cruft over the years.
  - jmspring 13y ago
    I think restructuring OpenSSL with better modularization could achieve / maintain more compatibility than just ripping things out.
    - talideon 13y ago
      The OpenSSL codebase is already pretty modular, but it's not very clean, and that's its major issue. Needless to say, it could be made even more modular, with the crypo code, X.509 code, the SSL/TLS protocol implementation, &c., entirely separated from one another, but doing that will still benefit from pulling out as much cruft as possible. They're ripping out all that stuff because it's more hard to test stuff that can just go wrong, and when it comes to critical security infrastructure, that's a bad thing to have lying around. All they really need to do to maintain compatibility is maintain the same API. And they're doing just that. In fact, it's in their best interests to maintain the same API as deviating in any significant way is just going to complicate porting software that uses OpenSSL over to use their implementation. So long as they do that, there's little to worry about. And maybe some day their fork will end up becoming the dominant SSL/TLS implementation, much as OpenSSH became the dominant SSH implementation.
Eyas 13y ago
A quote in there says that customers don't care for FIPS support. I hope OpenBSD is not planning on dropping FIPS support from LibreSSL -- it seems like that would be a huge mistake. Government agencies are among the many that are open to SSL vulnerabilities. Our tax, financial, SSN, and healthcare information are all open to vulnerabilities in buggy code. Dropping FIPS support means that government agencies whose security we (as consumers) care a whole lot about will not be benefiting from the massive improvements that the OpenBSD community hopes to bring about.
- mjibson 13y ago
  It appears doubtful that FIPS will be restored. See the commit: http://marc.info/?l=openbsd-cvs&m=139775148919571&w=2 http://marc.info/?l=openbsd-cvs&m=139775148919571&w=2 "Forcible certification conflicts with the goals of a free software project."
- throwaway2048 13y ago
  Even if OpenBSD did not drop the FIPS code, LibreSSL would not be FIPS compliant because that involves paying large amounts of money for certification, which with their limited resources, and the way FIPS nessisarily constrains development, they would be unwilling to do.
- nathanvanfleet 13y ago
  I wonder about if the NSA, who was supposedly aware of the bugs (and likely others), had to leave other US gov agencies open to the exploits because they would otherwise blow their cover? Or might they have had internal custom OpenSSL (or just didn't use OpenSSL)?
- tobiasu 13y ago
  Since when does OpenBSD (a Canadian project with a developer group all over the planet) need to care about some certification and standard bullshit by the government of the United States of America? What have the USA done for us lately? Oh yes, fought a long battle against cryptography in the hands of world citizens, spied on everyone and their dog, canceled OpenBSD funding, started multiple illegal wars and threatened every single country into submission (except for the one with nukes pointed at the White House). Does it get any more selfish? Bah.
benesch 13y ago
QAWSEDRTYGFRDESWAQSWEDSWAQ
jmspring 13y ago
Somehow, I don't see removing FIPS or Windows support being in the best interest of the community as a whole. For startups wanting to have a product that requires some level of FIPS support, starting with a decent base makes things easier. Windows support? There are more than a couple of applications out there running on Windows that make use of OpenSSL. Theo's quote -- "99.99% of the community does not care for VMS support, and 98% do not care for Windows support." -- is, at least to me, narrow minded in the scope of what he defines as "community" -- unless he specifically means the OpenBSD community.
- ahknight 13y ago
  He does.
  - jmspring 13y ago
    It would have been helpful if the article called this out. It could probably be implied, but forking/correcting/renaming left it a tad ambiguous.
- maaku 13y ago
  Except, for example, FIPS support mandates availability of operating modes which are broken by design.
  - jmspring 13y ago
    There are ways of crafting the software that could make sure the broken operating modes aren't necessarily part of the FIPS build profile. More than one company I have worked at built on top of OpenSSL and went for FIPS compliance. Sometimes, if you want to be able to qualify for contracts that require things like FIPS, building off a base is better than starting from complete scratch.
- xarball 13y ago
  I'm sure they have all the best intentions for sharing the wisdom they learn here. But right now OpenSSL is a VERY VERY messy and dangerous library. The OpenBSD devs need to get a working implementation, that catches up on 15 years of refactoring and clarity in (optimistically) 2 months -- or risk the bulk of the world's sensitive information leaking yet again (perhaps even sooner, depending on what is discovered!) It seems that with the kind of things they're finding, there are a lot lesser understood vulnerabilities in the code. With the kind of standardization they're working on, a Windows port will be a LOT easier and a LOT simpler after they're finished. Best let them make haste, tear out everything that's leaky, unclear, or dangerous -- and with that will come a lot of simplification and fixes for the entire implementation. Once it's clean, focus on a Ports. The code is too bloated to even consider ports right now. Quite frankly you might even be safer using Microsoft's SSL/TLS and other crypto implementation for the time being... (Only time will tell!)
mrmondo 13y ago
IMO, Putting libre in front of any project name is a surefire way to loose enterprise any and all enterprise / corporate acceptance.
purple_horse 13y ago
This seems like a good time to mention that 50% of the problem is the overcomplexity of TLS itself not necessairly OpenSSL...
thatthatis 13y ago
I can't help feeling a bit bad for the OpenSSL team in all of this. For years they did a thankless yeoman's work with insufficient resources and now that people are recognizing the work they did it's almost uniformly negative. Of course you can argue that they brought it upon themselves, but man would I love to watch a 60-minute documentary on how OpenSSL came to be, the people behind it, and how it went wrong.

62 points by zorked 13y ago

RRRA 13y ago

Cryptographic algorithm usually are pretty straight forward when you look at them. Beside good verifiable coding practice to have a quality implementation, it seems that the issue is with the many side channels attack that needs to be obfuscated and make the whole thing a lot harder to maintain and verify. So I'm guessing the issue is probably more about the very badly engineered protocols with blurry designs and too many bells and whistles rather than with the basic cryptographic building blocks?

marshray 13y ago
Cryptographic algorithm usually are pretty straight forward when you look at them. Take a look at the hand-optimized for speed immplementations in assembly sometime.
- userbinator 13y ago
  They're still just doing a bunch of math; the complexity is in I/O and protocol interactions, not the algorithms themselves.
  - Tloewald 13y ago
    In my experience, people tend to wildly over- or under- estimate the complexity of anything they don't understand.
  - marshray 13y ago
    Here you go. http://www.opensource.apple.com/source/OpenSSL098/OpenSSL098-27/src/crypto/aes/asm/ http://www.opensource.apple.com/source/OpenSSL098/OpenSSL098... Note: it's Perl.
  - pbsd 13y ago
    The AES is a good example of a cryptographic algorithm that is rather not straightforward to implement in an efficient and safe manner. AES is defined on a 4x4 byte state, and every operation (even the sbox) is defined over GF(2^8). However, fast implementations resemble very little of that definition, and end up being a large number of xor and (large) table lookups. This is a much faster way to do it, on general-purpose machines, but it often leaves you open to cache-timing attacks. So to come up with a fast and secure AES implementation you need to transpose the state in a way that resembles processing 8 blocks in parallel, and emulate hardware-level bitwise arithmetic on software. This is a so-called bitsliced implementation. By this point the code is much more complex and tricky than the original specification, and it will only actually be fast for parallel modes of operation such as CTR, leaving CBC hanging. Thankfully processors now implement AES round instructions, which simplifies secure implementations, but makes portable implementations difficult. Cryptographers have learned from this state of affairs, though. New algorithms being created (like Keccak, aka SHA-3) now try harder to make the straightforward implementation secure and reasonably fast by default. Public-key algorithms, on the other hand, are always tricky to implement, even if the arithmetic itself is not very complicated. One of the appeals of curve25519 (and other so-called "safe curves") is precisely that it is rather easy (but still not trivial) to implement them in a safe manner.
anaphor 13y ago
Protecting against sidechannels doesn't mean the code has to be obfuscated. You do need to reimplement some things that are usually pretty simple (like string comparison) but it does not mean you can't abstract that away into a library like anything else.
dsl 13y ago
This is a quote from a OpenBSD developer that sums up the work they are doing nicely: "Of necessity, some crypto code will be a bit scary to look at. anything that has to do regular jobs, like building a pathname, in a security library, should be a thing of beauty, not something that makes you want to throw up in your mouth."

leccine 13y ago

OpenSSL is beyond repair. We need a new clean start, sane design and rock solid implementation with principles and following best practices.

AceJohnny2 13y ago
Like NaCl? http://nacl.cr.yp.to/ http://nacl.cr.yp.to/
- tptacek 13y ago
  NaCl addresses a different problem than OpenSSL. NaCl is great if you have no legacy or compatibility requirements, and it should be the only crypto library most web and mobile developers ever come within a mile of. But it's not a workable replacement for OpenSSL.
  - Tomte 13y ago
    What about this from the libsodium web site? "NaCl ships with constructions that were just prototypes, and that shouldn’t be used any more." What does "shouldn't be used any more" mean? Certainly not security, or djb would have released an update. Compatibility with other products? Something else?
    - tptacek 13y ago
      No, IIRC Bernstein improved his public key signing system, and libsodium uses the new version. Neither construction is useful for standard TLS.
- uuid_to_string 13y ago
  For per packet encryption. For authentication, there is CurveCP. People seem confident with OpenSSH's authentication mechanism. Why not use that? At some point one has to trust that the IP address one is sending/retrieving data to/from is the correct one. That's easier said than done if some host wants to keep changing its IP address every few days. The SSL PKI scheme (the SSL approach to authentication), as implemented for public websites, is not much of a confidence-builder, IMO. Opinions may differ. If websites maintained consistent IP addresses and we could authenticate these machines using OpenSSH keys, I would be more willing to believe we could verify their "authenticity".

bla2 13y ago

It's cool they're doing this cleanup, but overly self-righteous commit messages (http://opensslrampage.org/ http://opensslrampage.org/) on patches like http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/crypto/asn1/a_time.c.diff?r1=1.15;r2=1.16 http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... rub me the wrong way (I'm sure comparing a string pointer to '5' / decimal 53 is _exactly_ what the old, ridiculed code meant to do…)

bla2 13y ago
(To their credit, they did fix this bug 2 hours later: http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/crypto/asn1/a_time.c.diff?r1=1.16;r2=1.17 http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/crypto/asn1/a_time.c http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/cry... )
Gigablah 13y ago
Well, fixing legacy code is generally regarded as crap work and developers need to vent. Still, they're lucky that they're not implementing the actual crypto code, otherwise they'll face much worse public beatdowns.
jevinskie 13y ago
That is pretty bad. I can't see how that wouldn't give a new compiler warning that would tell the dev that they probably shouldn't commit that patch.
- xarball 13y ago
  You may need a few compiler flags turned off to compile OpenSSL ;) That said, many opensource projects ignore the vast majority of compilation warnings. Many projects do the same.
Touche 13y ago
This is why I would never use OpenBSD (but yeah, I use OpenSSH). Can't use an OS for people with this type of attitude.

Tloewald 13y ago

I have to say -- and I am not a system programmer -- this shit is pretty hilarious. E.g. Multiple unnecessary and broken implementations of common library functions (printf!).

tedivm 13y ago

> Some of that is indentation, because we are trying to make the code more comprehensible. 99.99% of the community does not care for VMS support, and 98% do not care for Windows support. They care for POSIX support, so that the Unix and Unix derivatives can run. Do they not realize that clients also need SSL support? Servers that don't have anyone to talk to them are kind of useless, so at best this is half of solution (unless it really is the year of the linux desktop, of course). There are a ton of applications out there that run on Windows and use OpenSSL for handling SSL connections. That being said this may be a great replacement for servers, and there really isn't a requirement for there to be "one library to rule them all". This may be a great step towards diversifying the encryption-library ecosystem a bit.

talideon 13y ago
They're building it first and foremost for OpenBSD. If anybody's interested in using their fork, they'll create a portable version, as they do for OpenSSH, OpenSMTPD, &c. The act of stripping the library down is worthwhile though: OpenSSL has accumulated a lot of cruft over the years.
- jmspring 13y ago
  I think restructuring OpenSSL with better modularization could achieve / maintain more compatibility than just ripping things out.
  - talideon 13y ago
    The OpenSSL codebase is already pretty modular, but it's not very clean, and that's its major issue. Needless to say, it could be made even more modular, with the crypo code, X.509 code, the SSL/TLS protocol implementation, &c., entirely separated from one another, but doing that will still benefit from pulling out as much cruft as possible. They're ripping out all that stuff because it's more hard to test stuff that can just go wrong, and when it comes to critical security infrastructure, that's a bad thing to have lying around. All they really need to do to maintain compatibility is maintain the same API. And they're doing just that. In fact, it's in their best interests to maintain the same API as deviating in any significant way is just going to complicate porting software that uses OpenSSL over to use their implementation. So long as they do that, there's little to worry about. And maybe some day their fork will end up becoming the dominant SSL/TLS implementation, much as OpenSSH became the dominant SSH implementation.

Eyas 13y ago

A quote in there says that customers don't care for FIPS support. I hope OpenBSD is not planning on dropping FIPS support from LibreSSL -- it seems like that would be a huge mistake. Government agencies are among the many that are open to SSL vulnerabilities. Our tax, financial, SSN, and healthcare information are all open to vulnerabilities in buggy code. Dropping FIPS support means that government agencies whose security we (as consumers) care a whole lot about will not be benefiting from the massive improvements that the OpenBSD community hopes to bring about.

mjibson 13y ago
It appears doubtful that FIPS will be restored. See the commit: http://marc.info/?l=openbsd-cvs&m=139775148919571&w=2 http://marc.info/?l=openbsd-cvs&m=139775148919571&w=2 "Forcible certification conflicts with the goals of a free software project."
throwaway2048 13y ago
Even if OpenBSD did not drop the FIPS code, LibreSSL would not be FIPS compliant because that involves paying large amounts of money for certification, which with their limited resources, and the way FIPS nessisarily constrains development, they would be unwilling to do.
nathanvanfleet 13y ago
I wonder about if the NSA, who was supposedly aware of the bugs (and likely others), had to leave other US gov agencies open to the exploits because they would otherwise blow their cover? Or might they have had internal custom OpenSSL (or just didn't use OpenSSL)?
tobiasu 13y ago
Since when does OpenBSD (a Canadian project with a developer group all over the planet) need to care about some certification and standard bullshit by the government of the United States of America? What have the USA done for us lately? Oh yes, fought a long battle against cryptography in the hands of world citizens, spied on everyone and their dog, canceled OpenBSD funding, started multiple illegal wars and threatened every single country into submission (except for the one with nukes pointed at the White House). Does it get any more selfish? Bah.

benesch 13y ago

QAWSEDRTYGFRDESWAQSWEDSWAQ

jmspring 13y ago

Somehow, I don't see removing FIPS or Windows support being in the best interest of the community as a whole. For startups wanting to have a product that requires some level of FIPS support, starting with a decent base makes things easier. Windows support? There are more than a couple of applications out there running on Windows that make use of OpenSSL. Theo's quote -- "99.99% of the community does not care for VMS support, and 98% do not care for Windows support." -- is, at least to me, narrow minded in the scope of what he defines as "community" -- unless he specifically means the OpenBSD community.

ahknight 13y ago
He does.
- jmspring 13y ago
  It would have been helpful if the article called this out. It could probably be implied, but forking/correcting/renaming left it a tad ambiguous.
maaku 13y ago
Except, for example, FIPS support mandates availability of operating modes which are broken by design.
- jmspring 13y ago
  There are ways of crafting the software that could make sure the broken operating modes aren't necessarily part of the FIPS build profile. More than one company I have worked at built on top of OpenSSL and went for FIPS compliance. Sometimes, if you want to be able to qualify for contracts that require things like FIPS, building off a base is better than starting from complete scratch.
xarball 13y ago
I'm sure they have all the best intentions for sharing the wisdom they learn here. But right now OpenSSL is a VERY VERY messy and dangerous library. The OpenBSD devs need to get a working implementation, that catches up on 15 years of refactoring and clarity in (optimistically) 2 months -- or risk the bulk of the world's sensitive information leaking yet again (perhaps even sooner, depending on what is discovered!) It seems that with the kind of things they're finding, there are a lot lesser understood vulnerabilities in the code. With the kind of standardization they're working on, a Windows port will be a LOT easier and a LOT simpler after they're finished. Best let them make haste, tear out everything that's leaky, unclear, or dangerous -- and with that will come a lot of simplification and fixes for the entire implementation. Once it's clean, focus on a Ports. The code is too bloated to even consider ports right now. Quite frankly you might even be safer using Microsoft's SSL/TLS and other crypto implementation for the time being... (Only time will tell!)

mrmondo 13y ago

IMO, Putting libre in front of any project name is a surefire way to loose enterprise any and all enterprise / corporate acceptance.

purple_horse 13y ago

This seems like a good time to mention that 50% of the problem is the overcomplexity of TLS itself not necessairly OpenSSL...

thatthatis 13y ago

I can't help feeling a bit bad for the OpenSSL team in all of this. For years they did a thankless yeoman's work with insufficient resources and now that people are recognizing the work they did it's almost uniformly negative. Of course you can argue that they brought it upon themselves, but man would I love to watch a 60-minute documentary on how OpenSSL came to be, the people behind it, and how it went wrong.