5 ms·

Please characterize the proper way for us of the uninformed...

tptacek 13y ago
All software has bugs. There's no piece of real-world general-purpose software that you can't look at and say "this was improved incrementally as bugs were found". But not all software has the same sensitivity. In the weeks before Netscape Navigator shipped in the mid-90s, Jamie Zawinski slept under his desk at work trying to make it reliable. The builds from the night before launch crashed! Nobody thought they'd make it. That's a story anyone on a commercial dev team can probably tell you; it's the oldest story in product development. That kind of incremental development works fine when you're shipping the first web browser, or even the first version of a new web browser. It does not work when you are shipping cryptography. Nate Lawson is fond of saying, "plan to budget 10x for the validation of a new cryptosystem as you to implementation". You probably can't actually be doing that when your whole cryptosystem changes 4 times in 2 years. The difference is between handling bugs and expecting bugs. Netscape expected bugs. Crypto software can't do that.
- saosebastiao 13y ago
  You haven't explained anything about how crypto software is developed. You have told us how other software is developed and then told us that it isn't okay for crypto software.
  - tptacek 13y ago
    Spend 10x the resources on verification as on implementation. Spend 10x the resources on design as on implementation. Nobody does this with normal software.
    - jbri 13y ago
      And ultimately the biggest thing to do is that when you find a bug, don't just fix that one bug and call it a day. If you find a bug in your crypto code through your testing and validation, that means your development process screwed up and allowed that bug to slip through. What you need to do is figure out how that bug got through your development process, change your development process so it wouldn't slip through next time, and then have a thorough audit of your already-written code to make sure no other bugs have slipped through the same way. Developing must-not-fail critical software is the exact opposite of the traditional launch-fast-and-iterate model that most people use to produce web applications and such. This is a big reason that when web app developers try their hand at developing critical software like cryptography, they get it horribly wrong.
      - nekopa 13y ago
        This seems similar to way the people who developed the software for the space shuttle worked. Finding a bug was a huge deal, as it meant something was wrong in their process. It wasn't just fix the bug, but fix what caused the bug, and look to see where that faulty process may have caused other bugs. Google 'they write the right stuff' for a good article on it.
    - IsThisObvious 13y ago
      Another domain where you have things like this is embedded systems related to safety, ie, the micro-controller managing pressure equipment.
- windexh8er 13y ago
  Reading that it seems as if this space is almost, positively, stacked against any solo or small group dev. Thanks for the insight, but if crypto is hard and there's issues on both ends of the spectrum (commercial space generally ships for monetary reasons, small groups ship for notoriety / political discourse rationale / tinfoil hat. So who is building great crypto software that's built right from the ground up? OpenSSL AND OpenSSH struggle with this on a continual basis yet they seem to be the prime examples of doing it right. The problem, however seems to be unless every release is a complete rewrite eventually you will be working off of some foundational compromised base component. Thoughts?
  - tptacek 13y ago
    This space is positively stacked against any solo or small group dev. The way you deal with that is, you pick a credible crypto project with traction and build on top of it. The world badly needs a GPG UX that ordinary people can use without thinking about it. Bonus: nobody knows what that UX is! It's an open problem! And you can tackle it without getting anyone killed, because the GPG/PGP team has spent decades getting the "might get someone killed" problems addressed.
    - zipfle 13y ago
      If the user is placing their trust in the UX's ability to validate that PGP/GPG is working correctly and has not been tampered with, that sounds exactly like something that could get someone killed.
      - tptacek 13y ago
        You can say the same thing about any piece of software; if Excel forgets how to multiply numbers, think of the horrible things that could happen! The difference is that multiplying two numbers isn't hard compared to getting a cryptosystem right, and neither is accurately displaying a key fingerprint.
- ams6110 13y ago
  So what you're saying is, for cryptography software, "Agile" development doesn't work. For cryptography software you need very careful, deliberate, up-front design. You need careful, deliberate implementation that does not deviate from the design. You need careful deliberate validation. What we may be seeing is that we have roughly a decade now where "Agile" has been drummed into developers as the best way to write software. Perhaps it's the only approach they knew to take.
  - tptacek 13y ago
    That a much better way to put it than mine.
- chacham15 13y ago
  > The difference is between handling bugs and expecting bugs. Netscape expected bugs. Crypto software can't do that. I dont know if I believe this. I have never had a software system that was completely bug-free (independent of whether or not it had anything to do with crypto). Furthermore, I would argue that one of the main ideas behind SSL was to expect there to be flaws in the underlying crypto that it used and therefore be able to deal with switching out one algorithm for another. You might argue that there exists a difference between a bug and an algorithmic flaw, but realistically they both have the same effect. Therefore, I argue that the superior method is not to write code expecting no flaws, but to write code which does expect them and has a way to deal with them. That is not to say that you should release your code knowing that there exists a security flaw, but rather to acknowledge that you're not the smartest guy in the room and that the best way to harden a security system is by letting people attack it. Now, you cant force people who are good at crypto to attack your system in order to improve it. Hopefully though, security through obscurity will help. I.e. if your product is not well known it is not worth it for someone experienced to spend a lot of time to crack it. On the other hand, if it is, you then need to worry about the nature of the person attacking it. Is this person a researcher (in which case he will hopefully reveal his findings) or a threat? If you are a big product, you can hopefully afford to make this person a researcher who will give you his results. For the other case, I think that this is where auditing helps. Even if you have a broken system (which you can almost always safely assume you do), you can at least possibly detect a hack and then try to reverse engineer it. Lastly, to provide a solid example of 'bugs' which have been fixed in crypto code which everyone relied on, you need to look no further than unix's crypt.

All software has bugs. There's no piece of real-world general-purpose software that you can't look at and say "this was improved incrementally as bugs were found". But not all software has the same sensitivity. In the weeks before Netscape Navigator shipped in the mid-90s, Jamie Zawinski slept under his desk at work trying to make it reliable. The builds from the night before launch crashed! Nobody thought they'd make it. That's a story anyone on a commercial dev team can probably tell you; it's the oldest story in product development. That kind of incremental development works fine when you're shipping the first web browser, or even the first version of a new web browser. It does not work when you are shipping cryptography. Nate Lawson is fond of saying, "plan to budget 10x for the validation of a new cryptosystem as you to implementation". You probably can't actually be doing that when your whole cryptosystem changes 4 times in 2 years. The difference is between handling bugs and expecting bugs. Netscape expected bugs. Crypto software can't do that.

saosebastiao 13y ago
You haven't explained anything about how crypto software is developed. You have told us how other software is developed and then told us that it isn't okay for crypto software.
- tptacek 13y ago
  Spend 10x the resources on verification as on implementation. Spend 10x the resources on design as on implementation. Nobody does this with normal software.
  - jbri 13y ago
    And ultimately the biggest thing to do is that when you find a bug, don't just fix that one bug and call it a day. If you find a bug in your crypto code through your testing and validation, that means your development process screwed up and allowed that bug to slip through. What you need to do is figure out how that bug got through your development process, change your development process so it wouldn't slip through next time, and then have a thorough audit of your already-written code to make sure no other bugs have slipped through the same way. Developing must-not-fail critical software is the exact opposite of the traditional launch-fast-and-iterate model that most people use to produce web applications and such. This is a big reason that when web app developers try their hand at developing critical software like cryptography, they get it horribly wrong.
    - nekopa 13y ago
      This seems similar to way the people who developed the software for the space shuttle worked. Finding a bug was a huge deal, as it meant something was wrong in their process. It wasn't just fix the bug, but fix what caused the bug, and look to see where that faulty process may have caused other bugs. Google 'they write the right stuff' for a good article on it.
  - IsThisObvious 13y ago
    Another domain where you have things like this is embedded systems related to safety, ie, the micro-controller managing pressure equipment.
windexh8er 13y ago
Reading that it seems as if this space is almost, positively, stacked against any solo or small group dev. Thanks for the insight, but if crypto is hard and there's issues on both ends of the spectrum (commercial space generally ships for monetary reasons, small groups ship for notoriety / political discourse rationale / tinfoil hat. So who is building great crypto software that's built right from the ground up? OpenSSL AND OpenSSH struggle with this on a continual basis yet they seem to be the prime examples of doing it right. The problem, however seems to be unless every release is a complete rewrite eventually you will be working off of some foundational compromised base component. Thoughts?
- tptacek 13y ago
  This space is positively stacked against any solo or small group dev. The way you deal with that is, you pick a credible crypto project with traction and build on top of it. The world badly needs a GPG UX that ordinary people can use without thinking about it. Bonus: nobody knows what that UX is! It's an open problem! And you can tackle it without getting anyone killed, because the GPG/PGP team has spent decades getting the "might get someone killed" problems addressed.
  - zipfle 13y ago
    If the user is placing their trust in the UX's ability to validate that PGP/GPG is working correctly and has not been tampered with, that sounds exactly like something that could get someone killed.
    - tptacek 13y ago
      You can say the same thing about any piece of software; if Excel forgets how to multiply numbers, think of the horrible things that could happen! The difference is that multiplying two numbers isn't hard compared to getting a cryptosystem right, and neither is accurately displaying a key fingerprint.
ams6110 13y ago
So what you're saying is, for cryptography software, "Agile" development doesn't work. For cryptography software you need very careful, deliberate, up-front design. You need careful, deliberate implementation that does not deviate from the design. You need careful deliberate validation. What we may be seeing is that we have roughly a decade now where "Agile" has been drummed into developers as the best way to write software. Perhaps it's the only approach they knew to take.
- tptacek 13y ago
  That a much better way to put it than mine.
chacham15 13y ago
> The difference is between handling bugs and expecting bugs. Netscape expected bugs. Crypto software can't do that. I dont know if I believe this. I have never had a software system that was completely bug-free (independent of whether or not it had anything to do with crypto). Furthermore, I would argue that one of the main ideas behind SSL was to expect there to be flaws in the underlying crypto that it used and therefore be able to deal with switching out one algorithm for another. You might argue that there exists a difference between a bug and an algorithmic flaw, but realistically they both have the same effect. Therefore, I argue that the superior method is not to write code expecting no flaws, but to write code which does expect them and has a way to deal with them. That is not to say that you should release your code knowing that there exists a security flaw, but rather to acknowledge that you're not the smartest guy in the room and that the best way to harden a security system is by letting people attack it. Now, you cant force people who are good at crypto to attack your system in order to improve it. Hopefully though, security through obscurity will help. I.e. if your product is not well known it is not worth it for someone experienced to spend a lot of time to crack it. On the other hand, if it is, you then need to worry about the nature of the person attacking it. Is this person a researcher (in which case he will hopefully reveal his findings) or a threat? If you are a big product, you can hopefully afford to make this person a researcher who will give you his results. For the other case, I think that this is where auditing helps. Even if you have a broken system (which you can almost always safely assume you do), you can at least possibly detect a hack and then try to reverse engineer it. Lastly, to provide a solid example of 'bugs' which have been fixed in crypto code which everyone relied on, you need to look no further than unix's crypt.