11 ms·
All the crypto code you've ever written is probably broken
- tptacek 14y agoNACL and Keyczar are two viable alternatives to DIY crypto. I'd use NACL before I considered trying to DIY an AEAD secure transport; there are more things to get wrong than properly authenticating your data! You are always better off not working directly with crypto at all. For that, consider using PGP/GPG for data-at-rest (if you're in a JVM language, you can use Bouncycastle to get access to PGP) and SSL/TLS for data-in-motion. In our practice, password reset tokens and encrypted session cookies continue to be the top source of exploitable crypto vulnerabilities in web applications. You don't need encryption to build either of these features; send 128 bit random numbers that key a database row instead.
- Jabbles 14y agoCould you explain why "encrypt then MAC" is the right option please? It isn't obvious why the other two are broken. Luckily I don't work in crypto :)
- tptacek 14y agoIt allows you to use MAC validation as a screen for incoming messages and so prevents attacks that depend on ciphertext manipulation.
- dchest 14y agocperciva wrote a blog post about this: http://www.daemonology.net/blog/2009-06-24-encrypt-then-mac.html http://www.daemonology.net/blog/2009-06-24-encrypt-then-mac....
- forgotusername 14y agoIf I'm not mistaken (and haven't read up on this stuff in years, so probably), the majority of encryption modes rely on XORing the stream of bits from the cipher with your plain text. In that way, both sender and receiver need only generate the same cipher bits and apply XOR to encrypt and decrypt (meaning encryption and decryption are actually identical operations!). A side effect of XOR is that a single bit flip in the ciphertext corresponds exactly to a single bit flip in the cleartext. An attacker with knowledge of your cleartext can therefore modify it without ever needing to know the cipher parameters. Imagine a session cookie that contains a single 32bit integer, the user ID. Now attacker knows his user ID, so he merely needs to XOR the cookie with his ID, then XOR it again with his desired ID and voila admin privileges. Wrapping the cookie in a MAC prevents this kind of manipulation.
- tptacek 14y agoI'm not sure I totally follow this (you seem to be talking about an attack on CBC mode, but the mode you're describing sounds more like CTR mode), but a good rule of thumb is, without explicit authentication, attackers can alter and often rewrite messages even though all they can see is ciphertext. But there are even more problems than that with unauthenticated encryption. If you don't authenticate there is a good chance attackers will be able to decrypt your messages wholesale.
- forgotusername 14y ago> decrypt your messages wholesale. Eek, that sounds fun :) Tell us more?
- Nursie 14y agoIf they can get your system to tell them if a message is valid somehow, perhaps by making thousands of attempts to pass a message and noting where it says 'login failed' or '404' instead of invalid message (for instance) then there are all sorts of things that can be done to recover messages and keys. I highly recommend Dan Boneh's crypto 101 on coursera for anyone that has the time.
- tptacek 14y agoThe CBC padding oracle is one such attack. There are a bunch of similar ones. They're "chosen ciphertext" attacks. Again, even if you get this part right, there are other things that go wrong. TLS is authenticated, and it fell to two adaptive chosen plaintext attacks because of two different implementation details they messed up. And no public cryptosystem in the world has been as thoroughly tested and analyzed as TLS.
- pjscott 14y agoThere's an entertaining article about one such attack here: http://blog.cryptographyengineering.com/2011/10/attack-of-week-xml-encryption.html http://blog.cryptographyengineering.com/2011/10/attack-of-we... For some mind-boggling reason, the designers of the XML Encryption standard decided to make authentication optional, so an attacker can simply avoid sending an incorrect MAC.
- agwa 14y agoMoxie Marlinspike wrote a great blog post about this: http://www.thoughtcrime.org/blog/the-cryptographic-doom-principle/ http://www.thoughtcrime.org/blog/the-cryptographic-doom-prin...
- cdjk 14y agoShort answer - it lets you verify the data before decrypting so you can reject invalid data earlier.
- femto113 14y agoMissing from this blog post, but made clear here (http://blog.cryptographyengineering.com/2012/05/how-to-choose-authenticated-encryption.html http://blog.cryptographyengineering.com/2012/05/how-to-choos...) is that you need to USE TWO DIFFERENT KEYS, one for the cipher, the other for the MAC. This makes it practically impossible to forge a valid message, since even if someone forged an apparently valid cipher section they'd be unable generate the corresponding MAC (unless they already had your keys, in which case you're already toast).
- marshray 14y agoUsing two different keys is considered good practice, but I don't think it helps as much as you seem to think it does.
- tptacek 14y agoNot to mention: if you're doing it right, you really don't have to think about whether you have two keys; you just naturally kind of do. It's the systems where you consciously have to handle two different keys that you have to worry about, because they're probably not keying themselves safely to begin with.
- sweis 14y agoHere's a slide deck by Bellare talking about some of the differences between encrypt-and-MAC, MAC-then-encrypt, and encrypt-then-MAC: http://cseweb.ucsd.edu/~mihir/cse207/s-ae.pdf http://cseweb.ucsd.edu/~mihir/cse207/s-ae.pdf
- marshray 14y agoThe simplest way to think about it is during the receiving process: checking the MAC before decrypting catches and rejects evil messages earlier so there are fewer things that can (and will) go wrong. Even if you tell the world that the message wasn't invalid, you don't reveal any new information to the attacker. OTOH, if the receiver must decrypt before checking the MAC, any information leaked to the attacker (success or failure, timing, etc) is very likely to give the attacker a systematic method to decrypt some or all of your secret plaintext.
- dchest 14y agoBTW, there's NaCl port to Go in the official go.crypto repository: http://go.pkgdoc.org/code.google.com/p/go.crypto/nacl/ http://go.pkgdoc.org/code.google.com/p/go.crypto/nacl/ If you use Go, instead of hacking together things with AES from the standard library to encrypt something, just `go get code.google.com/p/go.crypto/nacl/secretbox` and call secretbox.Seal with unique [randomly generated is okay] nonce and your key to get the encrypted and authenticated "box". secretbox.Open with the same key and nonce will open it. Bonus: on amd64 it's extremely fast (portions are written in assembly, and even pure Go versions for other platforms are hand-optimized), faster than what you can get from any non-hardware AES implementation (let alone authenticated with HMAC), and provides better security margin.
- jamwt 14y agoWe ( http://bu.mp/company/ http://bu.mp/company/ ) used NACL client and server side when we built our new app ( https://theflockapp.com/ https://theflockapp.com/ ). It's a pretty great library--a shame it's not more widely used.
- lifeisstillgood 14y agoIn our practice, password reset tokens and encrypted session cookies continue to be the top source of exploitable crypto vulnerabilities in web applications. You don't need encryption to build either of these features; send 128 bit random numbers that key a database row instead. That's what I wanted to hear - thank you.
- languagehacker 14y agoWell thanks for scaring my pants off, Tony.
- Nursie 14y agoHmm, well it's a bit sensationalist, but not wrong. It's not exactly news either. Is the guy going for hits on his first blog entry or somesuch?
- bascule 14y agoFirst entry on the svbtle network sure. I've blogged extensively here before http://unlimitednovelty.com http://unlimitednovelty.com And while it shouldn't be news to anyone using crypto, I still feel like it's a poorly-known subject.
- Nursie 14y agoOh certainly. I learned a lot by doing the coursera course on it earlier this year including an intro to a whole new area of mathematics. Some sort of intro to crypto techniques and uses really ought to be mandatory for a lot of devs, certainly anyone tempted to use anything other than a well-coded TLS API or pre-provided GCM interface. --edit-- In fact it's also very important that even when you do know what you're doing (to a greater extent than the entirely uninitiated), you don't implement this stuff yourself, because there are more attacks than you can possibly imagine.
- bascule 14y agoSee my multiple "(and even then!)" caveats ;)
- tptacek 14y agoIn my experience it's news to something like 90% of all developers, who can generally be counted on to say things like "we used AES so not even the US military can break our encryption".
- ricardobeat 14y agoAcronym overdose. So, why should I use authenticated encryption and how can I use it on Node.JS/Rails/PHP/etc?
- tptacek 14y agoBecause if your ciphertext isn't authenticated attackers can modify it to launch adaptive attacks against you. Also, obviously, they can potentially rewrite messages. This is one of those situations where if you have to ask why you'd use authenticated encryption or how you'd do it, you simply shouldn't be using cryptography primitives directly at all. Use PGP/GPG instead.
- ricardobeat 14y agoGood, so the fear mongering is only targeted at crypto library authors.
- tptacek 14y agoHuh? Fear mongering? Crypto library authors? If that's what you think this post is about, stay away from crypto. I don't mean that offensively. Most developers should stay the hell away from crypto. They all know intuitively that they shouldn't be hacking together, say, kernel modules. But since crypto comes in library form, they assume it's safe to build with it. It isn't.
- ricardobeat 14y agoI wouldn't touch cryptography code in production with a long stick. Thing is the headline all crypto code you've written is broken is sensationalist and doesn't apply to 90% of developers, who "write crypto code" using third-party libraries.
- tptacek 14y agoNo. The "third party libraries" you're talking about are, from my experience (my whole job is looking at random applications deployed in production for problems like this) things like OpenSSL and crypto++ and CryptoAPI and Common Crypto. Tony isn't complaining that developers implement the AES cipher core themselves and screw it up, or even that they're implementing their own block cipher mode code. He's saying that even when you use third-party libraries, if you haven't chosen a library that is specifically designed to avoid mistakes, you are boned. And he's absolutely right about that. Also, from experience: virtually nobody uses the good high-level libraries. What library did you use last time you built a feature, like a password reset token, that relied on crypto?
- krapp 14y agoWell crapbaskets. Authentication and crypto are areas where I know I'm nowhere near good enough to tell if what i'm doing works or not, other than trying to find out what the best practices happen to be and use what seems to have a good reputation. But the only constant in this field seems to be that everything I thought I knew was fatally horribly flawed from the start and that I might as well have just stuck all of my passwords in a plaintext file called "passwords.txt" because the only thing keeping my site from getting rooted was just having a terrible site nobody ever goes to. Still this should be an informative thread. Crushingly depressing and informative.
- noonespecial 14y agoCrypto: If you don't know exactly what you're doing, you're not doing it at all, no matter how hard you try. It's very hard to know if you know. It's so hard that even when you try to "leave it to the experts" you often do that wrong as well. There seems to be no other place in programming where best practices are more important to study and observe.
- anonymouz 14y agoIt's very good that the author is trying to get the word out, but it would help tremendously if he tried to at least give some indication (or links to further material), of why certain modes are insecure. The people who already know this information don't need this blog posts, and the ones that don't get no information that helps them to understand the problem. I think stating what can go wrong with, for example, ECB, would make it much easier for people to remember which mode is good and which is bad. As it is at the moment one would have to remember a few cryptic acronyms as "good" and others as "bad", without any context. (Or go look it up somewhere, but the people looking these things up are not the ones in need of being alerted...)
- tptacek 14y agoIf you have to explain to someone why they shouldn't be using ECB, it's hopeless. Instead, you need to be educating them as to why they should be using someone else's whole cryptosystem, like NACL or Keyczar or PGP. Crypto is a place where a little knowledge is often harmful. You tell someone not to use ECB, so they use CBC instead. Now they're vulnerable to a bunch of new practical attacks. You list those off, and they switch to CTR, and their stuff is as trivially decryptable as simple XOR encryption. What's the point? It's better to give people the (honest, accurate) impression that they don't know anything resembling enough to build a cryptosystem so they won't try, fail, and screw over their users.
- anonymouz 14y agoI think educating people about some pitfalls and then recommending them a "full package" solution will stick better. For example, for ECB it's trivial to explain, that it allows reordering of encrypted blocks, an attack that someone not familiar with cryptography has probably never thought about. Of course they won't gain a full understanding of cryptography in this way, but they will hopefully start to appreciate how difficult it is to get right, and how many attacks there are that they have never thought about. This hopefully includes an understanding that possibly other modes that they think would be safe might also have problems that they have never considered, and that they should really use a full package solution. I think this will drive the point home better than, say, listing three opaque possibilities on how to use a MAC and then saying "no 1 is the right one", without any explanation whatsoever.
- benregenspan 14y agoThis post does not go at all into the "why" of authenticated encryption. If you're interested in more of the reasons for authenticating, including actual attacks it protects against, I found http://blog.cryptographyengineering.com/2012/05/how-to-choose-authenticated-encryption.html http://blog.cryptographyengineering.com/2012/05/how-to-choos... to be a good explanation of this.
- digsmahler 14y agoCongratulations on your first blog post! It looks pretty good, very attention grabbing. Please follow up with some more focus on the positives. I will be very interested to read more information on the correct way to implement crypto in all its subtle contextual glory.
- wonnage 14y agoSecurity is a series of tradeoffs. Remember the Debian SSL fiasco? Turns out even when every Debian/Ubuntu user's SSH keys were essentially useless, the world did not explode. Or consider the state of the internet in the 90-early 2000s - security was basically a joke, article author would've bust an artery raving about all the ways YOU ARE PWNED RIGHT NOW. Oh, and it looks like people are okay with Dropbox even though it occasionally fails horribly and lets you see other peoples' files. It turns out for the vast majority of use cases, even broken crypto is Good Enough. That has to do with two things. For you to be worrying about initialization vectors and block modes you need to first have a reason to be working at that level - answer "no" for most webapp use cases. Secondly, and ignored by security fearmongerers: you need to have something of value commensurate with the efforts you've put toward security. This is the mindset of a cryptography professional: true, chances are what you're using is broken. But whether that's a problem depends on who you are. If you're the NSA, you might be concerned with crazy shit like China building quantum computers and factoring all your primes (I'm making this up). Whereas you might be the latest social.ly startup, your users have no privacy anyway, security is more of an image concern (it looks bad to be on the front page of HN with security bug) than a real one.
- tptacek 14y agoNo, it does not turn out that for the vast majority of cases, broken crypto is good enough. It turns out that for the vast majority of cases, broken crypto schemes don't have enough users for the tiny minority of software security people who look at crypto to bother beating up. Once one of those systems becomes popular, crypto pentesters finally get around to poking at them, and, lo and behold, thousands of users are discovered to have been communicating effectively in the clear for years. It's one of the more pernicious evil memes in our field that "if you're not the NSA, everything you do is broken, so just try the best you can". No. The attacks we're talking about take 50-200 lines of Ruby code and less than 30 seconds to run. There's no excuse for being exposed to them. We mercilessly mock people who screw up SQL queries, but then act like it took a network of intelligence service supercomputers to break our amateurish cryptosystems. We'd all be better off if fewer people tried their hand at building these systems in the first place. Use PGP, or Keyczar, or NACL. If you're typing the letters "A-E-S" into your code, you're doing it wrong.
- acabal 14y agoOK, I get it. Encryption is hard. But just look at that post--there's so many acronyms and tech-talk that my head is spinning and I'm not halfway through. How can you expect anyone to do encryption properly when it's presented like that? In fact I'd say this post is literally useless FUD for anyone without a crypto background. It's unreadable unless you're an enthusiast. Worse still is the huge amount of conflicting advice and methods available on the internet. If you had asked me today if AES-256 was the best there was, I would have said yes, because that's what I've read, and I'm not a crypto expert; but now this posts makes it seem like I'm wrong. The truth is that now I have no idea. Crypto is hard, but it's so important today that it shouldn't be presented in such an obtuse way. There should be one resource that in big, bold headings and simple sentences outlines the best available crypto methods for each particular use case. Hashing password? Bcrypt. Asymmetrically encrypting plaintext? Algo XXX. Etc. Bonus points for step-by-step instructions on how to use those methods in common programs like GPG. Without a human-readable, trusted, official resource, it's just going to be crypto scientists shoving acronyms down our throats till we don't care any more. Please simplify it and eliminate the bad choices for us, because you know better! I want to be as secure as possible, but it's outrageously hard to do with the way things are presented today.
- tptacek 14y agoThat's the point. The post is hard to read because the subject is hard. If that post --- which is pretty breezy --- is too hard for you to grok, the message you should be taking away with it is "I shouldn't be designing systems with crypto". The headline is good enough for you.
- acabal 14y agoYeah but how do I use systems with crypto? What if I want to encrypt some text? Do I worry about a MAC? Does GPG do MACs? How do I know? Do I worry about EAX? GCM? CCM? AES-CTR? CMAC? OMAC1? AEAD? NIST? CBC? CFB? CTR? ECB? Programmers have to use crypto libraries all the time. The 3rd sentence in this post is "Chances are whenever you have tried to use a cryptographic library you made some sort of catastrophic mistake." Crypto is hard, but humans have to use it--this post is ridiculously complex and utterly unhelpful for anyone who's not already an expert, and that's not uncommon in crypto documentation, I think. That's not how it should be. Something so important and widespread should be written about and explained in a human manner. My point is we need some resource to de-obfusticate the technobabble for those of us who need security but have day jobs that aren't developing the latest and greatest hash algorithm.
- Imbue 14y agoWhy is a proper MAC preferred over simply encrypting the plaintext after appending a checksum or hash.
- tptacek 14y agoAmong other things, because the system you just described is vulnerable to chosen ciphertext attacks.
- Imbue 14y agoBut if someone chooses a ciphertext (without knowing the key), how could they get the checksum right (which is also encrypted)? If the checksum was wrong, the system would refuse to decrypt the corrupted message.
- tptacek 14y agoNo. There's no such thing as "refusing to decrypt a corrupted message"; cipher cores either produce an expected plaintext or, if the message is corrupted, an unexpected plaintext. Attackers can use changes in behavior based on the differences between different unexpected resulting plaintexts to infer the original plaintext.
- serge2k 14y agoDoesn't encrypt then mac allow you to reject messages prior to decryption?
- tptacek 14y agoYes. That's not what we're talking about here, though.
- Imbue 14y agoMy question was why is a proper MAC better than appending a checksum to the plain text and then encrypting. With an appended and encrypted checksum, the system could easily reject corrupted messages, that's the whole point of a checksum in the first place. So why is a proper MAC better than appending a checksum or hash to the plaintext and then encrypting? Or maybe I am misunderstanding something?
- pfortuny 14y agoThe real problem is the lack of a REAL crypto API which does not suck as an interface for security and with which, <b>using default parameters</b>, you get good security. That is the problem. All the ones I know are too low level. That is an engineering problem: there are the pieces but there is no real engine. But this is my biased and humble opinion after using OpenSSL. I am not advocating a 'just works one way' API. I am asking for a real engineering effort to create a 'dumb people can use this safely as long as they follow the simple instructions'. If you (developer) need to know why encrypt+mac != mac+encrypt, then the security engineers have not done their job. If you need to know the difference between ECB and CBC, the security engineers have not done their job. If you need to know about the IV in AES, then ... (just repeat).
- steve8918 14y agoNot that it will avoid errors on the part of the implementor, but if you do a thorough reading of the classic "Applied Cryptography" by Schneier, he pretty much covers all these problems. He covers all the issues such as ECB vs CBC, IVs, HMACs, etc. It's the definitive book on the topic and anyone who is interested in cryptography should buy this book.
- tptacek 14y agoNo, he famously and openly admits to not doing this. _Applied_ is not a definitive book, and _Practical Cryptography_ (now _Cryptography Engineering_), which he co-authors, starts out with a disclaimer to this effect.
- steve8918 14y agoInteresting, I'm admittedly a few years out of date, the last time I used the book heavily was about 10 years ago... I'll have to pick up this new(er) book.
- jacquesm 14y agoYou could make a much simpler and more powerful statement: All the code you've ever written is broken. Since cryptographic code is harder than most code and getting simple stuff right is impossible there is an extremely small chance that you'd get crypto code right 100%. With most code the consequences are not all that bad, with cryptographic code the consequences can be disastrous.
- tomjen3 14y agoHyperbole. It depends on what you are doing, what you are encryting for and what you do with the data.
- chuppo 14y agoI passed all the question, now what?
- zx2c4 14y agoSurprised OP didn't mention OCB which has advantages over his suggestions: http://en.wikipedia.org/wiki/OCB_mode http://en.wikipedia.org/wiki/OCB_mode http://www.cs.ucdavis.edu/~rogaway/ocb/ocb-faq.htm#what-is-ocb http://www.cs.ucdavis.edu/~rogaway/ocb/ocb-faq.htm#what-is-o...
- salem 14y agoThe tone is not very constructive. I would say, at the very least you should read "Applied Cryptography" by Bruce Schneier and the most recent NIST recommendations.