11 ms·

You are dangerously bad at cryptography

268 points by alinajaf 13y ago

davidw 13y ago
Saying stuff is hard is easy, and as far as I can tell, very correct about cryptography. More useful would be actual suggestions to enable average developers to do more things with some degree of confidence. In other words, libraries, recipes and other documentation, and so on. "Hire an expensive security guy" is probably not a feasible solution for many small startups, nor very scalable in any case. Also, out of curiosity, in his timing attack example, the difference in time caused by the string being equal seems like it'd get absolutely swallowed up by the random nature of the universe - do those things actually work in the real world, on real servers with varying loads and numbers of users and network traffic?
- alinajaf 13y ago
  Hi, author here. I would have liked to have made useful suggestions, but having made so many errors with cryptography in the past, I don't really feel qualified to do so. I do however recommend getting in touch with tptacek and co and doing the crypto challenges. I mentioned this at the bottom of the post. Edit: re: timing attacks, it helps if you're in the same data centre. I've never personally implemented a timing attack, so I'm hesitant to make any assertions further than that either way.
  - GhotiFish 13y ago
    I am doing this. It looks fun.
  - bazzargh 13y ago
    btw: isn't the scheme you posted vulnerable to replay attacks too? Much easier than the timing attack. edited to add: don't mean this as a nitpick. I've seen that very mistake made by two S&P 500 companies that had 'homebrew' SSO we had to integrate with.
    - alinajaf 13y ago
      > btw: isn't the scheme you posted vulnerable to replay attacks too? Much easier than the timing attack. Yep, probably. In fact now that I look at it definitely, I should have gone with that instead! I forget who said this but basically any feature that exists in TLS that doesn't exist in your hand-rolled authentication scheme is a vulnerability.
      - mcintyre1994 13y ago
        Does peer review count as a feature?
- moakakala 13y ago
  >Also, out of curiosity, in his timing attack example, the difference in time caused by the string being equal seems like it'd get absolutely swallowed up by the random nature of the universe - do those things actually work in the real world, on real servers with varying loads and numbers of users and network traffic? You could make each request many times, and then average them together. I don't know how many requests you'd have to make to overcome the random fluctuations though -- probably a lot.
  - zackangelo 13y ago
    Also, this random behavior will be significantly amplified if the host is running inside of a VM.
schabernakk 13y ago
When he is saying >Save yourself the trouble. Don't use cryptography. It is plutonium. There are millions of ways to mess it up and precious few ways of getting it right. I am sure he means don't use your own homegrown cryptography solution. Use something established and well tested. Good advice, probably can't be repeated enough. I liked the exampled he gives, I didnt know for example that you can basically extend an MD5ed string and keep the original MD5 value. Then again, I know that MD5 isn't a secure cryptographic hash function so I wouldn't have used it from the start. Nice to know why thats the case.
- normalfaults 13y ago
  Is there a repository or central location of the established and well tested solutions for developers to use? Where does one start?
  - fredsted 13y ago
    Wikipedia perhaps?
  - Osmium 13y ago
    Some ideas, depending on what you need (I can't say how good these are): http://www.keyczar.org http://www.keyczar.org http://nacl.cr.yp.to http://nacl.cr.yp.to https://github.com/jedisct1/libsodium https://github.com/jedisct1/libsodium http://www.gnupg.org http://www.gnupg.org
    - normalfaults 13y ago
      Thanks. These are straight crypto frameworks which are a good start.
  - betterunix 13y ago
    One of the problems with cryptography is that "solutions" are not as generally applicable as one might want. There are a lot of assumptions that are made, and violating those assumptions is usually a disaster. For example, when it comes to encryption, it is typically assumed that all messages are the same length. If your application does not make this guarantee, encryption may not provide you with any security. You could wind up in this situation: https://news.ycombinator.com/item?id=2661890 https://news.ycombinator.com/item?id=2661890 Think of it this way: asking for a list of tried-and-true crypto solutions is like asking for a list of tried-and-true database schemas. There might be cases where it will work, but for the most part you need to put some thought into what you are doing.
e1ven 13y ago
So, sure. I'll admit that I'm bad at cryptography. You win! But the problem is, the world needs more, not less, crypto. We need to integrate crypto into more places, not run and hide and declare it too hard every time we come across it. Is it easy to screw up? Sure. So is manually allocating memory. But we use higher level languages to help protect us from ourselves. So too can we use higher-level libraries to help protect us from some mistakes we can make with crypto. I'm working on an app which uses crypto heavily- In initial versions, I used manual RSA keys, but I recognize that no matter how smart I think I am (not very), it's easy to mess stuff up. I can screw up the padding, I can screw up the implementation. Other clients which interop with me can screw up... So Instead, I've swapped to using GPG keys. Is it perfect? Heck no! But it reduces the code that I have to get right. It outsources the parts that are most likely to be screwed up to a place where they're more likely to be reviewed, more likely to be written by non-fuckups. There are lots of higher level libs, gpg is just one of them. keyczar, etc are also ways to solve this problem. But we shouldn't run in fear everytime someone mentions the word Crypto, just because we have a chance of messing up.
- betterunix 13y ago
  What we really need is a cryptography language, to do for cryptography what SQL does for databases. I should be able to say, "Messages have this form, and they need to be encrypted end-to-end with authentication and without surreptitious forwarding," and have the right components assembled for me. Unfortunately, our understanding of cryptography is not well-developed enough to create such a system, and so we still fumble around and usually produce systems that are vulnerable (did you remember to have a good source of entropy? did you make sure your messages are all the same length? does your system compose well with others?).
  - EthanHeilman 13y ago
    Even if we had such a language it would probably not be high level enough because understanding what security you know is still a hard problem. Instead I'd suggest a high level library that gives you a complete security package with defaults that can't be easily changed. Something like what the CAESAR competition hopes to achieve: http://competitions.cr.yp.to/caesar.html http://competitions.cr.yp.to/caesar.html
moakakala 13y ago
> This means that as long as you have one example of a signed message, you can forge signatures for that message plus any arbitrary request parameters you like and they will authenticate under the above described scheme. If all requests are made over HTTPS, how could a third party intercept a signed message? How is this any greater of a risk than a third party intercepting user login information? (This is a serious question; I'm not being flippant or saying 'gotcha')
- jd007 13y ago
  If you do not manage SSL certificate properly (trust only your server's certificate), a MITM attack can break the system easily. Otherwise I don't think you can intercept the signed message if everything is over TLS.
  - moakakala 13y ago
    Is the risk any greater with APIs (like described in the article) compared to typical username/password login systems?
    - _phred 13y ago
      A lot of API authentication is half-assed, like the examples in the article. "OAuth is hard, roll your own" is a common approach. Even with, e.g., OAuth 2, who's to say that the scheme is completely safe and that your implementation is correct? As far as API vs. user account, it depends on the loot. An API might let me do more damage faster, or subtly lurk and alter/steal data over time. It might also be harder to detect from the UI, no "last logged in" giveaway. Also, some API vendors recommend disabling SSL cert validation client side. Even for credit card gateways, unbelievably. Since it's a script talking to a script, no one is going to see the cert problems from a MITM until it's too late.
  - _phred 13y ago
    Right. If the attack vector is "break SSL" I'm going to try some other attacks first. There's an underlying assumption in the question: my app (and everything else hosted on the box) is safe from XSS, CSRF, injections, and other information leakages. Is it really? How do I know for sure? And who's to say that your forum server (for example) is just as secure? That could be a foothold into your environment too. And let's not forget social manipulation of your staff and users. Maybe I'll just steal the machine in question, or your laptop. After I try all those avenues, I'm either finding another target or ramping up for a protracted attack on your SSL connections. If your site attracts this dedicated of an attack, you'd better get that high paid security consultant. ;)
geon 13y ago
Then there is the question; Is your app likely to ever be attacked at all? By someone good? With a lot of time? If not, perhaps hiring a cryptography expert is overkill, and you are better off just using whatever your framework offers, or integrate some library yourself. You are not a bank.
- scott_karana 13y ago
  The message extension attack is pretty well known by crackers, which could leave your webapp open to pre-scripted kit attacks that simply crawl the web.
- mcdigman 13y ago
  This is a very dangerous question. Thinking like this leads to huge security vulnerabilities in important systems. A better question is "What is the worst thing that could possibly happen if this system were successfully attacked?" If the answer is anything bad, it is probably best to assume someone will try to attack it. As for the second part of the question, are we talking about the same internet? Because the one I'm on appears to be full of technically savvy people with loads of free time.
jd007 13y ago
About the timing attack on HMAC that the article mentions. It takes thousands if not hundreds of thousands of requests to gather the data (and let's assume you can indeed extract the data out of all the noise cause by network latency etc...), and any properly designed API should have a throttle measure built in to prevent brute force attacks like this. A good, secure API is protected by a variety of measures, not just through the request authentication component. Of course you should still make your auth as secure as possible.
- teeja 13y ago
  "a throttle measure built in" It's my understanding that bittorrent sync uses the latency of the network for such a throttle. Wild hair: a cyclotron-style router roundabout could hold millions of packets "in suspension" for n seconds.
- teeja 13y ago
  "a throttle measure built in" It's my understanding that bittorrent sync uses the latency of the network for such a throttle. Wild hair: a cyclotron-style router roundabout could hold millions of packets "in suspension" for n seconds.
megrimlock 13y ago
His insight that crypto is hard because you don't get feedback when you mess up is good. It made me wonder what other domains are like that -- domains where correctness seems within reach, yet there are subtle aspects that are hard to state. Some that come to mind: * Concurrency. It's easy to introduce race conditions, livelocks, or deadlocks without even knowing. They are often difficult to observe or reproduce, and it takes a lot of hard-won experience to become sufficiently critical of your own incompetence. On the other hand, the tools are improving here (helgrind, relacy, threadsanitizer). * Exception-safety. I don't mean in the weak sense of "won't leak resources" but in the stronger sense of leaving the system in a consistent state after an exception. Again, it's difficult to analyze or induce all the different code paths that might occur, and difficult or expensive (depending on your starting point) to roll back your mutations. Again, there are tools that help with this, like STM or cheap persistent data structures (so you can do a bunch of mutation that is only committed atomically at the end). Are there others? Can the progress in these domains shed light on how to make crypto safer?
- LaGrange 13y ago
  Outside of computing: health and diet. Sure, you get some feedback, but you also can screw up things in some ridiculously slow-acting way. Pharma companies (are supposed to) monitor drug use for decades after introduction for the same reason. Construction. Just build that chunk out of two steel beams instead of one, and witness it work perfectly until someone decides to throw a party on that balcony. Or witness your construction slowly turn itself from a perfectly normal tower to a major tourist attraction over the centuries. Parenting. Things like sudden stimuli in early life have been linked to ADD, the way you do attachment can have tremendous consequences, and sometimes singular events you barely have any control over can have unpredictable consequences later in your life. In fact, I'm tempted to say "life in general".
  - jiggy2011 13y ago
    Economics is also an excellent example. Nobody knows how to do it properly, even establishing direct cause and effect is basically impossible so we just iterate through progressively less broken solutions.
cllns 13y ago
(Un)conscious (in)competence reminds me of Donald Rumsfeld's trichotomy of known knowns, known unknowns and unknown unknowns. (highlighted here: http://www.youtube.com/watch?v=2Ex8EEv-WPs http://www.youtube.com/watch?v=2Ex8EEv-WPs)
idupree 13y ago
How is the shared secret generated, stored and shared? (There are lots of ways to do this wrong, and the code and algorithm for that were not even shown.)
tveita 13y ago
Unless I'm missing something, the example code will just calculate the checksum of "key:value&key:value&key:value", so actually anything with the same number of parameters will pass. Assuming "key:value" is supposed to read "#{key}:#{value}", this may be vulnerable to a delimiter attack -- you couldn't tell the difference between {foo: 'bar', bar: 'baz'} and {foo: 'bar&bar:baz'}.
- alinajaf 13y ago
  You are of course correct. Thanks! Will fix and rollout once traffic from HN/reddit dies down.
saosebastiao 13y ago
I'm bad at crypto. I'm well aware that I'm bad at crypto. In fact, I would venture to guess that I know next to nothing about it. The problem isn't that I don't know what I don't know...the problem is that I don't know what (or whom) I can trust. And that is the bigger problem with crypto. It would be nice if we had resources that could tell us not only the best practices, but what their downfalls are and their relative difficulties of cracking. Some common sense tells me that there is a combination of techniques that are simple and robust enough that I would have to worry about phishing before I had to worry about better security. But I don't know what that combination is, and I don't know if I can trust anybody that tells me.
- alinajaf 13y ago
  Author here. A good place to start (mentioned in the article). http://www.matasano.com/articles/crypto-challenges/ http://www.matasano.com/articles/crypto-challenges/ Never met any of them in real life, but they seem like good people to me. I've actually used knowledge I picked up from those exercises to fix vulnerabilities in past clients code, and I'm not yet half-way through them.
- epsylon 13y ago
  Try Dan Boneh's crypto course on coursera [1]. It covers quite a lot of ground, both practical and theoretical, and includes programming exercises similar to the matasano puzzles. Without a doubt it's one of the best courses of the dozen MOOCs I've taken. There's also a followup course [2] (I haven't taken it yet personally, but I believe the currently scheduled run will be the first). Interestingly enough, there's also an upcoming security course (with no date planned yet) which will cover the application programming part of security and will be co-taught by him. [3] [1] https://www.coursera.org/course/crypto https://www.coursera.org/course/crypto [2] https://www.coursera.org/course/crypto2 https://www.coursera.org/course/crypto2 [3] https://www.coursera.org/course/security https://www.coursera.org/course/security
  - cryptbe 13y ago
    Glad that you like Dan Boneh's crypto class. I made the programming exercises :-).
almost 13y ago
I didn't know about Matasano's Crypto challenges. Looks like fun, going to give it a try! http://www.matasano.com/articles/crypto-challenges/ http://www.matasano.com/articles/crypto-challenges/
ams6110 13y ago
I have implemented an MD5-based scheme similar to what was described. At the time, MD5/hash extension attacks were not as well-known as they are today, at least I had not heard of them and I read up on MD5 before I designed the sytem, so another worry is that what is considered secure today might not be secure tomorrow.
- alinajaf 13y ago
  > another worry is that what is considered secure today might not be secure tomorrow If history is anything to go by then what is secure today will definitely be insecure tomorrow. No real solution to this AFAIK. If vulnerabilities in the crypto itself doesn't get you eventually, quantum computing will.
  - Dylan16807 13y ago
    Quantum computers can only solve certain types of problems efficiently. Hashing is not generally in that set.
lifeisstillgood 13y ago
I completely agree with this and have just released a session-cache for python that completely ignores encryption and just stores uuid in a cookie, and relies on server side lookups for session work.
- kbaker 13y ago
  Make sure you are using HTTPS exclusively so you aren't vulnerable to Firesheep-style attacks.
  - lifeisstillgood 13y ago
    That's in the docs :-)
  - lifeisstillgood 13y ago
    That's in the docs :-) Ooops no it's not - kind of assumed it was obvious - thank you for the reminder :-)
wzdd 13y ago
I'm just curious: the obvious response to the first problem (message extension attack) is to include the message length in the message. If this wouldn't work, could an expert please tell me why? Edit: To be specific, I was thinking of a Pascal-style length-at-start arrangement. It's clear that if you put the length at the end you may be vulnerable to exactly the same attack!
- bazzargh 13y ago
  I'm not an expert, but I can see a problem with schemes along those lines. MD5 (and similar) extension attacks also take advantage of the padding added after the thing you want to hash; so eg Secret: secret Message: foo&bar Let's include the length and hash 'secretfoo&bar13' What you actually hash is 'secretfoo&bar130x80...padding...0x20'. What the attacker sees as the 'known hash' includes this padding. When you perform an extension attack, the attack message would look like: secretfoo&bar130x80...padding...0x02&evil . At this point you're adding on the length and hashing it, so what you compare with is 'secretfoo&bar130x80...padding...0x02&evil93' ... but the attacker can precompute that hash just as easily as the one with just 'evil' appended. You could argue that your hashes would be constructed a bit differently (eg make messages that appear to have internal padding illegal; still vulnerable, btw), but the point is - you're still tinkering with the same basic structure, and are likely to make more mistakes. You're better off switching to HMAC for this application.
  - deleted 13y ago
    [deleted]
- jdbernard 13y ago
  The attacker just needs to include the updated length, which he can likely calculate since the shared secret is probably the same length for all accounts-even if it is not he only needs it for this request--and he knows the length of the parameters and their values. It would take some guesswork and trial and error to nail it down, and the attacker would have to guess that the length was included in the first place. Assuming he has plenty of time and processing power it is not infeasible. There are a lot of fairly easy things you could add to make it more difficult to attack.
hga 13y ago
Hmmm, one of the things I've noticed is that if you've studied pre-web cryptography extensively (as I did, starting even before I started programming), "Everything you know is wrong". Or thereabouts. The HMAC timing attack illustrates this well, "classical" systems as I will call them, e.g. Enigma and how it was used, are not subject to zillions of known plaintext attacks (although I've read one bit of known plaintext from an out of the way base ("I have nothing to report"...) regularly helped the Enigma decryptors when the key changed).
jiggy2011 13y ago
I would break the levels of crypto knowhow down thusly: 1) End user level: You want to buy things online, you should know to check for the padlock icon. 2) Deployment level: You run an online shop using an off the shelf eCommerce system. You should buy an SSL cert and know how to install it on your web server of choice. You should also know the difference between a public and private key (and hence why you should never give the private key out) and roughly what a certificate and digital signature are for. 3) Integration level: You are building a custom application/API which uses crypto for security/privacy. You should know the difference between symetric/asymmetric encryption, what things like HMACs are and what the purpose of a password hash is. You should know how to choose a good library or ensure that your framework is integrating a good library. You should also be mindful of which information you should never leak over cleartext channels and also things like MITM attacks. 4) Implementation level: You are using existing crypto primatives to develop a new use for encryption, maybe a crypto currency or a new type of auth protocol for example. You should know a lot of deep computer science related to crypto, like the difference between AES CBC and ECB modes, various issues related to padding, how to generate secure random numbers and countless other issues. Here be dragons, you probably need significant peer review before your work is "production ready". 5) Design level: You are trying to improve an existing crypto algorithm or develop an entirely new one. Here you are going to need a very deep understanding of mathematics and cryptography research, probably to PHD level. So all kinds of stuff about number theory and being able to formally prove everything. I would guess that most developers should stay at around level 2 as much as possible, delving into level 3 only when required and never further.
- StavrosK 13y ago
  > I would guess that most developers should stay at around level 2 as much as possible, delving into level 3 only when required and never further. Why would you advocate that people stay away from learning cryptography? They should know as much as possible, they just should avoid implementing it.
  - jiggy2011 13y ago
    Sure, I meant in terms of production implementation though that may not have been clear. It's probably wise to know basic things at least one level further down than your required level so you know when things are starting to smell funny.
perlgeek 13y ago
At $work we regularly have trainees who learn just a tiny bit of programming. We teach them not to touch cryptography if at all possible. And I don't touch it either. When we do web development, the admins take care that the application runs over https, and authentication is handled by some LDAP thingy in the web server. The only other cryptography we use is ssh, and signing submissions to the RIPE database with PGP, and stuff transparently done by some database client libraries. It seems to be a sweet spot where we're still flexible enough for what we do, and don't present too much attack surface.
JCordeiro 13y ago
The thing with cryptography is that you have to get it right every time. The person trying to crack in only has to get it right once.
venomsnake 13y ago
Few thoughts - why MD5 in the example at all? I thought that SHA-2 was the go to for hash functions nowadays. And a question on the timing attack - how real it is? Because in the response time you have a lot of random variables which with the current fast servers will take more time than the calculation itself. You receive request - how fast it will go trough the loadbalancer is random, then you must read the shared secret, then you must log the request somewhere and the whole hmac calculation is milisecond or less.
- StavrosK 13y ago
  With thousands of tries, you can apparently average the noise out pretty effectively, so timing attacks are very possible.
- Dylan16807 13y ago
  State of the art multiple years ago was 100ns over LAN (aka datacenter). It's something to worry about. Honestly, I'd be extra paranoid and use a construct that hashes the supplied digest an extra time on top of doing a timing-safe comparison.
- tptacek 13y ago
  SHA2 has the same problems.
  - ReidZB 13y ago
    To be more accurate, SHA-224, SHA-512/384, and SHA-512/256 don't suffer from length extension attacks. The rest of the SHA2 family (256 and 512) do. Of course, SHA-256 and SHA-512 are more popular than their truncated variants...
- wglb 13y ago
  So if the attacker can make a million attempts, don't you think that if the hmac calculation has the most variance of all the other moving parts, that it wont give up the secret?
deleted 13y ago
[deleted]
jarrett 13y ago
This argument keeps coming up, and while its premises are valid, its conclusion never sits right with me, namely: "Don't use cryptography." That advice isn't practical for developers. There are plenty of systems we have to design where crypto is not optional. Examples: * Storing passwords. You can't store them in plaintext. * Signing requests (like in the OP's example). What are the alternatives? You can store some kind of authentication in the cookie, which then gets sent with each request. You can use HTTP auth. But in any scenario, you're definitely using SSL, and you're probably using some other kind of crypto on top of that. * Encrypting cookies. You should store as little as possible in cookies--ideally just a single identifier. But even so, it's very important to make cookies tamper-resistant. You can't, e.g. just have a plaintext cookie that says "userid=42." So what does the OP propose we do? If this article were making the usual case that we shouldn't implement our own crypto algorithms, I'd agree 100%. But he seems to be taking it a step farther, and saying we shouldn't even use existing crypto libraries, because we'll misuse them. But what are the alternatives?
- StavrosK 13y ago
  There aren't any, learn how the libraries work and use them. I agree with your point, in that respect. Also, about the cookies, you can definitely have a plaintext cookie saying userid=42, as long as it's signed.
  - jarrett 13y ago
    Yes, that's what I'm driving at: You have no choice but to learn the crypto libraries and use them. I've argued before that authors of crypto libraries have a sort of professional duty to document their libraries well, including info on all the mistakes developers are likely to make. I think if you're going to claim to offer crypto for average developers, you owe it to them to document it properly. It's always seemed a little unreasonable to me to blame non-cryptographers for being unaware of obscure vulnerabilities. We can't all be experts in everything. Yeah, agreed about the cookie. I guess I should have said "a plaintext cookie only containing 'userid=42.'" Because clearly you can't just let people edit their cookies and take on any user ID, session ID, or other such identifier.
  - 13y ago
scg 13y ago
> Measure the time each request takes to complete. Since string equality takes a tiny bit longer to complete when the first char matches, the message that takes the longest to return will have the correct first character. Always wondered if this really works in practice ... I imagined the time it takes to compare 2 strings should be negligible / indistinguishable in a full HTTP request over the wire. Among all the other things happening in the network stack, OS, Rails, HTTP, database access, and everything else you're doing in your request, a string comparison of a few dozen characters should be well within the noise range. I've Googled around and it seems like it really is a viable attack: http://www.cs.rice.edu/~dwallach/pub/crosby-timing2009.pdf http://www.cs.rice.edu/~dwallach/pub/crosby-timing2009.pdf > We have shown that, even though the Internet induces significant timing jitter, we can reliably distinguish remote timing differences as low as 20µs. A LAN environment has lower timing jitter, allowing us to reliably distinguish remote timing differences as small as 100ns (possibly even smaller). These precise timing differences can be distinguished with only hundreds or possibly thousands of measurements. #mindblown
- tptacek 13y ago
  Roughly a hundred people have implemented a milliseconds-granular timing attack in our crypto challenges. Our challenge isn't totally realistic (in reality, you'd be timestamping as close to the wire as possible, and if you used Python, you'd be using it to postprocess samples you took in C) but I think if you get through it you'll understand why the attack is viable. Keep in mind that most target applications will allow you to colocate yourself for ~$100; you just get your sampling server onto a VM in the same hosting center. There are programming environments that don't readily admit to over-the-wire timing attacks. Nate Lawson did a Black Hat talk about this a few years ago. It is difficult- bordering- on- implausible to remotely time memcmp. It's hard to say whether it's going to get harder to carry out this attack as machines get faster, or easier as attackers discover more filtering techniques and better ways of pessimizing execution on target machines.
  - dbarlett 13y ago
    Even better - with some creative "cloud cartography"[1], you have a 40% chance of launching a VM on the same physical host as your target. [1] https://www.cs.cornell.edu/courses/cs6460/2011sp/papers/cloudsec-ccs09.pdf https://www.cs.cornell.edu/courses/cs6460/2011sp/papers/clou...
jimmaswell 13y ago
For the timing-based one, you I would've made the server wait a small random amount of time during the process. The xor is more clever, though.
- Filligree 13y ago
  Not the way a naive reading of your post suggests, no. If you add a random amount of time, the timing attack still works; it averages out, the same way network jitter does. What you need to do is make every request take the same amount of time; randomness is not helpful here.
  - jimmaswell 13y ago
    I guess I've unintentionally helped prove the article's point, then!
andrewcooke 13y ago
by coincidence, saw this just now http://stackoverflow.com/questions/16777091/aes-counter-mode-encryption-library-has-hard-coded-its-initialization-vector http://stackoverflow.com/questions/16777091/aes-counter-mode...
dakimov 13y ago
Very good point. I would extend it to security in general. Security is like medicine: do not try it yourself, unless you are a specialist or want to end up dead. Too bad a lot of developers still do not understand this and remain too self-confident, from what I personally have seen.
tptacek 13y ago
Note: we're still running these challenges: http://www.matasano.com/articles/crypto-challenges/ http://www.matasano.com/articles/crypto-challenges/ The current standings are: * level 0 (4362 players), * level 1 (335 players), * level 2 (123 players), * level 3 (40 players), * level 4 (21 players), * level 5 (23 players), * level 6 (32 players) We're still donating $20 to PIH or Watsi for everyone who finishes all 6 sets. The top languages finishers are using are (in order) Python, Go, Ruby, C/C++, C#, Clojure, and Haskell. People are also using Java, PHP, OCaml, Scala, Racket, Javascript, Common Lisp, Perl, Julia(!), erlang, Rust, Visual Basic, Groovy, ObjC, F#, Factor, Dart, and Microsoft Excel(!!!).
- dsl 13y ago
  Just curious how far the Excel hacker(s) got... Also, have you made any hires from the contest yet?
  - tptacek 13y ago
    All I'm going to say here is that you would not freaking believe how far the Excel guy has gotten. No VBA, either; it's pure spreadsheet cells. Yes, we've hired from the challenges (I don't like thinking of it as a contest; I think of it more as an extremely engaging blog post).
    - Mandatum 13y ago
      Haha I love that, "the Excel guy".
  - rwg 13y ago
    The Excel thing started out as someone else's funny tweet and turned into me riding the bomb down while waving my cowboy hat, Slim Pickens style. This tweet started the ball rolling: https://twitter.com/sachinag/status/329701402546941953 https://twitter.com/sachinag/status/329701402546941953 The spreadsheets use no VBA or AppleScript or external code -- just Excel formulas and the functions Excel 2008 offers. I submitted the first four spreadsheets along with my "real" (Python) code/answers for set #2, just for the shock/horror value. Later, despite what I said about giving up on Excel, I implemented the Mersenne Twister PRNG and single-block AES encryption (all three key sizes!): https://twitter.com/areuugee/status/333344208154923008 https://twitter.com/areuugee/status/333344208154923008 https://twitter.com/areuugee/status/334142574296199168 https://twitter.com/areuugee/status/334142574296199168 One of these days, I'm going to start a blog just so I can write a series of posts about Excel. There are definitely limitations in what the design allows you to do, but it's more powerful than anyone gives it credit for. It'll also be cathartic to share how I made the AES and Mersenne Twister spreadsheets. (Spoiler: The C preprocessor was involved.) That said, I don't think the Matasano people have to worry about me solving more problems in Excel. Programs that I could bang out in five minutes in Python turned into all-day, profanity-laden affairs with Excel, which turns Excel into a distraction from the crypto stuff. Also, if anyone decides to give Matasano's crypto challenge a try, it's a great opportunity to learn a new programming language or a great excuse to get better at a programming language that you don't know so well. My Python knowledge was pretty minimal at best (the only languages I've ever really cared about are C, Fortran, Perl, and [spit!] PHP), but now I'm much more comfortable with it. In fact, if I ever finish, I might go back and redo everything in Ruby just to learn Ruby...

268 points by alinajaf 13y ago

Saying stuff is hard is easy, and as far as I can tell, very correct about cryptography. More useful would be actual suggestions to enable average developers to do more things with some degree of confidence. In other words, libraries, recipes and other documentation, and so on. "Hire an expensive security guy" is probably not a feasible solution for many small startups, nor very scalable in any case. Also, out of curiosity, in his timing attack example, the difference in time caused by the string being equal seems like it'd get absolutely swallowed up by the random nature of the universe - do those things actually work in the real world, on real servers with varying loads and numbers of users and network traffic?

alinajaf 13y ago
Hi, author here. I would have liked to have made useful suggestions, but having made so many errors with cryptography in the past, I don't really feel qualified to do so. I do however recommend getting in touch with tptacek and co and doing the crypto challenges. I mentioned this at the bottom of the post. Edit: re: timing attacks, it helps if you're in the same data centre. I've never personally implemented a timing attack, so I'm hesitant to make any assertions further than that either way.
- GhotiFish 13y ago
  I am doing this. It looks fun.
- bazzargh 13y ago
  btw: isn't the scheme you posted vulnerable to replay attacks too? Much easier than the timing attack. edited to add: don't mean this as a nitpick. I've seen that very mistake made by two S&P 500 companies that had 'homebrew' SSO we had to integrate with.
  - alinajaf 13y ago
    > btw: isn't the scheme you posted vulnerable to replay attacks too? Much easier than the timing attack. Yep, probably. In fact now that I look at it definitely, I should have gone with that instead! I forget who said this but basically any feature that exists in TLS that doesn't exist in your hand-rolled authentication scheme is a vulnerability.
    - mcintyre1994 13y ago
      Does peer review count as a feature?
moakakala 13y ago
>Also, out of curiosity, in his timing attack example, the difference in time caused by the string being equal seems like it'd get absolutely swallowed up by the random nature of the universe - do those things actually work in the real world, on real servers with varying loads and numbers of users and network traffic? You could make each request many times, and then average them together. I don't know how many requests you'd have to make to overcome the random fluctuations though -- probably a lot.
- zackangelo 13y ago
  Also, this random behavior will be significantly amplified if the host is running inside of a VM.

schabernakk 13y ago

When he is saying >Save yourself the trouble. Don't use cryptography. It is plutonium. There are millions of ways to mess it up and precious few ways of getting it right. I am sure he means don't use your own homegrown cryptography solution. Use something established and well tested. Good advice, probably can't be repeated enough. I liked the exampled he gives, I didnt know for example that you can basically extend an MD5ed string and keep the original MD5 value. Then again, I know that MD5 isn't a secure cryptographic hash function so I wouldn't have used it from the start. Nice to know why thats the case.

normalfaults 13y ago
Is there a repository or central location of the established and well tested solutions for developers to use? Where does one start?
- fredsted 13y ago
  Wikipedia perhaps?
- Osmium 13y ago
  Some ideas, depending on what you need (I can't say how good these are): http://www.keyczar.org http://www.keyczar.org http://nacl.cr.yp.to http://nacl.cr.yp.to https://github.com/jedisct1/libsodium https://github.com/jedisct1/libsodium http://www.gnupg.org http://www.gnupg.org
  - normalfaults 13y ago
    Thanks. These are straight crypto frameworks which are a good start.
- betterunix 13y ago
  One of the problems with cryptography is that "solutions" are not as generally applicable as one might want. There are a lot of assumptions that are made, and violating those assumptions is usually a disaster. For example, when it comes to encryption, it is typically assumed that all messages are the same length. If your application does not make this guarantee, encryption may not provide you with any security. You could wind up in this situation: https://news.ycombinator.com/item?id=2661890 https://news.ycombinator.com/item?id=2661890 Think of it this way: asking for a list of tried-and-true crypto solutions is like asking for a list of tried-and-true database schemas. There might be cases where it will work, but for the most part you need to put some thought into what you are doing.

e1ven 13y ago

So, sure. I'll admit that I'm bad at cryptography. You win! But the problem is, the world needs more, not less, crypto. We need to integrate crypto into more places, not run and hide and declare it too hard every time we come across it. Is it easy to screw up? Sure. So is manually allocating memory. But we use higher level languages to help protect us from ourselves. So too can we use higher-level libraries to help protect us from some mistakes we can make with crypto. I'm working on an app which uses crypto heavily- In initial versions, I used manual RSA keys, but I recognize that no matter how smart I think I am (not very), it's easy to mess stuff up. I can screw up the padding, I can screw up the implementation. Other clients which interop with me can screw up... So Instead, I've swapped to using GPG keys. Is it perfect? Heck no! But it reduces the code that I have to get right. It outsources the parts that are most likely to be screwed up to a place where they're more likely to be reviewed, more likely to be written by non-fuckups. There are lots of higher level libs, gpg is just one of them. keyczar, etc are also ways to solve this problem. But we shouldn't run in fear everytime someone mentions the word Crypto, just because we have a chance of messing up.

betterunix 13y ago
What we really need is a cryptography language, to do for cryptography what SQL does for databases. I should be able to say, "Messages have this form, and they need to be encrypted end-to-end with authentication and without surreptitious forwarding," and have the right components assembled for me. Unfortunately, our understanding of cryptography is not well-developed enough to create such a system, and so we still fumble around and usually produce systems that are vulnerable (did you remember to have a good source of entropy? did you make sure your messages are all the same length? does your system compose well with others?).
- EthanHeilman 13y ago
  Even if we had such a language it would probably not be high level enough because understanding what security you know is still a hard problem. Instead I'd suggest a high level library that gives you a complete security package with defaults that can't be easily changed. Something like what the CAESAR competition hopes to achieve: http://competitions.cr.yp.to/caesar.html http://competitions.cr.yp.to/caesar.html

moakakala 13y ago

> This means that as long as you have one example of a signed message, you can forge signatures for that message plus any arbitrary request parameters you like and they will authenticate under the above described scheme. If all requests are made over HTTPS, how could a third party intercept a signed message? How is this any greater of a risk than a third party intercepting user login information? (This is a serious question; I'm not being flippant or saying 'gotcha')

jd007 13y ago
If you do not manage SSL certificate properly (trust only your server's certificate), a MITM attack can break the system easily. Otherwise I don't think you can intercept the signed message if everything is over TLS.
- moakakala 13y ago
  Is the risk any greater with APIs (like described in the article) compared to typical username/password login systems?
  - _phred 13y ago
    A lot of API authentication is half-assed, like the examples in the article. "OAuth is hard, roll your own" is a common approach. Even with, e.g., OAuth 2, who's to say that the scheme is completely safe and that your implementation is correct? As far as API vs. user account, it depends on the loot. An API might let me do more damage faster, or subtly lurk and alter/steal data over time. It might also be harder to detect from the UI, no "last logged in" giveaway. Also, some API vendors recommend disabling SSL cert validation client side. Even for credit card gateways, unbelievably. Since it's a script talking to a script, no one is going to see the cert problems from a MITM until it's too late.
- _phred 13y ago
  Right. If the attack vector is "break SSL" I'm going to try some other attacks first. There's an underlying assumption in the question: my app (and everything else hosted on the box) is safe from XSS, CSRF, injections, and other information leakages. Is it really? How do I know for sure? And who's to say that your forum server (for example) is just as secure? That could be a foothold into your environment too. And let's not forget social manipulation of your staff and users. Maybe I'll just steal the machine in question, or your laptop. After I try all those avenues, I'm either finding another target or ramping up for a protracted attack on your SSL connections. If your site attracts this dedicated of an attack, you'd better get that high paid security consultant. ;)

geon 13y ago

Then there is the question; Is your app likely to ever be attacked at all? By someone good? With a lot of time? If not, perhaps hiring a cryptography expert is overkill, and you are better off just using whatever your framework offers, or integrate some library yourself. You are not a bank.

scott_karana 13y ago
The message extension attack is pretty well known by crackers, which could leave your webapp open to pre-scripted kit attacks that simply crawl the web.
mcdigman 13y ago
This is a very dangerous question. Thinking like this leads to huge security vulnerabilities in important systems. A better question is "What is the worst thing that could possibly happen if this system were successfully attacked?" If the answer is anything bad, it is probably best to assume someone will try to attack it. As for the second part of the question, are we talking about the same internet? Because the one I'm on appears to be full of technically savvy people with loads of free time.

jd007 13y ago

About the timing attack on HMAC that the article mentions. It takes thousands if not hundreds of thousands of requests to gather the data (and let's assume you can indeed extract the data out of all the noise cause by network latency etc...), and any properly designed API should have a throttle measure built in to prevent brute force attacks like this. A good, secure API is protected by a variety of measures, not just through the request authentication component. Of course you should still make your auth as secure as possible.

teeja 13y ago
"a throttle measure built in" It's my understanding that bittorrent sync uses the latency of the network for such a throttle. Wild hair: a cyclotron-style router roundabout could hold millions of packets "in suspension" for n seconds.
teeja 13y ago
"a throttle measure built in" It's my understanding that bittorrent sync uses the latency of the network for such a throttle. Wild hair: a cyclotron-style router roundabout could hold millions of packets "in suspension" for n seconds.

megrimlock 13y ago

His insight that crypto is hard because you don't get feedback when you mess up is good. It made me wonder what other domains are like that -- domains where correctness seems within reach, yet there are subtle aspects that are hard to state. Some that come to mind: * Concurrency. It's easy to introduce race conditions, livelocks, or deadlocks without even knowing. They are often difficult to observe or reproduce, and it takes a lot of hard-won experience to become sufficiently critical of your own incompetence. On the other hand, the tools are improving here (helgrind, relacy, threadsanitizer). * Exception-safety. I don't mean in the weak sense of "won't leak resources" but in the stronger sense of leaving the system in a consistent state after an exception. Again, it's difficult to analyze or induce all the different code paths that might occur, and difficult or expensive (depending on your starting point) to roll back your mutations. Again, there are tools that help with this, like STM or cheap persistent data structures (so you can do a bunch of mutation that is only committed atomically at the end). Are there others? Can the progress in these domains shed light on how to make crypto safer?

LaGrange 13y ago
Outside of computing: health and diet. Sure, you get some feedback, but you also can screw up things in some ridiculously slow-acting way. Pharma companies (are supposed to) monitor drug use for decades after introduction for the same reason. Construction. Just build that chunk out of two steel beams instead of one, and witness it work perfectly until someone decides to throw a party on that balcony. Or witness your construction slowly turn itself from a perfectly normal tower to a major tourist attraction over the centuries. Parenting. Things like sudden stimuli in early life have been linked to ADD, the way you do attachment can have tremendous consequences, and sometimes singular events you barely have any control over can have unpredictable consequences later in your life. In fact, I'm tempted to say "life in general".
- jiggy2011 13y ago
  Economics is also an excellent example. Nobody knows how to do it properly, even establishing direct cause and effect is basically impossible so we just iterate through progressively less broken solutions.

cllns 13y ago

(Un)conscious (in)competence reminds me of Donald Rumsfeld's trichotomy of known knowns, known unknowns and unknown unknowns. (highlighted here: http://www.youtube.com/watch?v=2Ex8EEv-WPs http://www.youtube.com/watch?v=2Ex8EEv-WPs)

idupree 13y ago

How is the shared secret generated, stored and shared? (There are lots of ways to do this wrong, and the code and algorithm for that were not even shown.)

tveita 13y ago

Unless I'm missing something, the example code will just calculate the checksum of "key:value&key:value&key:value", so actually anything with the same number of parameters will pass. Assuming "key:value" is supposed to read "#{key}:#{value}", this may be vulnerable to a delimiter attack -- you couldn't tell the difference between {foo: 'bar', bar: 'baz'} and {foo: 'bar&bar:baz'}.

alinajaf 13y ago
You are of course correct. Thanks! Will fix and rollout once traffic from HN/reddit dies down.

saosebastiao 13y ago

I'm bad at crypto. I'm well aware that I'm bad at crypto. In fact, I would venture to guess that I know next to nothing about it. The problem isn't that I don't know what I don't know...the problem is that I don't know what (or whom) I can trust. And that is the bigger problem with crypto. It would be nice if we had resources that could tell us not only the best practices, but what their downfalls are and their relative difficulties of cracking. Some common sense tells me that there is a combination of techniques that are simple and robust enough that I would have to worry about phishing before I had to worry about better security. But I don't know what that combination is, and I don't know if I can trust anybody that tells me.

alinajaf 13y ago
Author here. A good place to start (mentioned in the article). http://www.matasano.com/articles/crypto-challenges/ http://www.matasano.com/articles/crypto-challenges/ Never met any of them in real life, but they seem like good people to me. I've actually used knowledge I picked up from those exercises to fix vulnerabilities in past clients code, and I'm not yet half-way through them.
epsylon 13y ago
Try Dan Boneh's crypto course on coursera [1]. It covers quite a lot of ground, both practical and theoretical, and includes programming exercises similar to the matasano puzzles. Without a doubt it's one of the best courses of the dozen MOOCs I've taken. There's also a followup course [2] (I haven't taken it yet personally, but I believe the currently scheduled run will be the first). Interestingly enough, there's also an upcoming security course (with no date planned yet) which will cover the application programming part of security and will be co-taught by him. [3] [1] https://www.coursera.org/course/crypto https://www.coursera.org/course/crypto [2] https://www.coursera.org/course/crypto2 https://www.coursera.org/course/crypto2 [3] https://www.coursera.org/course/security https://www.coursera.org/course/security
- cryptbe 13y ago
  Glad that you like Dan Boneh's crypto class. I made the programming exercises :-).

almost 13y ago

I didn't know about Matasano's Crypto challenges. Looks like fun, going to give it a try! http://www.matasano.com/articles/crypto-challenges/ http://www.matasano.com/articles/crypto-challenges/

ams6110 13y ago

I have implemented an MD5-based scheme similar to what was described. At the time, MD5/hash extension attacks were not as well-known as they are today, at least I had not heard of them and I read up on MD5 before I designed the sytem, so another worry is that what is considered secure today might not be secure tomorrow.

alinajaf 13y ago
> another worry is that what is considered secure today might not be secure tomorrow If history is anything to go by then what is secure today will definitely be insecure tomorrow. No real solution to this AFAIK. If vulnerabilities in the crypto itself doesn't get you eventually, quantum computing will.
- Dylan16807 13y ago
  Quantum computers can only solve certain types of problems efficiently. Hashing is not generally in that set.

lifeisstillgood 13y ago

I completely agree with this and have just released a session-cache for python that completely ignores encryption and just stores uuid in a cookie, and relies on server side lookups for session work.

kbaker 13y ago
Make sure you are using HTTPS exclusively so you aren't vulnerable to Firesheep-style attacks.
- lifeisstillgood 13y ago
  That's in the docs :-)
- lifeisstillgood 13y ago
  That's in the docs :-) Ooops no it's not - kind of assumed it was obvious - thank you for the reminder :-)

wzdd 13y ago

I'm just curious: the obvious response to the first problem (message extension attack) is to include the message length in the message. If this wouldn't work, could an expert please tell me why? Edit: To be specific, I was thinking of a Pascal-style length-at-start arrangement. It's clear that if you put the length at the end you may be vulnerable to exactly the same attack!

bazzargh 13y ago
I'm not an expert, but I can see a problem with schemes along those lines. MD5 (and similar) extension attacks also take advantage of the padding added after the thing you want to hash; so eg Secret: secret Message: foo&bar Let's include the length and hash 'secretfoo&bar13' What you actually hash is 'secretfoo&bar130x80...padding...0x20'. What the attacker sees as the 'known hash' includes this padding. When you perform an extension attack, the attack message would look like: secretfoo&bar130x80...padding...0x02&evil . At this point you're adding on the length and hashing it, so what you compare with is 'secretfoo&bar130x80...padding...0x02&evil93' ... but the attacker can precompute that hash just as easily as the one with just 'evil' appended. You could argue that your hashes would be constructed a bit differently (eg make messages that appear to have internal padding illegal; still vulnerable, btw), but the point is - you're still tinkering with the same basic structure, and are likely to make more mistakes. You're better off switching to HMAC for this application.
- deleted 13y ago
  [deleted]
jdbernard 13y ago
The attacker just needs to include the updated length, which he can likely calculate since the shared secret is probably the same length for all accounts-even if it is not he only needs it for this request--and he knows the length of the parameters and their values. It would take some guesswork and trial and error to nail it down, and the attacker would have to guess that the length was included in the first place. Assuming he has plenty of time and processing power it is not infeasible. There are a lot of fairly easy things you could add to make it more difficult to attack.

hga 13y ago

Hmmm, one of the things I've noticed is that if you've studied pre-web cryptography extensively (as I did, starting even before I started programming), "Everything you know is wrong". Or thereabouts. The HMAC timing attack illustrates this well, "classical" systems as I will call them, e.g. Enigma and how it was used, are not subject to zillions of known plaintext attacks (although I've read one bit of known plaintext from an out of the way base ("I have nothing to report"...) regularly helped the Enigma decryptors when the key changed).

jiggy2011 13y ago

I would break the levels of crypto knowhow down thusly: 1) End user level: You want to buy things online, you should know to check for the padlock icon. 2) Deployment level: You run an online shop using an off the shelf eCommerce system. You should buy an SSL cert and know how to install it on your web server of choice. You should also know the difference between a public and private key (and hence why you should never give the private key out) and roughly what a certificate and digital signature are for. 3) Integration level: You are building a custom application/API which uses crypto for security/privacy. You should know the difference between symetric/asymmetric encryption, what things like HMACs are and what the purpose of a password hash is. You should know how to choose a good library or ensure that your framework is integrating a good library. You should also be mindful of which information you should never leak over cleartext channels and also things like MITM attacks. 4) Implementation level: You are using existing crypto primatives to develop a new use for encryption, maybe a crypto currency or a new type of auth protocol for example. You should know a lot of deep computer science related to crypto, like the difference between AES CBC and ECB modes, various issues related to padding, how to generate secure random numbers and countless other issues. Here be dragons, you probably need significant peer review before your work is "production ready". 5) Design level: You are trying to improve an existing crypto algorithm or develop an entirely new one. Here you are going to need a very deep understanding of mathematics and cryptography research, probably to PHD level. So all kinds of stuff about number theory and being able to formally prove everything. I would guess that most developers should stay at around level 2 as much as possible, delving into level 3 only when required and never further.

StavrosK 13y ago
> I would guess that most developers should stay at around level 2 as much as possible, delving into level 3 only when required and never further. Why would you advocate that people stay away from learning cryptography? They should know as much as possible, they just should avoid implementing it.
- jiggy2011 13y ago
  Sure, I meant in terms of production implementation though that may not have been clear. It's probably wise to know basic things at least one level further down than your required level so you know when things are starting to smell funny.

perlgeek 13y ago

At $work we regularly have trainees who learn just a tiny bit of programming. We teach them not to touch cryptography if at all possible. And I don't touch it either. When we do web development, the admins take care that the application runs over https, and authentication is handled by some LDAP thingy in the web server. The only other cryptography we use is ssh, and signing submissions to the RIPE database with PGP, and stuff transparently done by some database client libraries. It seems to be a sweet spot where we're still flexible enough for what we do, and don't present too much attack surface.

JCordeiro 13y ago

The thing with cryptography is that you have to get it right every time. The person trying to crack in only has to get it right once.

venomsnake 13y ago

Few thoughts - why MD5 in the example at all? I thought that SHA-2 was the go to for hash functions nowadays. And a question on the timing attack - how real it is? Because in the response time you have a lot of random variables which with the current fast servers will take more time than the calculation itself. You receive request - how fast it will go trough the loadbalancer is random, then you must read the shared secret, then you must log the request somewhere and the whole hmac calculation is milisecond or less.

StavrosK 13y ago
With thousands of tries, you can apparently average the noise out pretty effectively, so timing attacks are very possible.
Dylan16807 13y ago
State of the art multiple years ago was 100ns over LAN (aka datacenter). It's something to worry about. Honestly, I'd be extra paranoid and use a construct that hashes the supplied digest an extra time on top of doing a timing-safe comparison.
tptacek 13y ago
SHA2 has the same problems.
- ReidZB 13y ago
  To be more accurate, SHA-224, SHA-512/384, and SHA-512/256 don't suffer from length extension attacks. The rest of the SHA2 family (256 and 512) do. Of course, SHA-256 and SHA-512 are more popular than their truncated variants...
wglb 13y ago
So if the attacker can make a million attempts, don't you think that if the hmac calculation has the most variance of all the other moving parts, that it wont give up the secret?

deleted 13y ago

[deleted]

jarrett 13y ago

This argument keeps coming up, and while its premises are valid, its conclusion never sits right with me, namely: "Don't use cryptography." That advice isn't practical for developers. There are plenty of systems we have to design where crypto is not optional. Examples: * Storing passwords. You can't store them in plaintext. * Signing requests (like in the OP's example). What are the alternatives? You can store some kind of authentication in the cookie, which then gets sent with each request. You can use HTTP auth. But in any scenario, you're definitely using SSL, and you're probably using some other kind of crypto on top of that. * Encrypting cookies. You should store as little as possible in cookies--ideally just a single identifier. But even so, it's very important to make cookies tamper-resistant. You can't, e.g. just have a plaintext cookie that says "userid=42." So what does the OP propose we do? If this article were making the usual case that we shouldn't implement our own crypto algorithms, I'd agree 100%. But he seems to be taking it a step farther, and saying we shouldn't even use existing crypto libraries, because we'll misuse them. But what are the alternatives?

StavrosK 13y ago
There aren't any, learn how the libraries work and use them. I agree with your point, in that respect. Also, about the cookies, you can definitely have a plaintext cookie saying userid=42, as long as it's signed.
- jarrett 13y ago
  Yes, that's what I'm driving at: You have no choice but to learn the crypto libraries and use them. I've argued before that authors of crypto libraries have a sort of professional duty to document their libraries well, including info on all the mistakes developers are likely to make. I think if you're going to claim to offer crypto for average developers, you owe it to them to document it properly. It's always seemed a little unreasonable to me to blame non-cryptographers for being unaware of obscure vulnerabilities. We can't all be experts in everything. Yeah, agreed about the cookie. I guess I should have said "a plaintext cookie only containing 'userid=42.'" Because clearly you can't just let people edit their cookies and take on any user ID, session ID, or other such identifier.
- 13y ago

scg 13y ago

> Measure the time each request takes to complete. Since string equality takes a tiny bit longer to complete when the first char matches, the message that takes the longest to return will have the correct first character. Always wondered if this really works in practice ... I imagined the time it takes to compare 2 strings should be negligible / indistinguishable in a full HTTP request over the wire. Among all the other things happening in the network stack, OS, Rails, HTTP, database access, and everything else you're doing in your request, a string comparison of a few dozen characters should be well within the noise range. I've Googled around and it seems like it really is a viable attack: http://www.cs.rice.edu/~dwallach/pub/crosby-timing2009.pdf http://www.cs.rice.edu/~dwallach/pub/crosby-timing2009.pdf > We have shown that, even though the Internet induces significant timing jitter, we can reliably distinguish remote timing differences as low as 20µs. A LAN environment has lower timing jitter, allowing us to reliably distinguish remote timing differences as small as 100ns (possibly even smaller). These precise timing differences can be distinguished with only hundreds or possibly thousands of measurements. #mindblown

tptacek 13y ago
Roughly a hundred people have implemented a milliseconds-granular timing attack in our crypto challenges. Our challenge isn't totally realistic (in reality, you'd be timestamping as close to the wire as possible, and if you used Python, you'd be using it to postprocess samples you took in C) but I think if you get through it you'll understand why the attack is viable. Keep in mind that most target applications will allow you to colocate yourself for ~$100; you just get your sampling server onto a VM in the same hosting center. There are programming environments that don't readily admit to over-the-wire timing attacks. Nate Lawson did a Black Hat talk about this a few years ago. It is difficult- bordering- on- implausible to remotely time memcmp. It's hard to say whether it's going to get harder to carry out this attack as machines get faster, or easier as attackers discover more filtering techniques and better ways of pessimizing execution on target machines.
- dbarlett 13y ago
  Even better - with some creative "cloud cartography"[1], you have a 40% chance of launching a VM on the same physical host as your target. [1] https://www.cs.cornell.edu/courses/cs6460/2011sp/papers/cloudsec-ccs09.pdf https://www.cs.cornell.edu/courses/cs6460/2011sp/papers/clou...

jimmaswell 13y ago

For the timing-based one, you I would've made the server wait a small random amount of time during the process. The xor is more clever, though.

Filligree 13y ago
Not the way a naive reading of your post suggests, no. If you add a random amount of time, the timing attack still works; it averages out, the same way network jitter does. What you need to do is make every request take the same amount of time; randomness is not helpful here.
- jimmaswell 13y ago
  I guess I've unintentionally helped prove the article's point, then!

andrewcooke 13y ago

by coincidence, saw this just now http://stackoverflow.com/questions/16777091/aes-counter-mode-encryption-library-has-hard-coded-its-initialization-vector http://stackoverflow.com/questions/16777091/aes-counter-mode...

dakimov 13y ago

Very good point. I would extend it to security in general. Security is like medicine: do not try it yourself, unless you are a specialist or want to end up dead. Too bad a lot of developers still do not understand this and remain too self-confident, from what I personally have seen.

tptacek 13y ago

Note: we're still running these challenges: http://www.matasano.com/articles/crypto-challenges/ http://www.matasano.com/articles/crypto-challenges/ The current standings are: * level 0 (4362 players), * level 1 (335 players), * level 2 (123 players), * level 3 (40 players), * level 4 (21 players), * level 5 (23 players), * level 6 (32 players) We're still donating $20 to PIH or Watsi for everyone who finishes all 6 sets. The top languages finishers are using are (in order) Python, Go, Ruby, C/C++, C#, Clojure, and Haskell. People are also using Java, PHP, OCaml, Scala, Racket, Javascript, Common Lisp, Perl, Julia(!), erlang, Rust, Visual Basic, Groovy, ObjC, F#, Factor, Dart, and Microsoft Excel(!!!).

dsl 13y ago
Just curious how far the Excel hacker(s) got... Also, have you made any hires from the contest yet?
- tptacek 13y ago
  All I'm going to say here is that you would not freaking believe how far the Excel guy has gotten. No VBA, either; it's pure spreadsheet cells. Yes, we've hired from the challenges (I don't like thinking of it as a contest; I think of it more as an extremely engaging blog post).
  - Mandatum 13y ago
    Haha I love that, "the Excel guy".
- rwg 13y ago
  The Excel thing started out as someone else's funny tweet and turned into me riding the bomb down while waving my cowboy hat, Slim Pickens style. This tweet started the ball rolling: https://twitter.com/sachinag/status/329701402546941953 https://twitter.com/sachinag/status/329701402546941953 The spreadsheets use no VBA or AppleScript or external code -- just Excel formulas and the functions Excel 2008 offers. I submitted the first four spreadsheets along with my "real" (Python) code/answers for set #2, just for the shock/horror value. Later, despite what I said about giving up on Excel, I implemented the Mersenne Twister PRNG and single-block AES encryption (all three key sizes!): https://twitter.com/areuugee/status/333344208154923008 https://twitter.com/areuugee/status/333344208154923008 https://twitter.com/areuugee/status/334142574296199168 https://twitter.com/areuugee/status/334142574296199168 One of these days, I'm going to start a blog just so I can write a series of posts about Excel. There are definitely limitations in what the design allows you to do, but it's more powerful than anyone gives it credit for. It'll also be cathartic to share how I made the AES and Mersenne Twister spreadsheets. (Spoiler: The C preprocessor was involved.) That said, I don't think the Matasano people have to worry about me solving more problems in Excel. Programs that I could bang out in five minutes in Python turned into all-day, profanity-laden affairs with Excel, which turns Excel into a distraction from the crypto stuff. Also, if anyone decides to give Matasano's crypto challenge a try, it's a great opportunity to learn a new programming language or a great excuse to get better at a programming language that you don't know so well. My Python knowledge was pretty minimal at best (the only languages I've ever really cared about are C, Fortran, Perl, and [spit!] PHP), but now I'm much more comfortable with it. In fact, if I ever finish, I might go back and redo everything in Ruby just to learn Ruby...