28 ms·

How I implemented my own crypto

380 points by loup-vaillant 9y ago

deleted 9y ago
[deleted]
sigjuice 9y ago
Are there other fields where the slogan "don't roll your own XXX unless you are an infallible expert" is applicable?
- kobeya 9y ago
  Aviation, medical devices, civil engineering.
- jrimbault 9y ago
  If you were a cook you wouldn't really be qualified to build a stove or make your own knives, but you'd know what kind of stove and knives you'd want.
- dlgeek 9y ago
  Parachute?
  - kabouseng 9y ago
    Actually you do roll your own parachute once you have done enough jumps. But your reserve chute is always rolled by a certified professional specifically certified to roll reserve chutes, and not by yourself. (if I remember correctly, my memory is a bit fuzzy from when I jumped a couple of years back)
    - Piskvorrr 9y ago
      Well, you are using the product, but not manufacturing it.
  - sebcat 9y ago
    You don't have to be an expert to make/jump the actual parachute/canopy. As long as the harness, container and reserve are TSO'd you're good to go. It's a lot of work though, at least for a ram-air. EDIT: Some interesting discussion here: http://www.dropzone.com/cgi-bin/forum/gforum.cgi?post=4822410 http://www.dropzone.com/cgi-bin/forum/gforum.cgi?post=482241...
  - karrotwaltz 9y ago
    I'm not sure about "don't roll your own parachute" as it should be common sense, but well, in 1912 a parachute inventor died while testing his creation. https://en.wikipedia.org/wiki/Franz_Reichelt https://en.wikipedia.org/wiki/Franz_Reichelt
  - cromulent 9y ago
    I know a few guys who have manufactured their own canopies and harness/container systems in their garage, and I've jumped them - even though I consider these guys fallible :)
- KineticLensman 9y ago
  "Hey y'all, hold my beer and look at this!" There are [1] several non-aviation examples: - Blood transfusion - Steam-powered bicycles - Rotary printing presses - Submarines - Luminescent paint - Hi-speed jet-powered railcars - Ropes /pulleys to help the user get out of bed [1] https://en.wikipedia.org/wiki/List_of_inventors_killed_by_their_own_inventions https://en.wikipedia.org/wiki/List_of_inventors_killed_by_th...
  - ithkuil 9y ago
    > - Steam-powered bicycles On a first read I though "yeah, sure, looks quite dangerous indeed", then I read: Sylvester H. Roper, inventor of the eponymous steam-powered bicycle, died of a heart attack or subsequent crash during a public speed trial in 1896. It is unknown whether the crash caused the heart attack or vice versa.
    - Piskvorrr 9y ago
      This seems to mean "killed in an interesting way". People killed by their inventions in petrol-based transportation seem to be too numerous even for a Wikipedia list: "yeah, people die in cars en masse, that's not really newsworthy."
  - kpil 9y ago
    Best Wikipedia page ever!
- socceroos 9y ago
  reefer?
- inopinatus 9y ago
  - Terms & conditions of use - Chocolate soufflé - Distributed lock manager - Autonomous war robot - init(1) replacement I am personally guilty of attempting four of these things without adequate preparation or expertise.
  - pzduniak 9y ago
    I seriously hope that you didn't try to write an init(1) replacement!
mathieubordere 9y ago
I still believe it's a bad idea to roll your own crypto, interesting as an exercise, but I would never use it in a production environment.
- merb 9y ago
  well if you have enough people especially working with crypto, than why not? It's not like that the existing solutions came out of nowhere. some people actually wrote them. I mean libsodium is actually really simple to use. however before libsodium it was probably easier and mostly more correct to roll your own. there are so many things you can do wrong with other libraries that the chance to do it wrong is as high as in your own implementation.
- timclark 9y ago
  How will cryptography libraries ever develop if someone doesn't roll their own?
  - rootlocus 9y ago
    Through sweat and blood. Don't expect crypto libraries to be secure the first few years after their release. That's why people favor the mature, battle-tested libraries.
    - jwl 9y ago
      But then again, we still got Heartbleed.
      - bananaboy 9y ago
        Yeah and imagine the kind of exciting bugs that exist in new and immature libraries! :)
        jwl 9y ago
        Yes, but are hackers going after common used libraries to get more vulnerable systems to attack or are they going to spend time on some unknown homebuilt crypto? In some cases, security through obscurity works well in practice.
conradk 9y ago
The v1.0.0 was published the day a bug was fixed and yet the first sentence of this post is "Monocypher is ready for production". Sure, OpenSSL has bugfixes on regular basis as well, but I believe that further fragmenting the crypto ecosystem with a new library and little commercial support to properly back it and pay for security audits is risky.
- baby 9y ago
  It's risky but sometimes it's the only way. To be fair the code size is relatively small so it should be easier to audit than other products. My guess is that his blog post is also some motivation to get new pair of eyes on the library. If there's any fund for that, I'd be happy to look at it ;)
- loup-vaillant 9y ago
  > The v1.0.0 was published the day a bug was fixed and yet the first sentence of this post is "Monocypher is ready for production". You have to consider the nature of the bug, as well as the fix. The bug was an Undefined Behaviour that had no effect on the binaries I could generate. The fix was 10 characters. Running the automated test suite took longer than fixing the bug. If there were reasons to delay 1.0, this bug was not it. > I believe that further fragmenting the crypto ecosystem with a new library and little commercial support to properly back it and pay for security audits is risky. I agree fragmentation is a problem. I just couldn't satisfy myself with existing alternatives. As for support, I designed Monocypher specifically to need very little of it. Ideally, the only notable changes from now on will be about the manual and the test suite.
baby 9y ago
First let me say that I've followed your work a bit and I'm really impressed with the library. I'm currently planning on rolling my own crypto as well and your library is on my list of the things to look at. One question: > I was shifting a uint8_t, 24 bits to the left. I failed to realise that integer promotion means this unsigned byte would be converted to a signed integer, and overflow if the byte exceeded 127. This sounds weird to me, how is a shift (which only apply itself on the bit value, not the number value) triggering a sign extension? I don't think you're talking about integer promotion here unless you had a uint16_t = uint8_t << 24 or something like this.
- kbeckmann 9y ago
  I believe this[1] is the patch that fixes this bug. I tried to reproduce the behavior but couldn't succeed. Maybe I was doing something wrong. Would really appreciate it if someone here could show a test case where this matters. [1] https://github.com/LoupVaillant/Monocypher/commit/347189c50c053cf13ce1310818c2913f4904c1eb https://github.com/LoupVaillant/Monocypher/commit/347189c50c...
  - baby 9y ago
    `ctx->c` is a u32[1] and msg is u8. So yeah there is integer promotion here. [1]: https://github.com/LoupVaillant/Monocypher/blob/master/src/monocypher.h#L58 https://github.com/LoupVaillant/Monocypher/blob/master/src/m...
  - loup-vaillant 9y ago
    Your failure to reproduce the behaviour is normal: on the compilers I have tested, this doesn't affect the generated binary. To see the difference, you need to first modify the makefile to use UBSan instead of just GCC (just comment the active CC line, and uncomment the right one). You may want to tone down the optimisations for faster compilation as well. Then you can run `make clean` and `./test.sh`. You should have a warning on commits prior to this patch.
  - deleted 9y ago
    [deleted]
AstralStorm 9y ago
Main lesson not learned: instead of testing, prove correctness in high assurance code like this. Rigorously and formally. Preferably even refine the proof to executable code. (Yes, it would take somewhere on the order of 10k LOC to prove correctness of this 1k.)
- 2bitencryption 9y ago
  I'm sure you encounter "high assurance code" like this every day -- your operating system, the secure protocol used to post your comment, the controls for your Anti-Lock Breaking System in your car. How much of those were formally proven?
  - yorwba 9y ago
    TLS has been subject to lots of formal verification. The other things, probably not so much. Verifying some (but not all) properties of crypto is relatively (but not absolutely) easy, because it's essentially just math with pure functions. Operating systems and physical controllers with real-time constraints have different requirements and more interacting state, which means that the desired properties are not easily expressed as a mathematical expression.
- baby 9y ago
  https://github.com/mitls/hacl-star https://github.com/mitls/hacl-star
- jfries 9y ago
  In this case 3 of the bugs would be caught when trying to prove the absense of undefined behaviour/use of uninitialized value. Can existing formal proof tools be used for that?
- SAI_Peregrinus 9y ago
  Proving correctness isn't enough. You have to test it too. It's perfectly possible for the proof to have errors (bugs). It's also possible for the proof to be incomplete: you can prove the algorithm is correct and implemented correctly to spec and still have vulnerabilities.
reikonomusha 9y ago
It was a fun read, except when all the C bugs were listed. It's getting tiresome to hear about folks writing nominally Important software in a language where it is extraordinarily difficult to get things absolutely correct, with paltry excuses such as "it needs to be fast" or "it needs to be portable [0] to a VAX-11/750." It doesn't give me confidence that these completely usual bugs popped up early on, and it will not surprise me when more show themselves. The author's lack of practical familiarity with the memory hierarchy, caching, and pipelining—evidenced by his surprise by the speed gained by fetching more than a byte at a time—also makes me suspicious of his understanding of the semantics of his program. (I understand that this is a form of ad hominem as it relates to arguing the correctness of software, but it's a consideration worth noting in a broader context of trust.) It would be great if this was written in another language. [0] Most "portable" code written in C is not portable. Look at any mature C project and see the layers and layers of macros and hacks to make things portable.
- taneq 9y ago
  > Look at any mature C project and see the layers and layers of macros and hacks to make things portable. So.. it's portable, then?
  - reikonomusha 9y ago
    Relying on a macro processor to patch and bandage your program at compile time depending on a collection of symbolic features isn't really representative of portability of a language and your code. To me, that is brute forcing portability, and is a web of conditional inclusions/exclusions that needs to be maintained and added to per-architecture and per-OS. Something that is, in some sense, truly portable is one that has a single specification that runs on all platforms the base implementation supports. Virtual machines and certain styles of language standard are often able to accomplish portability for a vast number of applications.
    - SmallDeadGuy 9y ago
      I would argue the other way. Something which requires a base implementation to be installed on a system which generates different machine code based on a single specification is less portable than a macro processor generating different versions of a program. Sure, people will have had something like python installed since a long time ago, but a new setup requires downloading 2 objects (the base implementation and the program) compared to a single one (a platform-specific copy of a program).
ktta 9y ago
I'm amazed that the reference implementation of Argon2 had a bug. So that means anyone who deployed Argon2 today didn't really use Argon2 (I'm being pedantic), but something else? libsodium also got it wrong, so that brought down my opinion of it being a trusted and well-reviewed library. Now the question is, will they continue to use the same implementation or move to the fixed code?
- loup-vaillant 9y ago
  Maybe the authors will fix the code in subsequent versions of Argon2. Right now, backward compatibility is deemed more important. The effects of this bug are practically negligible anyway. I bet a single bit of additional entropy in a password would compensate that a hundred fold. I'm a bit disappointed however at their not updating the specs. I signalled the bug in January, and the latest version of the appear to have been published in March. I guess they had other priorities. Don't be too hard on Libsodium: they only got it wrong because they reused the reference implementation. (Also, I don't see them breaking backward compatibility either.)
  - ktta 9y ago
    I'm also referring to the orgs who used Argon2 in production. If they move to the fixed version of Argon2 in libsodium, they'll have to make a plan to securely move to the correct implementation by hashing the passwords again. It can be done, but I'm wondering if they'll even bother.
    - user5994461 9y ago
      There is hardly anyone using Argon in production. The few hipsters who jumped on the new hashing algorithm bandwagon can't have much users.
- mrb 9y ago
  Those who deployed Argon2 must have deployed the reference implementation, therefore the code became the de facto standard. Khovratovich confirmed in https://github.com/P-H-C/phc-winner-argon2/issues/183#issuecomment-270814216 https://github.com/P-H-C/phc-winner-argon2/issues/183#issuec... that the code is the de facto standard and the spec will be edited to reflect this. libsodium implemented the same behavior simply because it reused the Argon2 reference implementation.
ktta 9y ago
I've always seen 'rolling your own crypto' as not being recommendation against writing your own library, but creating your own primitive. Sure, writing your own library is very difficult, but you have a simpler set of problems, which proper testing, another set of eyes and enough tools will take care of the big problems. Now, implementing your own primitive and recommending to use it is bad. For a primitive to be deemed secure, it has to undergo years of review and testing by seasoned cryptographers. This isn't a recommendation to write your own library, but my opinion that it is okay to do so if you want something that none of the major libraries offer.
- viraptor 9y ago
  Even just using crypto primitives is dangerous unless you've got experience. There's an old post from matasano about it https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2009/july/if-youre-typing-the-letters-a-e-s-into-your-code-youre-doing-it-wrong/ https://www.nccgroup.trust/us/about-us/newsroom-and-events/b...
  - ktta 9y ago
    Every line of code written can be a severe vulnerability, not just code dealing with crypto. In case of crypto, being well-informed is good enough to not mess up those kind of implementations (on a theoretical level). The problems mentioned can be pointed out and verified on paper. But you can't say the same thing for code.
    - Ar-Curunir 9y ago
      No, you can write crypto code, even when knowing stuff about the crypto, and still mess up. Crypto is problematic because you often cannot easily check whether you're wrong (it could be a tiny edge case that isn't tested because your input space is huge).
      - ktta 9y ago
        I didn't deny messing up crypto code is a thing. I'm saying messing up code is a separate problem from messing up how one architects projects using crypto. What you say as messing up with edge cases, etc. is a programming problem. The errors talked about in the link he posted is about using faulty crypto. Today it is common knowledge to never use ECB. What cipher mode you use is a problem you think about before you even start writing a single line of code. But you are talking about programming errors, which one can make in all areas, not only ones involving crypto. Those errors can break security similar to messing up crypto implementation.
fefe23 9y ago
I really liked that read, good writing style and the author was coming from the same place that I'm usually coming from (can we redo this ourselves and shave off some bloat in the process). However, as it stands now, it looks more like a cautionary tale on why he should have listened when people told him not to do it :-) I suspect a worthier goal would have been to look for ways to improve the linking footprint of libsodium instead of rolling your own library. Maybe turn the "I want the smaller version without the huge lookup table please" in a compile time option or so. That way more people could have used it without having to commit to "I'll use this little known experimental library instead of libsodium". Commit in the sense of "people will ask why and I will have to defend my decision", not in the sense of "oh no we are locked in!!"
- loup-vaillant 9y ago
  Reducing the linking footprint wasn't my only goal. Reducing the source code is also important, if only to facilitate tests and audits —thus increasing confidence. That huge table for instance would need to be tested. Every single value would need to be checked for correctness, or I would have to demonstrate they came from a sound generator. --- I've been careful not to lock people in. If people want to upgrade to faster primitives, they can. I don't think they will ever need to, though.
ZoFreX 9y ago
I am terrified that I do not consider myself competent enough to write a crypto library, and yet there isn't a single mention - in this article, nor at the time of writing the comments here on Hacker News - of many of the pitfalls I know to avoid when undertaking such an endeavour. There is even a list of "you have to do A, B, C, and that's about it" that is missing some major - well known, even! - items. I know "don't roll your own crypto" comes across as dismissive or patronising. I'm not a huge fan of the phrasing. But as a first-order approximation, it is correct. If you want to roll your own crypto, that needs to be your _thing_. It's very unlikely you're going to be a fantastic full-stack web developer _and_ be able to do that. If it's really what you want to do then awesome! Go study it, learn it, practice it. But if you're doing it as a side-hobby, never put it into production.
- loup-vaillant 9y ago
  Could you list the main pitfalls you are thinking about? Also, I may have mentioned some of them in this earlier article: http://loup-vaillant.fr/articles/rolling-your-own-crypto http://loup-vaillant.fr/articles/rolling-your-own-crypto
  - msl09 9y ago
    Hey, like you I decided to dive into cryptography coming from a different background. Although this quote is not directly related to your problem it can be safely applied to it: "Almost certainly you will get the urge to invent new cryptographic algorithms, and will believe that they are unbreakable. Don't resist the urge; this is one of the fun parts. But resist the belief; almost certainly your creations will be breakable, and almost certainly no one will spend the time breaking them for you. You can break them yourself as you get better."[1] The problem that Schneier is referring to is that doing a careful analysis (that is required if you want to use your crypto in production) is tedious, time consuming, and requires expertise in the area to know most blank spots of the algorithms (heck a single one is hard enough). That's why he recommends you to be have enough experience breaking many algorithms before you make any serious claim about your the safety of your crypto. So all in all it's great that you got the interest in the area, there is a lot of work needed in OSS. But be very careful about claiming that your lib is ready for production. What you got is a bunch of volunteers to glance at your code. Few cryptographers (if any) will seriously try to break it. [1] https://www.schneier.com/crypto-gram/archives/1999/1015.html#SoYouWanttobeaCryptographer https://www.schneier.com/crypto-gram/archives/1999/1015.html...
HurrdurrHodor 9y ago
TLDR: Crypto library with two claimed advantages: Better usability and smaller code than libsodium. However, it is also slower.
- Piskvorrr 9y ago
  Having all of that would be great, but most of the time you need to weigh one against another. Smaller code means smaller attack surface and supposedly fewer ways to shoot yourself in the foot ("I know, I'll use CBC"); of course, checking whether the author doesn't shoot you in the foot by choice of algorithms, or their implementation, is advised.
technion 9y ago
found an error in the Argon2 reference implementation. This is exactly why I like as many people writing crypto as possible[0]. Bugs like the one the writer found show up in reference implementations, and are found by absolutely no one unless someone writes an alternate implementation that behaves differently. Edit: A further example, a lot of people write crypto as part of various CTFs. This is a case of "writing crypto" that poses no threat to the community. [0] That doesn't mean they should use it in production.
flanfly 9y ago
I only had a brief look at it but it seems that crypto_check doesn't verify whenever the signers public key is on the curve.
- loup-vaillant 9y ago
  I'm not sure. If there is such a test, you should find it on `ge_frombytes_neg`. I bet it returns -1 in line 1315 if that happens. (It triggers a rejection when that happens.) If I'm right about this, it would explain why this code path is never hit: it would only trigger with invalid public keys. I'll test that as soon as I can.
rijoja 9y ago
Whenever I feel the need for a tin foil hat I start to wonder if there is a FUD campaign powered by the "establishment" to encourage people not to investigate this area of computer science so that security holes will remain unnoticed. But, yes I wouldn't start out on writing a crypto library, then again I wouldn't attempt to build an OS or a 3D stack or even an web server either. All cases where a security breach could have devastating effects as well.
- mrschwabe 9y ago
  Another angle is that a compromised security protocol used by a 100,000 developers is far more fruitful than having to assign a human specialist to crack & reverse engineer 100,000 uniquely thought-out implementations.
  - wolco 9y ago
    This is the point people miss. Same as wordpress vs creating your own blog. The more popular the software the larger the interest and the bigger exploit if successful.
    - arcticbull 9y ago
      It is less likely, though, and more likely to get fixed everywhere fast. 100,000 easy to crack implementations with obvious errors isn't really more secure than 1 super difficult, super well optimized/secured implementation, is it?
      - wolco 9y ago
        It is more difficult to crack 100,000 different interfaces because the time required to learn each one is a fixed cost. Having one central implementation makes it much easier to expliot because if an issue is discovered it can be immediately used everywhere. Don't put all your eggs in one basket.
      - calafrax 9y ago
        > more likely to get fixed everywhere fast openssl bugs were around for a very long time
snakeanus 9y ago
I really liked your previous articles/tutorials about Poly1305 and Chacha20. Is there any plain to make similar articles about the other constructions that you implemented on this library? Is there also any plain to implement scrypt (based on the salsa), SPHINCS (uses BLAKE(1) and Chacha), Ed448-Goldilocks, Keccak or any of the CAESAR candidates? (The ones that seemed the most interesting to me were NORX, Keyak (based on keccak) and HS1-SIV (based on chacha)).
- loup-vaillant 9y ago
  I don't think I'll write for the other primitives, I don't know them well enough to comment on them properly. I won't add other primitives to Monocypher, that would defeat the purpose of a "one true cipher suite". One stream cipher, one authenticator, one hash, one password key derivation, one key exchange, one signature. That is enough. When some of those primitives becomes obsolete, I'll consider writing a new, incompatible version of Monocypher. For instance, we can switch to curve448 if curve25519 isn't secure enough. I think at least a decade will pass before we come to that, though.
jesse_m 9y ago
It's interesting that Frama-C was mentioned. I have been playing with it the last few months and would love to hear exactly how it was used. I've read through a lot of the documentation but haven't found many examples of it being used
- dhekir 9y ago
  Part of it is described in the Frama-C blog (http://blog.frama-c.com/index.php?post/2017/03/07/A-simple-EVA-tutorial http://blog.frama-c.com/index.php?post/2017/03/07/A-simple-E...). It actually presents a tutorial on the EVA plug-in (former Value Analysis) with Monocypher as a case study. Note that this is the initial stage of using Frama-C. The next step would probably include writing some specifications for functional behavior (in ACSL), and then using the WP plug-in (based on weakest precondition calculus) to prove those specifications, providing much stronger guarantees. A few Frama-C case studies are on Github (https://github.com/Frama-C/open-source-case-studies https://github.com/Frama-C/open-source-case-studies), including an old release of Monocypher, but they constitute mostly a initial parametrization of the EVA plug-in. Most advanced users of Frama-C apply it on industrial code that is not available, but academic users regularly publish papers with case studies (searching "frama-c" on Google Scholar gives good results).
etatoby 9y ago
I have read the article and I still don't understand what was the purpose of such a project. What's wrong with ZIP or 7-Zip + AES encryption? Or GPG with a GUI? Or any other existing tool that's already stable and multi-platform?
- snakeanus 9y ago
  This is a library, unlike the ones that you mentioned.
xmichael99 9y ago
So where is the Mono / C# wrapper, or C# implementation of this? (:
- loup-vaillant 9y ago
  The only bindings I'm aware of right now are for Crystal (it may or may not be up to date): https://github.com/konovod/monocypher.cr https://github.com/konovod/monocypher.cr I think I will personally do C++, OCaml, and perhaps Lua. For the rest, either someone contributes, or you'll have to look at the FFI yourself…
chmike 9y ago
Have the library functions been tested (or proved) to be constant time ? This is a critical property of libsodium. Another question is what make the difference in number of line of codes with libsodium ? I mean what do these lines of of libsodium that your lib doesn't have contain ? Also, is this a fair metric ? You know we can have 1000 char long lines in C. ;)
- loup-vaillant 9y ago
  > Have the library functions been tested (or proved) to be constant time ? Just check the absence of secret dependent branches, and secret dependent array indices, it's pretty simple. > what make the difference in number of line of codes with libsodium ? Less lines means less need for eyeballs, tests, and audits. Libraries the size of TweetNaCl (and, to a lesser extent, Monocypher), can be subjected to very rigorous audits, and increase confidence accordingly. > what do these lines of of libsodium that your lib doesn't have contain ? I can see 3 sources: redundant primitives (Monocypher is not redundant), alternate implementation (Monocypher sticks to portable C), and verbose implementations (Monocypher keeps it simple). > Also, is this a fair metric ? Not quite: I did not count the size of the header in Monocypher and TweetNaCl. Still, I don't think Libsodium has less than 22K lines of actual code. (Note: I counted everything with sloccount) On the other hand, my lines never exceed 80 characters.
chmike 9y ago
Have the library functions been tested (or proved) to be constant time ? This is a critical property of libsodium. Another question is what make the difference in number of line of codes with libsodium ? Is this a fair metric ? You know we van have 1000 cha long lines in C. ;)
chmike 9y ago
Have the library functions been tested (or proved) to be constant time ? This is a critical property of libsodium. Another question is what make the difference in number of line of codes with libsodium ? Is this a fair metric ? You know we van have 1000 cha long lines in C. ;)
glacier 9y ago
It's good that you found an error in the argon2 test-vectors. Implementing a basic crypto-library is one of my side-projects and argon2 is basically what is next on my list.
dexterdog 9y ago
I thought the first rule of crypto was never role your own crypto.
- loup-vaillant 9y ago
  I'm the exception. :-) Seriously though, Monocypher is getting getting past the "own crypto" stage at this point.
tptacek 9y ago
I would like to clarify: The thing that I said was "table stakes" for implementing cryptography was passing the algorithm test vectors, which this author's previous post claimed as a security feature. If you're unfamiliar with the concept, a test vector is a series of strings and intermediate values used to ensure that your (say) OCB3 is the same as everyone else's OCB3. Had he asked, I'd further claim that not having dumb C bugs is also table stakes for cryptography, since serious crypto vulnerabilities happen at a higher level of abstraction. The author grazes past (somewhat disconcertingly) one such class of bugs when he discusses carry propagation and limb scheduling in the context of Poly1305. Improper carry propagation is an example of the kind of security vulnerability for which there are no test vectors and no memory safety validation tools. You just have to know what a carry propagation bug is, where to look for them (not Poly1305), and what the impact of one is. Hopefully, the author of this library does.
- lisper 9y ago
  > You just have to know what a carry propagation bug is This is one of the things that drives me nuts about the crypto community. The natural response upon reading, "You just have to know X" if you don't already know is to go search for X. Well, if you go search for "carry propagation bug" you will find lots of examples of carry propagation bugs being found and fixed, but no explanation of what one is. The thing that "you just have to know" is not so easy to find. The courteous thing to do when mentioning something that "you just have to know" but which is not easy to find is to provide a pointer or a short in-line explanation. When you don't do that you leave your reader with no choice but to publicly expose their ignorance of this purportedly crucial knowledge by asking, which I will now proceed to do as a public service: What the hell is a carry propagation bug? Where do you look for them? And what is their impact? And while we're at it, what are "table stakes"?
  - steveklabnik 9y ago
    "table stakes" is an idiom that comes from poker. It's the amount you can win or lose in a single hand, that is, the smallest amount of money you need to have to be able to sit at the table and play the game.
HurrdurrHodor 9y ago
Write a crypto library and blog about how great it is. Apply the tools that other people used to find bugs in it and repeat blogging about how great it is. ... In the end you get a library that is slower and smaller (because you left out the optimized code, duh). It's still not as small as the smallest. Is this easier to audit? Dubious but it doesn't matter because if people hadn't wanted the extra speed they wouldn't have added the optimized code anyway. Plus, that code is already audited and rolled out and works! Who cares about auditing another library that provides nothing new? The only non obviously-garbage argument here is usability which I am too lazy to look at because it is too fuzzy to refute anyway. By all means, write your own crypto but DON'T USE IT! And of course don't tell other people to use it either.
- loup-vaillant 9y ago
  Let's list the pros and cons of each libraries (let's assume we trust all 3 libraries): Monocypher vs TweetNaCl: Good: Monocypher uses more modern primitives (Blake2b, Chacha20) Monocypher provides password derivation Monocypher is a bit easier to use Monocypher is much faster Bad: Monocypher is twice as big Conclusion: Monocypher utterly outclasses TweetNaCl. Monocypher vs Libsodium: Good: Monocypher is much smaller Monocypher is easier to deploy Monocypher is a bit easier to use Bad: Monocypher is a bit slower (except Argon2i, which is faster) Conclusion: If performance matters, use NaCl. Otherwise, use Monocypher. Nothing new, you say? > By all means, write your own crypto but DON'T USE IT! And of course don't tell other people to use it either. Get back to me in a couple years, we'll count the CEVs since version 1.0.
vadiml 9y ago
Hi, I've browsed through the code and i don't see why say it is not good in MT setting... an first view the code is completely reentrant... Another thought: i strongly suspect that inlining these store_le... and load_le... function will show pretty significant performance boost
gizmo385 9y ago
Heads up the the author: The Reddit and HN links in the timeline are switched :)
pier25 9y ago
I see crypto as black magic. Can anyone recommend some introductory resource to demystify it a bit? Why is it so hard?
- loup-vaillant 9y ago
  Here's a good introduction to cryptography: https://www.crypto101.io/ https://www.crypto101.io/
convivialdingo 9y ago
Hey, fantastic work! Really happy to see some amateur crypto development out there. I started as an amateur also, now I work in crypto product dev. Definitely impressed if this is your first crypto work. I really like the simplicity, reminds me of OpenSSH. I gotta agree with some of the naysayers though - this needs more time to brew. Even if it's perfect, there still more proof to show. My suggestions: * A long-running interoperability test between your lib and another. Test vectors don't catch everything. Run it for a week, check results and publish the results. * Fuzzing. You need to to catch false results, random crashes, stack overflows, etc. Fuzz every input, do it for random lengths (both underflow and overflow). * Maybe port it and test on Win, Mac, ARM, MIPS. I find a lot of bugs shake out during ports. * Timing. You can't prevent all side-channel attacks, but bits going out are easy attack vectors. Use a high-precision timing mechanism to catch any unusual spikes. Capture a few hours of points for each functional path, and publish the results. (crypto_aead_unlock has two return paths, it might be better to maintain timing). * Personal preferences: Rather than using int returns, use bool. Zero your ctx vars, e.g. a_ctx ctx = {0,}; I spend more time in testing products than development, honestly. Publish as much relevant data as you can.
- loup-vaillant 9y ago
  > Definitely impressed if this is your first crypto work. It is. :-) > A long-running interoperability test between your lib and another. Already have one for all primitives. Just tweak the parameters to make it run for a long time. The test code is in test/sodium.c and test/donna.c > Fuzzing. You need to to catch false results, random crashes, stack overflows, etc. Fuzz every input, do it for random lengths (both underflow and overflow). I'm not sure how to go about it. I already check every input size, from zero to several times the size of the internal block, in my comparison tests. Of course, all inputs are correct. Incorrect inputs result in undefined behaviour, checking for that is out of scope. (That's why most functions return void.) > Maybe port it and test on Win, Mac, ARM, MIPS. I find a lot of bugs shake out during ports. Yeah, I only tried GCC and Clang on my Intel Ubuntu machine. But I did use Valgrind, ASan, MSan, UBSan, and the TIS-Interpreter. I also compiled under various standards: C99, C11, C++98, C++11, C++14, C++17. > Timing. […] Use a high-precision timing mechanism to catch any unusual spikes Isn't that a bit overkill? I'd rather manually review the source code for the absence of secret dependant branches or secret dependent array indices. I'm not implementing AES or RSA, the primitive I have chosen were specifically designed to easily avoid timing attacks. > crypto_aead_unlock has two return paths, it might be better to maintain timing. No. There are 2 cases to consider: failure, and success. Each path runs in constant time, so timing can only distinguish success from failure… which is revealed anyway by the return code. > Rather than using int returns, use bool. I'm afraid bool is a tiny bit less portable than int. I also went with the flow about error codes, which are traditionally int. > Zero your ctx vars, e.g. a_ctx ctx = {0,}; I'm not sure how to do it portably, with zero dependency.
gwerks 9y ago
Inb4 "don't roll your own crypto"

380 points by loup-vaillant 9y ago

deleted 9y ago

[deleted]

sigjuice 9y ago

Are there other fields where the slogan "don't roll your own XXX unless you are an infallible expert" is applicable?

kobeya 9y ago
Aviation, medical devices, civil engineering.
jrimbault 9y ago
If you were a cook you wouldn't really be qualified to build a stove or make your own knives, but you'd know what kind of stove and knives you'd want.
dlgeek 9y ago
Parachute?
- kabouseng 9y ago
  Actually you do roll your own parachute once you have done enough jumps. But your reserve chute is always rolled by a certified professional specifically certified to roll reserve chutes, and not by yourself. (if I remember correctly, my memory is a bit fuzzy from when I jumped a couple of years back)
  - Piskvorrr 9y ago
    Well, you are using the product, but not manufacturing it.
- sebcat 9y ago
  You don't have to be an expert to make/jump the actual parachute/canopy. As long as the harness, container and reserve are TSO'd you're good to go. It's a lot of work though, at least for a ram-air. EDIT: Some interesting discussion here: http://www.dropzone.com/cgi-bin/forum/gforum.cgi?post=4822410 http://www.dropzone.com/cgi-bin/forum/gforum.cgi?post=482241...
- karrotwaltz 9y ago
  I'm not sure about "don't roll your own parachute" as it should be common sense, but well, in 1912 a parachute inventor died while testing his creation. https://en.wikipedia.org/wiki/Franz_Reichelt https://en.wikipedia.org/wiki/Franz_Reichelt
- cromulent 9y ago
  I know a few guys who have manufactured their own canopies and harness/container systems in their garage, and I've jumped them - even though I consider these guys fallible :)
KineticLensman 9y ago
"Hey y'all, hold my beer and look at this!" There are [1] several non-aviation examples: - Blood transfusion - Steam-powered bicycles - Rotary printing presses - Submarines - Luminescent paint - Hi-speed jet-powered railcars - Ropes /pulleys to help the user get out of bed [1] https://en.wikipedia.org/wiki/List_of_inventors_killed_by_their_own_inventions https://en.wikipedia.org/wiki/List_of_inventors_killed_by_th...
- ithkuil 9y ago
  > - Steam-powered bicycles On a first read I though "yeah, sure, looks quite dangerous indeed", then I read: Sylvester H. Roper, inventor of the eponymous steam-powered bicycle, died of a heart attack or subsequent crash during a public speed trial in 1896. It is unknown whether the crash caused the heart attack or vice versa.
  - Piskvorrr 9y ago
    This seems to mean "killed in an interesting way". People killed by their inventions in petrol-based transportation seem to be too numerous even for a Wikipedia list: "yeah, people die in cars en masse, that's not really newsworthy."
- kpil 9y ago
  Best Wikipedia page ever!
socceroos 9y ago
reefer?
inopinatus 9y ago
- Terms & conditions of use - Chocolate soufflé - Distributed lock manager - Autonomous war robot - init(1) replacement I am personally guilty of attempting four of these things without adequate preparation or expertise.
- pzduniak 9y ago
  I seriously hope that you didn't try to write an init(1) replacement!

mathieubordere 9y ago

I still believe it's a bad idea to roll your own crypto, interesting as an exercise, but I would never use it in a production environment.

merb 9y ago
well if you have enough people especially working with crypto, than why not? It's not like that the existing solutions came out of nowhere. some people actually wrote them. I mean libsodium is actually really simple to use. however before libsodium it was probably easier and mostly more correct to roll your own. there are so many things you can do wrong with other libraries that the chance to do it wrong is as high as in your own implementation.
timclark 9y ago
How will cryptography libraries ever develop if someone doesn't roll their own?
- rootlocus 9y ago
  Through sweat and blood. Don't expect crypto libraries to be secure the first few years after their release. That's why people favor the mature, battle-tested libraries.
  - jwl 9y ago
    But then again, we still got Heartbleed.
    - bananaboy 9y ago
      Yeah and imagine the kind of exciting bugs that exist in new and immature libraries! :)
      - jwl 9y ago
        Yes, but are hackers going after common used libraries to get more vulnerable systems to attack or are they going to spend time on some unknown homebuilt crypto? In some cases, security through obscurity works well in practice.

conradk 9y ago

The v1.0.0 was published the day a bug was fixed and yet the first sentence of this post is "Monocypher is ready for production". Sure, OpenSSL has bugfixes on regular basis as well, but I believe that further fragmenting the crypto ecosystem with a new library and little commercial support to properly back it and pay for security audits is risky.

baby 9y ago
It's risky but sometimes it's the only way. To be fair the code size is relatively small so it should be easier to audit than other products. My guess is that his blog post is also some motivation to get new pair of eyes on the library. If there's any fund for that, I'd be happy to look at it ;)
loup-vaillant 9y ago
> The v1.0.0 was published the day a bug was fixed and yet the first sentence of this post is "Monocypher is ready for production". You have to consider the nature of the bug, as well as the fix. The bug was an Undefined Behaviour that had no effect on the binaries I could generate. The fix was 10 characters. Running the automated test suite took longer than fixing the bug. If there were reasons to delay 1.0, this bug was not it. > I believe that further fragmenting the crypto ecosystem with a new library and little commercial support to properly back it and pay for security audits is risky. I agree fragmentation is a problem. I just couldn't satisfy myself with existing alternatives. As for support, I designed Monocypher specifically to need very little of it. Ideally, the only notable changes from now on will be about the manual and the test suite.

baby 9y ago

First let me say that I've followed your work a bit and I'm really impressed with the library. I'm currently planning on rolling my own crypto as well and your library is on my list of the things to look at. One question: > I was shifting a uint8_t, 24 bits to the left. I failed to realise that integer promotion means this unsigned byte would be converted to a signed integer, and overflow if the byte exceeded 127. This sounds weird to me, how is a shift (which only apply itself on the bit value, not the number value) triggering a sign extension? I don't think you're talking about integer promotion here unless you had a uint16_t = uint8_t << 24 or something like this.

kbeckmann 9y ago
I believe this[1] is the patch that fixes this bug. I tried to reproduce the behavior but couldn't succeed. Maybe I was doing something wrong. Would really appreciate it if someone here could show a test case where this matters. [1] https://github.com/LoupVaillant/Monocypher/commit/347189c50c053cf13ce1310818c2913f4904c1eb https://github.com/LoupVaillant/Monocypher/commit/347189c50c...
- baby 9y ago
  `ctx->c` is a u32[1] and msg is u8. So yeah there is integer promotion here. [1]: https://github.com/LoupVaillant/Monocypher/blob/master/src/monocypher.h#L58 https://github.com/LoupVaillant/Monocypher/blob/master/src/m...
- loup-vaillant 9y ago
  Your failure to reproduce the behaviour is normal: on the compilers I have tested, this doesn't affect the generated binary. To see the difference, you need to first modify the makefile to use UBSan instead of just GCC (just comment the active CC line, and uncomment the right one). You may want to tone down the optimisations for faster compilation as well. Then you can run `make clean` and `./test.sh`. You should have a warning on commits prior to this patch.
- deleted 9y ago
  [deleted]

AstralStorm 9y ago

Main lesson not learned: instead of testing, prove correctness in high assurance code like this. Rigorously and formally. Preferably even refine the proof to executable code. (Yes, it would take somewhere on the order of 10k LOC to prove correctness of this 1k.)

2bitencryption 9y ago
I'm sure you encounter "high assurance code" like this every day -- your operating system, the secure protocol used to post your comment, the controls for your Anti-Lock Breaking System in your car. How much of those were formally proven?
- yorwba 9y ago
  TLS has been subject to lots of formal verification. The other things, probably not so much. Verifying some (but not all) properties of crypto is relatively (but not absolutely) easy, because it's essentially just math with pure functions. Operating systems and physical controllers with real-time constraints have different requirements and more interacting state, which means that the desired properties are not easily expressed as a mathematical expression.
baby 9y ago
https://github.com/mitls/hacl-star https://github.com/mitls/hacl-star
jfries 9y ago
In this case 3 of the bugs would be caught when trying to prove the absense of undefined behaviour/use of uninitialized value. Can existing formal proof tools be used for that?
SAI_Peregrinus 9y ago
Proving correctness isn't enough. You have to test it too. It's perfectly possible for the proof to have errors (bugs). It's also possible for the proof to be incomplete: you can prove the algorithm is correct and implemented correctly to spec and still have vulnerabilities.

reikonomusha 9y ago

It was a fun read, except when all the C bugs were listed. It's getting tiresome to hear about folks writing nominally Important software in a language where it is extraordinarily difficult to get things absolutely correct, with paltry excuses such as "it needs to be fast" or "it needs to be portable [0] to a VAX-11/750." It doesn't give me confidence that these completely usual bugs popped up early on, and it will not surprise me when more show themselves. The author's lack of practical familiarity with the memory hierarchy, caching, and pipelining—evidenced by his surprise by the speed gained by fetching more than a byte at a time—also makes me suspicious of his understanding of the semantics of his program. (I understand that this is a form of ad hominem as it relates to arguing the correctness of software, but it's a consideration worth noting in a broader context of trust.) It would be great if this was written in another language. [0] Most "portable" code written in C is not portable. Look at any mature C project and see the layers and layers of macros and hacks to make things portable.

taneq 9y ago
> Look at any mature C project and see the layers and layers of macros and hacks to make things portable. So.. it's portable, then?
- reikonomusha 9y ago
  Relying on a macro processor to patch and bandage your program at compile time depending on a collection of symbolic features isn't really representative of portability of a language and your code. To me, that is brute forcing portability, and is a web of conditional inclusions/exclusions that needs to be maintained and added to per-architecture and per-OS. Something that is, in some sense, truly portable is one that has a single specification that runs on all platforms the base implementation supports. Virtual machines and certain styles of language standard are often able to accomplish portability for a vast number of applications.
  - SmallDeadGuy 9y ago
    I would argue the other way. Something which requires a base implementation to be installed on a system which generates different machine code based on a single specification is less portable than a macro processor generating different versions of a program. Sure, people will have had something like python installed since a long time ago, but a new setup requires downloading 2 objects (the base implementation and the program) compared to a single one (a platform-specific copy of a program).

ktta 9y ago

I'm amazed that the reference implementation of Argon2 had a bug. So that means anyone who deployed Argon2 today didn't really use Argon2 (I'm being pedantic), but something else? libsodium also got it wrong, so that brought down my opinion of it being a trusted and well-reviewed library. Now the question is, will they continue to use the same implementation or move to the fixed code?

loup-vaillant 9y ago
Maybe the authors will fix the code in subsequent versions of Argon2. Right now, backward compatibility is deemed more important. The effects of this bug are practically negligible anyway. I bet a single bit of additional entropy in a password would compensate that a hundred fold. I'm a bit disappointed however at their not updating the specs. I signalled the bug in January, and the latest version of the appear to have been published in March. I guess they had other priorities. Don't be too hard on Libsodium: they only got it wrong because they reused the reference implementation. (Also, I don't see them breaking backward compatibility either.)
- ktta 9y ago
  I'm also referring to the orgs who used Argon2 in production. If they move to the fixed version of Argon2 in libsodium, they'll have to make a plan to securely move to the correct implementation by hashing the passwords again. It can be done, but I'm wondering if they'll even bother.
  - user5994461 9y ago
    There is hardly anyone using Argon in production. The few hipsters who jumped on the new hashing algorithm bandwagon can't have much users.
mrb 9y ago
Those who deployed Argon2 must have deployed the reference implementation, therefore the code became the de facto standard. Khovratovich confirmed in https://github.com/P-H-C/phc-winner-argon2/issues/183#issuecomment-270814216 https://github.com/P-H-C/phc-winner-argon2/issues/183#issuec... that the code is the de facto standard and the spec will be edited to reflect this. libsodium implemented the same behavior simply because it reused the Argon2 reference implementation.

ktta 9y ago

I've always seen 'rolling your own crypto' as not being recommendation against writing your own library, but creating your own primitive. Sure, writing your own library is very difficult, but you have a simpler set of problems, which proper testing, another set of eyes and enough tools will take care of the big problems. Now, implementing your own primitive and recommending to use it is bad. For a primitive to be deemed secure, it has to undergo years of review and testing by seasoned cryptographers. This isn't a recommendation to write your own library, but my opinion that it is okay to do so if you want something that none of the major libraries offer.

viraptor 9y ago
Even just using crypto primitives is dangerous unless you've got experience. There's an old post from matasano about it https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2009/july/if-youre-typing-the-letters-a-e-s-into-your-code-youre-doing-it-wrong/ https://www.nccgroup.trust/us/about-us/newsroom-and-events/b...
- ktta 9y ago
  Every line of code written can be a severe vulnerability, not just code dealing with crypto. In case of crypto, being well-informed is good enough to not mess up those kind of implementations (on a theoretical level). The problems mentioned can be pointed out and verified on paper. But you can't say the same thing for code.
  - Ar-Curunir 9y ago
    No, you can write crypto code, even when knowing stuff about the crypto, and still mess up. Crypto is problematic because you often cannot easily check whether you're wrong (it could be a tiny edge case that isn't tested because your input space is huge).
    - ktta 9y ago
      I didn't deny messing up crypto code is a thing. I'm saying messing up code is a separate problem from messing up how one architects projects using crypto. What you say as messing up with edge cases, etc. is a programming problem. The errors talked about in the link he posted is about using faulty crypto. Today it is common knowledge to never use ECB. What cipher mode you use is a problem you think about before you even start writing a single line of code. But you are talking about programming errors, which one can make in all areas, not only ones involving crypto. Those errors can break security similar to messing up crypto implementation.

fefe23 9y ago

I really liked that read, good writing style and the author was coming from the same place that I'm usually coming from (can we redo this ourselves and shave off some bloat in the process). However, as it stands now, it looks more like a cautionary tale on why he should have listened when people told him not to do it :-) I suspect a worthier goal would have been to look for ways to improve the linking footprint of libsodium instead of rolling your own library. Maybe turn the "I want the smaller version without the huge lookup table please" in a compile time option or so. That way more people could have used it without having to commit to "I'll use this little known experimental library instead of libsodium". Commit in the sense of "people will ask why and I will have to defend my decision", not in the sense of "oh no we are locked in!!"

loup-vaillant 9y ago
Reducing the linking footprint wasn't my only goal. Reducing the source code is also important, if only to facilitate tests and audits —thus increasing confidence. That huge table for instance would need to be tested. Every single value would need to be checked for correctness, or I would have to demonstrate they came from a sound generator. --- I've been careful not to lock people in. If people want to upgrade to faster primitives, they can. I don't think they will ever need to, though.

ZoFreX 9y ago

I am terrified that I do not consider myself competent enough to write a crypto library, and yet there isn't a single mention - in this article, nor at the time of writing the comments here on Hacker News - of many of the pitfalls I know to avoid when undertaking such an endeavour. There is even a list of "you have to do A, B, C, and that's about it" that is missing some major - well known, even! - items. I know "don't roll your own crypto" comes across as dismissive or patronising. I'm not a huge fan of the phrasing. But as a first-order approximation, it is correct. If you want to roll your own crypto, that needs to be your _thing_. It's very unlikely you're going to be a fantastic full-stack web developer _and_ be able to do that. If it's really what you want to do then awesome! Go study it, learn it, practice it. But if you're doing it as a side-hobby, never put it into production.

loup-vaillant 9y ago
Could you list the main pitfalls you are thinking about? Also, I may have mentioned some of them in this earlier article: http://loup-vaillant.fr/articles/rolling-your-own-crypto http://loup-vaillant.fr/articles/rolling-your-own-crypto
- msl09 9y ago
  Hey, like you I decided to dive into cryptography coming from a different background. Although this quote is not directly related to your problem it can be safely applied to it: "Almost certainly you will get the urge to invent new cryptographic algorithms, and will believe that they are unbreakable. Don't resist the urge; this is one of the fun parts. But resist the belief; almost certainly your creations will be breakable, and almost certainly no one will spend the time breaking them for you. You can break them yourself as you get better."[1] The problem that Schneier is referring to is that doing a careful analysis (that is required if you want to use your crypto in production) is tedious, time consuming, and requires expertise in the area to know most blank spots of the algorithms (heck a single one is hard enough). That's why he recommends you to be have enough experience breaking many algorithms before you make any serious claim about your the safety of your crypto. So all in all it's great that you got the interest in the area, there is a lot of work needed in OSS. But be very careful about claiming that your lib is ready for production. What you got is a bunch of volunteers to glance at your code. Few cryptographers (if any) will seriously try to break it. [1] https://www.schneier.com/crypto-gram/archives/1999/1015.html#SoYouWanttobeaCryptographer https://www.schneier.com/crypto-gram/archives/1999/1015.html...

HurrdurrHodor 9y ago

TLDR: Crypto library with two claimed advantages: Better usability and smaller code than libsodium. However, it is also slower.

Piskvorrr 9y ago
Having all of that would be great, but most of the time you need to weigh one against another. Smaller code means smaller attack surface and supposedly fewer ways to shoot yourself in the foot ("I know, I'll use CBC"); of course, checking whether the author doesn't shoot you in the foot by choice of algorithms, or their implementation, is advised.

technion 9y ago

found an error in the Argon2 reference implementation. This is exactly why I like as many people writing crypto as possible[0]. Bugs like the one the writer found show up in reference implementations, and are found by absolutely no one unless someone writes an alternate implementation that behaves differently. Edit: A further example, a lot of people write crypto as part of various CTFs. This is a case of "writing crypto" that poses no threat to the community. [0] That doesn't mean they should use it in production.

flanfly 9y ago

I only had a brief look at it but it seems that crypto_check doesn't verify whenever the signers public key is on the curve.

loup-vaillant 9y ago
I'm not sure. If there is such a test, you should find it on `ge_frombytes_neg`. I bet it returns -1 in line 1315 if that happens. (It triggers a rejection when that happens.) If I'm right about this, it would explain why this code path is never hit: it would only trigger with invalid public keys. I'll test that as soon as I can.

rijoja 9y ago

Whenever I feel the need for a tin foil hat I start to wonder if there is a FUD campaign powered by the "establishment" to encourage people not to investigate this area of computer science so that security holes will remain unnoticed. But, yes I wouldn't start out on writing a crypto library, then again I wouldn't attempt to build an OS or a 3D stack or even an web server either. All cases where a security breach could have devastating effects as well.

mrschwabe 9y ago
Another angle is that a compromised security protocol used by a 100,000 developers is far more fruitful than having to assign a human specialist to crack & reverse engineer 100,000 uniquely thought-out implementations.
- wolco 9y ago
  This is the point people miss. Same as wordpress vs creating your own blog. The more popular the software the larger the interest and the bigger exploit if successful.
  - arcticbull 9y ago
    It is less likely, though, and more likely to get fixed everywhere fast. 100,000 easy to crack implementations with obvious errors isn't really more secure than 1 super difficult, super well optimized/secured implementation, is it?
    - wolco 9y ago
      It is more difficult to crack 100,000 different interfaces because the time required to learn each one is a fixed cost. Having one central implementation makes it much easier to expliot because if an issue is discovered it can be immediately used everywhere. Don't put all your eggs in one basket.
    - calafrax 9y ago
      > more likely to get fixed everywhere fast openssl bugs were around for a very long time

snakeanus 9y ago

I really liked your previous articles/tutorials about Poly1305 and Chacha20. Is there any plain to make similar articles about the other constructions that you implemented on this library? Is there also any plain to implement scrypt (based on the salsa), SPHINCS (uses BLAKE(1) and Chacha), Ed448-Goldilocks, Keccak or any of the CAESAR candidates? (The ones that seemed the most interesting to me were NORX, Keyak (based on keccak) and HS1-SIV (based on chacha)).

loup-vaillant 9y ago
I don't think I'll write for the other primitives, I don't know them well enough to comment on them properly. I won't add other primitives to Monocypher, that would defeat the purpose of a "one true cipher suite". One stream cipher, one authenticator, one hash, one password key derivation, one key exchange, one signature. That is enough. When some of those primitives becomes obsolete, I'll consider writing a new, incompatible version of Monocypher. For instance, we can switch to curve448 if curve25519 isn't secure enough. I think at least a decade will pass before we come to that, though.

jesse_m 9y ago

It's interesting that Frama-C was mentioned. I have been playing with it the last few months and would love to hear exactly how it was used. I've read through a lot of the documentation but haven't found many examples of it being used

dhekir 9y ago
Part of it is described in the Frama-C blog (http://blog.frama-c.com/index.php?post/2017/03/07/A-simple-EVA-tutorial http://blog.frama-c.com/index.php?post/2017/03/07/A-simple-E...). It actually presents a tutorial on the EVA plug-in (former Value Analysis) with Monocypher as a case study. Note that this is the initial stage of using Frama-C. The next step would probably include writing some specifications for functional behavior (in ACSL), and then using the WP plug-in (based on weakest precondition calculus) to prove those specifications, providing much stronger guarantees. A few Frama-C case studies are on Github (https://github.com/Frama-C/open-source-case-studies https://github.com/Frama-C/open-source-case-studies), including an old release of Monocypher, but they constitute mostly a initial parametrization of the EVA plug-in. Most advanced users of Frama-C apply it on industrial code that is not available, but academic users regularly publish papers with case studies (searching "frama-c" on Google Scholar gives good results).

etatoby 9y ago

I have read the article and I still don't understand what was the purpose of such a project. What's wrong with ZIP or 7-Zip + AES encryption? Or GPG with a GUI? Or any other existing tool that's already stable and multi-platform?

snakeanus 9y ago
This is a library, unlike the ones that you mentioned.

xmichael99 9y ago

So where is the Mono / C# wrapper, or C# implementation of this? (:

loup-vaillant 9y ago
The only bindings I'm aware of right now are for Crystal (it may or may not be up to date): https://github.com/konovod/monocypher.cr https://github.com/konovod/monocypher.cr I think I will personally do C++, OCaml, and perhaps Lua. For the rest, either someone contributes, or you'll have to look at the FFI yourself…

chmike 9y ago

Have the library functions been tested (or proved) to be constant time ? This is a critical property of libsodium. Another question is what make the difference in number of line of codes with libsodium ? I mean what do these lines of of libsodium that your lib doesn't have contain ? Also, is this a fair metric ? You know we can have 1000 char long lines in C. ;)

loup-vaillant 9y ago
> Have the library functions been tested (or proved) to be constant time ? Just check the absence of secret dependent branches, and secret dependent array indices, it's pretty simple. > what make the difference in number of line of codes with libsodium ? Less lines means less need for eyeballs, tests, and audits. Libraries the size of TweetNaCl (and, to a lesser extent, Monocypher), can be subjected to very rigorous audits, and increase confidence accordingly. > what do these lines of of libsodium that your lib doesn't have contain ? I can see 3 sources: redundant primitives (Monocypher is not redundant), alternate implementation (Monocypher sticks to portable C), and verbose implementations (Monocypher keeps it simple). > Also, is this a fair metric ? Not quite: I did not count the size of the header in Monocypher and TweetNaCl. Still, I don't think Libsodium has less than 22K lines of actual code. (Note: I counted everything with sloccount) On the other hand, my lines never exceed 80 characters.

chmike 9y ago

Have the library functions been tested (or proved) to be constant time ? This is a critical property of libsodium. Another question is what make the difference in number of line of codes with libsodium ? Is this a fair metric ? You know we van have 1000 cha long lines in C. ;)

chmike 9y ago

Have the library functions been tested (or proved) to be constant time ? This is a critical property of libsodium. Another question is what make the difference in number of line of codes with libsodium ? Is this a fair metric ? You know we van have 1000 cha long lines in C. ;)

glacier 9y ago

It's good that you found an error in the argon2 test-vectors. Implementing a basic crypto-library is one of my side-projects and argon2 is basically what is next on my list.

dexterdog 9y ago

I thought the first rule of crypto was never role your own crypto.

loup-vaillant 9y ago
I'm the exception. :-) Seriously though, Monocypher is getting getting past the "own crypto" stage at this point.

tptacek 9y ago

I would like to clarify: The thing that I said was "table stakes" for implementing cryptography was passing the algorithm test vectors, which this author's previous post claimed as a security feature. If you're unfamiliar with the concept, a test vector is a series of strings and intermediate values used to ensure that your (say) OCB3 is the same as everyone else's OCB3. Had he asked, I'd further claim that not having dumb C bugs is also table stakes for cryptography, since serious crypto vulnerabilities happen at a higher level of abstraction. The author grazes past (somewhat disconcertingly) one such class of bugs when he discusses carry propagation and limb scheduling in the context of Poly1305. Improper carry propagation is an example of the kind of security vulnerability for which there are no test vectors and no memory safety validation tools. You just have to know what a carry propagation bug is, where to look for them (not Poly1305), and what the impact of one is. Hopefully, the author of this library does.

lisper 9y ago
> You just have to know what a carry propagation bug is This is one of the things that drives me nuts about the crypto community. The natural response upon reading, "You just have to know X" if you don't already know is to go search for X. Well, if you go search for "carry propagation bug" you will find lots of examples of carry propagation bugs being found and fixed, but no explanation of what one is. The thing that "you just have to know" is not so easy to find. The courteous thing to do when mentioning something that "you just have to know" but which is not easy to find is to provide a pointer or a short in-line explanation. When you don't do that you leave your reader with no choice but to publicly expose their ignorance of this purportedly crucial knowledge by asking, which I will now proceed to do as a public service: What the hell is a carry propagation bug? Where do you look for them? And what is their impact? And while we're at it, what are "table stakes"?
- steveklabnik 9y ago
  "table stakes" is an idiom that comes from poker. It's the amount you can win or lose in a single hand, that is, the smallest amount of money you need to have to be able to sit at the table and play the game.

HurrdurrHodor 9y ago

Write a crypto library and blog about how great it is. Apply the tools that other people used to find bugs in it and repeat blogging about how great it is. ... In the end you get a library that is slower and smaller (because you left out the optimized code, duh). It's still not as small as the smallest. Is this easier to audit? Dubious but it doesn't matter because if people hadn't wanted the extra speed they wouldn't have added the optimized code anyway. Plus, that code is already audited and rolled out and works! Who cares about auditing another library that provides nothing new? The only non obviously-garbage argument here is usability which I am too lazy to look at because it is too fuzzy to refute anyway. By all means, write your own crypto but DON'T USE IT! And of course don't tell other people to use it either.

loup-vaillant 9y ago
Let's list the pros and cons of each libraries (let's assume we trust all 3 libraries): Monocypher vs TweetNaCl: Good: Monocypher uses more modern primitives (Blake2b, Chacha20) Monocypher provides password derivation Monocypher is a bit easier to use Monocypher is much faster Bad: Monocypher is twice as big Conclusion: Monocypher utterly outclasses TweetNaCl. Monocypher vs Libsodium: Good: Monocypher is much smaller Monocypher is easier to deploy Monocypher is a bit easier to use Bad: Monocypher is a bit slower (except Argon2i, which is faster) Conclusion: If performance matters, use NaCl. Otherwise, use Monocypher. Nothing new, you say? > By all means, write your own crypto but DON'T USE IT! And of course don't tell other people to use it either. Get back to me in a couple years, we'll count the CEVs since version 1.0.

vadiml 9y ago

Hi, I've browsed through the code and i don't see why say it is not good in MT setting... an first view the code is completely reentrant... Another thought: i strongly suspect that inlining these store_le... and load_le... function will show pretty significant performance boost

gizmo385 9y ago

Heads up the the author: The Reddit and HN links in the timeline are switched :)

pier25 9y ago

I see crypto as black magic. Can anyone recommend some introductory resource to demystify it a bit? Why is it so hard?

loup-vaillant 9y ago
Here's a good introduction to cryptography: https://www.crypto101.io/ https://www.crypto101.io/

convivialdingo 9y ago

Hey, fantastic work! Really happy to see some amateur crypto development out there. I started as an amateur also, now I work in crypto product dev. Definitely impressed if this is your first crypto work. I really like the simplicity, reminds me of OpenSSH. I gotta agree with some of the naysayers though - this needs more time to brew. Even if it's perfect, there still more proof to show. My suggestions: * A long-running interoperability test between your lib and another. Test vectors don't catch everything. Run it for a week, check results and publish the results. * Fuzzing. You need to to catch false results, random crashes, stack overflows, etc. Fuzz every input, do it for random lengths (both underflow and overflow). * Maybe port it and test on Win, Mac, ARM, MIPS. I find a lot of bugs shake out during ports. * Timing. You can't prevent all side-channel attacks, but bits going out are easy attack vectors. Use a high-precision timing mechanism to catch any unusual spikes. Capture a few hours of points for each functional path, and publish the results. (crypto_aead_unlock has two return paths, it might be better to maintain timing). * Personal preferences: Rather than using int returns, use bool. Zero your ctx vars, e.g. a_ctx ctx = {0,}; I spend more time in testing products than development, honestly. Publish as much relevant data as you can.

loup-vaillant 9y ago
> Definitely impressed if this is your first crypto work. It is. :-) > A long-running interoperability test between your lib and another. Already have one for all primitives. Just tweak the parameters to make it run for a long time. The test code is in test/sodium.c and test/donna.c > Fuzzing. You need to to catch false results, random crashes, stack overflows, etc. Fuzz every input, do it for random lengths (both underflow and overflow). I'm not sure how to go about it. I already check every input size, from zero to several times the size of the internal block, in my comparison tests. Of course, all inputs are correct. Incorrect inputs result in undefined behaviour, checking for that is out of scope. (That's why most functions return void.) > Maybe port it and test on Win, Mac, ARM, MIPS. I find a lot of bugs shake out during ports. Yeah, I only tried GCC and Clang on my Intel Ubuntu machine. But I did use Valgrind, ASan, MSan, UBSan, and the TIS-Interpreter. I also compiled under various standards: C99, C11, C++98, C++11, C++14, C++17. > Timing. […] Use a high-precision timing mechanism to catch any unusual spikes Isn't that a bit overkill? I'd rather manually review the source code for the absence of secret dependant branches or secret dependent array indices. I'm not implementing AES or RSA, the primitive I have chosen were specifically designed to easily avoid timing attacks. > crypto_aead_unlock has two return paths, it might be better to maintain timing. No. There are 2 cases to consider: failure, and success. Each path runs in constant time, so timing can only distinguish success from failure… which is revealed anyway by the return code. > Rather than using int returns, use bool. I'm afraid bool is a tiny bit less portable than int. I also went with the flow about error codes, which are traditionally int. > Zero your ctx vars, e.g. a_ctx ctx = {0,}; I'm not sure how to do it portably, with zero dependency.

gwerks 9y ago

Inb4 "don't roll your own crypto"