7 ms·

Responsibly Bringing a new Cryptography Product to Market

74 points by rarrrrrr 13y ago

dougfort 13y ago
This is critical. If you don't think about this stuff, use the existing code.
kenmanheimer 13y ago
Some programming efforts can be casual, some can not. Cryptography is is one that often requires not just rigor, but also expensive investment. The combination can be an obstacle to open source inroads. I'm part of SpiderOak, and proud of their investment in Crypton. It is expensive - like, not just the coding, but sponsoring these audits - but, I believe, extremely valuable.
ekrizzle 13y ago
Glad to see real numbers and specifics about the audit being used in the post.
- peawee 13y ago
  For people to give a crap about FOSS crypto, you need the audits, and there's just not that much information out there. Audits are pricy and a great way to prematurely age a developer team, but the alternative is "looks OK" crypto code like the Debian SSL debacle. Crypton's definitely an effort at all-cards-on-the-table corporate-sponsored FOSS, and this includes the project itself and not just the code.
rarrrrrr 13y ago
By the way, once we've established confidence in the code directly included in the project, we're hoping to also arrange audits for some of the popular crypto libraries involved such as SJCL.
- benzim 13y ago
  Have any of those libraries been audited so far? Seems like a necessary step for widespread js crypto.
  - rarrrrrr 13y ago
    SJCL had substantial peer review during its development, but I'm not aware of a public 3rd party audit from a security firm. Oh yeah, and before people with torches show up, let me clarify that is a Javascript based crypto product but not browser and website based crypto. The deployment target is situations where the code delivery problem is solved: HTML5 mobile apps (including phonegap / cordova), desktop apps with things like AppJS, browser extensions, etc. It's a high level secure-by-default framework for building collaborative realtime and storage applications. Javascript is a natural language for this, because the framework provides storage through an object database, and that approach is natural an convenient in Javascript. We're spending the time and money to make a secure framework now so we don't have to spend quite as much time in reinvention and security review for each new crypto application we build.
    - tptacek 13y ago
      SJCL has been audited. You should ping me via email if you're serious about arranging for OSS crypto to get reviewed, not because I want to sell you audit services, but because someone already beat you to the punch in a big way, and you should talk to them. :)
      - deezthugs 13y ago
        Will do! Thanks! Is there a public report available? How long ago was this?
        tptacek 13y ago
        There's no public report available.
    - 13y ago
ostensibly 13y ago
It's interesting to see what's involved in a security audit and how much it costs. It's nice to see this kind of information being freely given.
tptacek 13y ago
Be careful. It is easy to pay for an audit from a firm that doesn't have a serious cryptography practice. You would reasonably think that any software security firm would be competent in evaluating crypto, or at least crypto basics like whether you're using a sane block cipher mode or failing to authenticate your ciphertext. But it turns out that the opposite is true: the overwhelming majority of firms, including some of the best, have no crypto literacy at all. (The best firms that don't do crypto will tell you this and refer you). We've seen some horror stories. It's hard not to get a little irritated when you're the second firm to assess a target and own it up on day 2 with a trivial CBC padding oracle. All this means is, when you're talking to a potential auditor, ask them hard questions about cryptography. Ask them to describe some of the crypto vulnerabilities they have found on projects. If they talk about "weak keys" or "bad ciphers", they're unserious. Zooko's team at Least Authority is a serious crypto practice. Engaging Zooko was a good call! $1000 per auditor/day is less than you'd pay to get someone to run Nessus on your network from a normal firm. Zooko did you an _enormous_ favor. For crypto work, a rate four times as high wouldn't be out of the ordinary.
- deezthugs 13y ago
  We will be conducting ongoing audits and also would like to conduct an audit (hopefully at least partially crowd-funded)on SJCL by itself. Thanks for this valuable feedback!
- kenmanheimer 13y ago
  Thanks for this - really valuable info! (Ka Ping Yee's report, attributed in Alan's blog post, is more testimony on the difficulty in getting useful audit.)
- deleted 13y ago
  [deleted]
  - deleted 13y ago
    [deleted]
- rarrrrrr 13y ago
  Working with the Least Authority folks has been elaborately enjoyable. Highly recommended. I've started the long journey of reading the code and discussion history of their Tahoe-LAFS project -- action packed with fascinating details.
zooko_LeastAuth 13y ago
I just posted a comment on the SpiderOak blog about how I was rather startled when I found out that they had waited until after we did the security audit before they informed us that they knew about bugs going in: https://spideroak.com/blog/20140220090004-responsibly-bringing-new-cryptography-product-market#comment-27142 https://spideroak.com/blog/20140220090004-responsibly-bringi... However, after I got over my surprise, I started thinking that this was a really good move on SpiderOak's part. If you hire a security auditor, it might be hard for you to tell whether you're getting value for your money. Leaving known bugs in the code and then observing whether the auditors find them is potentially a good way to overcome that. Mind you: this will make life harder for we in the security auditing industry if this practice takes off. ☺
- deezthugs 13y ago
  :)
daira 13y ago
I was one of the auditors for Least Authority. Just wanted to add that this audit was really fun to do! If you have the necessary experience and are part of a good team, security auditing is a fantastic job. -- Daira Hopwood
dmunoz 13y ago
Interesting read. I remember Crypton being announced almost a year ago [0], and for awhile I paid attention the GitHub repo, but it really felt like a dead product. Looking back at the commit history, I'm not so sure why I had this impression, as it seems pretty lively. I'm thinking that perhaps a lot of those commits were not pushed publicly until recently, but in either case it's good to see it active again. On a bit of a tangent, regarding this post and the requirements (1-5) they posted, I really wish they would hurry up and finally bring this to their backup clients. It's for far too long been a joke in my mind that they promise "100% Private", "Zero-knowledge", "Secure" online backup, sync and sharing with a closed-source client. I went as far as to sign up with them and download the client, only to never install it because I realized I didn't believe a word they said. They promised (again) at the start of this year to open source their client [1]. I have actually followed the issue closely, and just last year one of the founders was admitting on reddit that it simply wasn't a priority. I don't see why not, as it makes their service complete snake oil, and I can't be the only one who refuses to use their service because of it. Anyway, my point is... perhaps from their perspective this was because of point #4 in the post for this submission (Have clear, well documented code). [0] https://news.ycombinator.com/item?id=5283072 https://news.ycombinator.com/item?id=5283072 [1] https://spideroak.com/blog/20140110144419-spideroak-to-become-oss-what-to-expect-in-2014 https://spideroak.com/blog/20140110144419-spideroak-to-becom... Edit: Here [3] is where they prioritize "growth and profit" over providing what they promise. The granularity on the visible submission date is not very informative, but if you look at the posts JSON, you can see the Unix time in the field created_utc. It was the fist of April, so a little less than a year ago. [3] http://www.reddit.com/r/IAmA/comments/rmp9l/we_are_spideroak_and_we_are_handing_out_5gb_free/c4725dt http://www.reddit.com/r/IAmA/comments/rmp9l/we_are_spideroak...
- bytex64 13y ago
  > I can't be the only one who refuses to use their service because of it Certainly not, but the vast majority of people seem to be willing to trust them and pay for service, so I can understand why it's not a priority. I'm not trying to say you're wrong -- I'd like to see source for SpiderOak, too -- I'm just saying that I understand why it hasn't happened.
- 13y ago
ekrizzle 13y ago
Least Authority's blog post about the audit: https://leastauthority.com/blog/least_authority_performs_security_audit_for_spideroak.html https://leastauthority.com/blog/least_authority_performs_sec...
0xdeadbeefbabe 13y ago
That's great you found good auditors, and that will probably work out just fine. Another idea is that real attackers are your only hope (https://news.ycombinator.com/item?id=7238750 https://news.ycombinator.com/item?id=7238750) maybe. Auditors are supposed to simulate real attackers, and maybe these serious auditors are identical to attackers except you don't pay them as much as they could get some other way like publishing your exploits on the black market. Speaking of market, Schneier said you really need a mass market to test things (from the above link).

74 points by rarrrrrr 13y ago

This is critical. If you don't think about this stuff, use the existing code.

Some programming efforts can be casual, some can not. Cryptography is is one that often requires not just rigor, but also expensive investment. The combination can be an obstacle to open source inroads. I'm part of SpiderOak, and proud of their investment in Crypton. It is expensive - like, not just the coding, but sponsoring these audits - but, I believe, extremely valuable.

ekrizzle 13y ago

Glad to see real numbers and specifics about the audit being used in the post.

peawee 13y ago
For people to give a crap about FOSS crypto, you need the audits, and there's just not that much information out there. Audits are pricy and a great way to prematurely age a developer team, but the alternative is "looks OK" crypto code like the Debian SSL debacle. Crypton's definitely an effort at all-cards-on-the-table corporate-sponsored FOSS, and this includes the project itself and not just the code.

rarrrrrr 13y ago

By the way, once we've established confidence in the code directly included in the project, we're hoping to also arrange audits for some of the popular crypto libraries involved such as SJCL.

benzim 13y ago
Have any of those libraries been audited so far? Seems like a necessary step for widespread js crypto.
- rarrrrrr 13y ago
  SJCL had substantial peer review during its development, but I'm not aware of a public 3rd party audit from a security firm. Oh yeah, and before people with torches show up, let me clarify that is a Javascript based crypto product but not browser and website based crypto. The deployment target is situations where the code delivery problem is solved: HTML5 mobile apps (including phonegap / cordova), desktop apps with things like AppJS, browser extensions, etc. It's a high level secure-by-default framework for building collaborative realtime and storage applications. Javascript is a natural language for this, because the framework provides storage through an object database, and that approach is natural an convenient in Javascript. We're spending the time and money to make a secure framework now so we don't have to spend quite as much time in reinvention and security review for each new crypto application we build.
  - tptacek 13y ago
    SJCL has been audited. You should ping me via email if you're serious about arranging for OSS crypto to get reviewed, not because I want to sell you audit services, but because someone already beat you to the punch in a big way, and you should talk to them. :)
    - deezthugs 13y ago
      Will do! Thanks! Is there a public report available? How long ago was this?
      - tptacek 13y ago
        There's no public report available.
  - 13y ago

ostensibly 13y ago

It's interesting to see what's involved in a security audit and how much it costs. It's nice to see this kind of information being freely given.

tptacek 13y ago

Be careful. It is easy to pay for an audit from a firm that doesn't have a serious cryptography practice. You would reasonably think that any software security firm would be competent in evaluating crypto, or at least crypto basics like whether you're using a sane block cipher mode or failing to authenticate your ciphertext. But it turns out that the opposite is true: the overwhelming majority of firms, including some of the best, have no crypto literacy at all. (The best firms that don't do crypto will tell you this and refer you). We've seen some horror stories. It's hard not to get a little irritated when you're the second firm to assess a target and own it up on day 2 with a trivial CBC padding oracle. All this means is, when you're talking to a potential auditor, ask them hard questions about cryptography. Ask them to describe some of the crypto vulnerabilities they have found on projects. If they talk about "weak keys" or "bad ciphers", they're unserious. Zooko's team at Least Authority is a serious crypto practice. Engaging Zooko was a good call! $1000 per auditor/day is less than you'd pay to get someone to run Nessus on your network from a normal firm. Zooko did you an _enormous_ favor. For crypto work, a rate four times as high wouldn't be out of the ordinary.

deezthugs 13y ago
We will be conducting ongoing audits and also would like to conduct an audit (hopefully at least partially crowd-funded)on SJCL by itself. Thanks for this valuable feedback!
kenmanheimer 13y ago
Thanks for this - really valuable info! (Ka Ping Yee's report, attributed in Alan's blog post, is more testimony on the difficulty in getting useful audit.)
deleted 13y ago
[deleted]
- deleted 13y ago
  [deleted]
rarrrrrr 13y ago
Working with the Least Authority folks has been elaborately enjoyable. Highly recommended. I've started the long journey of reading the code and discussion history of their Tahoe-LAFS project -- action packed with fascinating details.

zooko_LeastAuth 13y ago

I just posted a comment on the SpiderOak blog about how I was rather startled when I found out that they had waited until after we did the security audit before they informed us that they knew about bugs going in: https://spideroak.com/blog/20140220090004-responsibly-bringing-new-cryptography-product-market#comment-27142 https://spideroak.com/blog/20140220090004-responsibly-bringi... However, after I got over my surprise, I started thinking that this was a really good move on SpiderOak's part. If you hire a security auditor, it might be hard for you to tell whether you're getting value for your money. Leaving known bugs in the code and then observing whether the auditors find them is potentially a good way to overcome that. Mind you: this will make life harder for we in the security auditing industry if this practice takes off. ☺

deezthugs 13y ago
:)

daira 13y ago

I was one of the auditors for Least Authority. Just wanted to add that this audit was really fun to do! If you have the necessary experience and are part of a good team, security auditing is a fantastic job. -- Daira Hopwood

dmunoz 13y ago

Interesting read. I remember Crypton being announced almost a year ago [0], and for awhile I paid attention the GitHub repo, but it really felt like a dead product. Looking back at the commit history, I'm not so sure why I had this impression, as it seems pretty lively. I'm thinking that perhaps a lot of those commits were not pushed publicly until recently, but in either case it's good to see it active again. On a bit of a tangent, regarding this post and the requirements (1-5) they posted, I really wish they would hurry up and finally bring this to their backup clients. It's for far too long been a joke in my mind that they promise "100% Private", "Zero-knowledge", "Secure" online backup, sync and sharing with a closed-source client. I went as far as to sign up with them and download the client, only to never install it because I realized I didn't believe a word they said. They promised (again) at the start of this year to open source their client [1]. I have actually followed the issue closely, and just last year one of the founders was admitting on reddit that it simply wasn't a priority. I don't see why not, as it makes their service complete snake oil, and I can't be the only one who refuses to use their service because of it. Anyway, my point is... perhaps from their perspective this was because of point #4 in the post for this submission (Have clear, well documented code). [0] https://news.ycombinator.com/item?id=5283072 https://news.ycombinator.com/item?id=5283072 [1] https://spideroak.com/blog/20140110144419-spideroak-to-become-oss-what-to-expect-in-2014 https://spideroak.com/blog/20140110144419-spideroak-to-becom... Edit: Here [3] is where they prioritize "growth and profit" over providing what they promise. The granularity on the visible submission date is not very informative, but if you look at the posts JSON, you can see the Unix time in the field created_utc. It was the fist of April, so a little less than a year ago. [3] http://www.reddit.com/r/IAmA/comments/rmp9l/we_are_spideroak_and_we_are_handing_out_5gb_free/c4725dt http://www.reddit.com/r/IAmA/comments/rmp9l/we_are_spideroak...

bytex64 13y ago
> I can't be the only one who refuses to use their service because of it Certainly not, but the vast majority of people seem to be willing to trust them and pay for service, so I can understand why it's not a priority. I'm not trying to say you're wrong -- I'd like to see source for SpiderOak, too -- I'm just saying that I understand why it hasn't happened.
13y ago

ekrizzle 13y ago

Least Authority's blog post about the audit: https://leastauthority.com/blog/least_authority_performs_security_audit_for_spideroak.html https://leastauthority.com/blog/least_authority_performs_sec...

0xdeadbeefbabe 13y ago

That's great you found good auditors, and that will probably work out just fine. Another idea is that real attackers are your only hope (https://news.ycombinator.com/item?id=7238750 https://news.ycombinator.com/item?id=7238750) maybe. Auditors are supposed to simulate real attackers, and maybe these serious auditors are identical to attackers except you don't pay them as much as they could get some other way like publishing your exploits on the black market. Speaking of market, Schneier said you really need a mass market to test things (from the above link).