20 ms·
CryptoCat iOS Application Penetration Test [pdf]
- deleted 13y ago[deleted]
- imkevinxu 13y agoDidn't realize how vulnerable even a simple NSLog was... I wonder how many websites have sensitive information they console.log but forgot to take out for production
- pokstad 13y agoRemember when the iPhone was under fire for logging everywhere a user went? That was due to careless devs logging location data. Also, logging data is a blocking operation, so if done too often it slows the performance of an app. CORRECTION: the 3rd party apps were part of it, along with Apple's own logging, when reported in the news.
- rancor 13y agoWhen I saw one of the main CryptoCat developers present in 2012, I came away with the impression that nobody on the core team understood crypto, security, or software engineering. This audit is another rock on the mountain of evidence I've seen supporting this impression in the following years. A really nice job by iSec, though.
- sentenza 13y agoIt is also important to always remember that a crypto app isn't like other apps. If the protesters in Turkey rely on shoddy crypto today, it might cost them their lives a few months from now. I sincerely hope it doesn't come to that but it is a realistic example. Always be sure of what you are doing, rely on external review and never, I repeat, _never_ overstate the security of your crypto system.
- rancor 13y agoThis point cannot be emphasized enough. In the extreme case, which Cryptocat marketing materials have employed, secure communications software is life safety critical on a level similar to medical or aviation software. As such, the admit-your-mistakes-and-fix-them-later model of development isn't agile, open, or any of those buzzwords. It's a way to get people killed.
- kaeporan 13y agoCryptocat has always provided ample warnings that no software can ever be trusted with your life. These warnings appear every time you launch Cryptocat, on the website and in various guides and blog posts.
- rancor 13y agoAnd I agree, that's a bare minimum warning for all such software. I appreciate your efforts to make strong crypto more accessible to the general public. That being said, you and I both know that people are using Cryptocat in dangerous situations. And having worked on both medical imaging and secure messaging systems, I have a healthy respect for the consequences of implementation failure. As such, I feel that your disregard for these consequences in broadly releasing such broken software would displease any professional review board, and I frankly doubt you'd ever attain such a license given such a history of poor professional judgment. In short, I take my profession damn seriously, and jokers like you are why nobody trusts software.
- sillysaurus3 13y agoPerhaps we shouldn't call people names?
- diminoten 13y agoYou lampoon yourself with this extremist attitude.
- 13y ago
- computer 13y agoI wasn't going to post this, but I had the same feeling at a conference. Also, if I remember correctly, the first version that got audited turned out to be extremely insecure as well, with their own custom crypto protocols? I haven't recommended CryptoCat to anyone since, and still wouldn't. This audit report is another fascinating read to see all the mistakes I would probably have made as well. Secure crypto is so incredibly difficult to get right...
- kaeporan 13y agoIt's important to note that this audit concerned a pre-release, debugging version of Cryptocat for iPhone. The audit document alone doesn't give enough context; I strongly urge reading our blog post: https://blog.crypto.cat/2014/04/recent-audits-and-coming-improvements/ https://blog.crypto.cat/2014/04/recent-audits-and-coming-imp...
- tptacek 13y agoAre you saying that it's good news because it means that no actual users were exposed to the flaws? To the extent that those flaws apply only to the iOS version, I agree: that's good news. Are you saying that it's good news because they tested something that you weren't ever going to release in that state? That's a tougher row to hoe, unless you're going to claim that your team inevitably would have found the same set of vulnerabilities that Scott, David, Alban, and Zooko's team found. For a typical application --- yours isn't typical for any number of reasons --- prerelease or not, the state the application is in when a pentest team gets it is, from the perspective of security, the application customers would have received.
- zooko_LeastAuth 13y agoSpeaking of the vulnerabilities that our team found, here is our blog post about it and a link to our report and the github issue tickets that we opened: Here is our blog post about our audit of Cryptocat, which was also announced today: https://leastauthority.com/blog/ https://leastauthority.com/blog/
- natdempk 13y agoAlso of interest is their blog post about how they plan to handle the issues described in this report: https://blog.crypto.cat/2014/04/recent-audits-and-coming-improvements/ https://blog.crypto.cat/2014/04/recent-audits-and-coming-imp... Reading that, I still am not sure why anyone would use CryptoCat especially with things like TextSecure on the market that seem to take crypto far more seriously. The only reason I can see for that is that they have clients on more platforms, but if this is similar to the state of all of them, then what's the point?
- kaeporan 13y agoThanks for linking to the blog post. This audit concerns a pre-release version of Cryptocat for iPhone. Many of the bugs were due to debugging code and were fixed before release.
- tptacek 13y agoWhich of the bugs were due to debugging code?
- daira 13y agoFindings iSEC-RFACC0114-1 and iSEC-RFACC0114-3. (2 out of the 17 vulnerabilities found by iSec, of varying severity.)
- sneak 13y agoWhy use donated money to pay for an audit of software that has known bugs and isn't ready yet? That's wasteful. The point of an audit is to find bugs you don't already know about.
- ibmthrowaway218 13y agoIt can be a useful technique for testing the quality of the audit. At a previous company we had to have words with a company that performed a software audit as they failed to find two issues we'd planted to test them. (Of course, they did find several things we didn't know about.)
- venomsnake 13y agoThere is a "many ways to skin a cat" joke here somewhere. It is actually terrible - the hmac timing attack requires around 3 minutes of google searching to avoid and is basic public domain knowledge. The other are much worse.
- sliverstorm 13y agoI wish I knew how to find my way into security as a hobby. Such a fun topic.
- oliwary 13y agoI am in the same position. Stanford Online started a Coursera course on cryptography yesterday, might be interesting for you. https://www.coursera.org/course/crypto https://www.coursera.org/course/crypto
- scott_s 13y agoLong-time HN member tptacek's company has a challenge set that many praise highly: http://www.matasano.com/articles/crypto-challenges/ http://www.matasano.com/articles/crypto-challenges/
- schoen 13y agoAre the Matasano crypto challenges currently stuck in some way, like with a grading backlog? I signed up some months ago, sent my first set of answers just after the new year, and have never heard back about the second challenge set.
- tptacek 13y agoWe are way. way. way. backlogged. If anyone has any idea on how to help a hapless team of security researchers manage many thousands of people looking to get through the crypto challenges, we'd be t-h-r-i-l-l-e-d. We were able to keep up last summer, but then Microcorruption happened, we got into a hole, and we're only slowly digging ourselves out of it. Alex, Sean, and I are turning the challenges into a book, which we're going to release "choose-your-price" with all funds directly going to a charity (I like Watsi, but who knows); that book will also include Set 8, which is all ECC. I'm hoping we'll be done with that by August.
- deleted 13y ago[deleted]
- primitivesuave 13y agoThis is most alarming. CryptoCat's OTR implementation on all platforms allows a chat peer to change their OTR key during a chat session without user notification. An attacker performing a man-in-the-middle attack against the client's XMPP or HTTPS stream can inject their own OTR key in the discussion after a user has authenticated their peer's OTR fingerprint. This permits the attacker to decrypt all messages that follow, and no user would have reason to suspect the compromise.
- kaeporan 13y agoFixes and improvements to this, and more, are covered in our blog post. I strongly urge you to read it. This audit alone doesn't give enough context. https://blog.crypto.cat/2014/04/recent-audits-and-coming-improvements/ https://blog.crypto.cat/2014/04/recent-audits-and-coming-imp...
- anologwintermut 13y agoWhat's your fix for the man-in-the-middle attack on all platforms (including deployed ones) the audit identifies and your blog acknowledges? In your blog post there seems to be little context that can excuse such a mistake and nothing that explains how you fix it? Am I correct in reading your blog post that right now there isn't a fix? I.e. it's an open attack assuming someone compromises a CA or a cryptocat server? Isn't this a rather big issue since the only point of cryptocat is to protect against that kind of an attack. If you just wanted security only against eavesdroppers(i.e. you trusted the chat server), xmpp over TLS would work fine.
- kaeporan 13y agoHi there, The fix for the MITM bug is to offer proper notice via the user interface when a user re-keys with a different public key. There's a demonstration of the user interface element in the blog post.
- sneak 13y ago> In your blog post there seems to be little context that can excuse such a mistake This is a recurring theme with cryptocat. Stay away for 5-10 years until they get their act together.
- lincolnq 13y agoThis is awesome. I'm sad that CryptoCat is getting slammed for this for being one of the brave few to post this online. I am sure there are an infinite number of "security-critical" apps which would fail an audit like this, but who never even thought to GET an audit -- much less post it online. The software development community is much stronger for being able to see professional stuff like this posted. Does anyone know how much these audits typically cost, if you're not being subsidized?
- rdl 13y agoGenerally 10-50k is a good starting point for this level. Most firms are full up on work most of the time, but will often try to get interesting new companies or projects even if they're less profitable since 1) they can grow into better stuff 2) good for reputation and for retention of their own employees. Compliance-only is usually cheaper; you can buy rubber stamps for <$10k.
- m0nastic 13y agoIt's important to distinguish that the places offering the <10k rubber stamps aren't really offering the same service (even if you concede that it's only for compliance). Places like that are just running a scanner against your website. In the case of an app like this (which was an iOS app), you might find a cheap place to run it through a source code analyzer (either through a cloud-hosted service like Veracode, or by running an app like AppScan). Assuming you wanted to hire a "respectable" firm to actually perform a real application assessment, I'd say it's closer to 30-50k. If you look at the report, it was scoped at 3 man-weeks of testing (which in this case looks like it was 3 engineers for 1 week). Even if you don't include any additional overhead (like hours for the report generation, or project management hours), you're looking at ~24k just for the engineers effort (if they just priced it t&m, which hopefully they don't). To be fair, this is a pretty exotic application though, compared to what a lot of other people might be working on. The scope for a project to test a more "normal" app would be less. Maybe even half that.
- rdl 13y ago
- kaeporan 13y agoHi, I'm the lead developer for Cryptocat. I strongly urge you all to please read our blog post regarding this audit: https://blog.crypto.cat/2014/04/recent-audits-and-coming-improvements/ https://blog.crypto.cat/2014/04/recent-audits-and-coming-imp... This audit document alone does not give enough context. This audit was commissioned by us and concerns a pre-release version of Cryptocat for iPhone. Many of the bugs it found are due to the fact that it was reviewing a prototype with debugging features (such as NSLog) turned on. While this audit definitely does find some vulnerabilities and room for improvement, none of the critical bugs in this audit ever made it to Cryptocat for iPhone's release. It's very unfortunate that this audit is being taken out of context like this and used to attack our effort. I'd appreciate it if you could please upvote this comment and help me contextualize this audit. Again, please, read the blog post for context (and also for the results of another audit we comissioned in parallel.) We've done our best to address these issues and are working towards an open discussion on how to improve accessible encryption. https://blog.crypto.cat/2014/04/recent-audits-and-coming-improvements/ https://blog.crypto.cat/2014/04/recent-audits-and-coming-imp... The blog post's last section ("On the Significance of Audits") discusses why it is that Cryptocat has seen more audits published about it than other encryption projects. Please, dare to discern. Read what we're doing to improve the security of accessible encryption and our reasoning for publishing these audits. I'll be grateful for you taking the time to read on what we're doing and I am more than happy to discuss with you and answer your questions.
- deleted 13y ago[deleted]
- kaeporan 13y agoWe commissioned this audit in late December and iSec began working on it in January. They audited a pre-release prototype that we provided. This is noted in the audit document, but it's hard to spot unfortunately. The reason we commissioned this audit is to make sure our prototype was audited before release on the App Store. We're very happy to have benefited from this audit, but linking to this PDF alone de-contextualizes the effort and makes it seem like it's an audit of the production version of Cryptocat for iPhone, whereas the version we provided was an early prototype. The audit did find some issues with the (already-released) desktop version and server configuration, and those were also fixed and documented in our blog post. I sincerely appreciate you taking the time to read our blog post on the matter and thank you for your understanding.
- lawnchair_larry 13y agoHuh, this was apparently submitted by Alex Stamos, a co-founder of iSec partners (who did this audit). And he editorialized the title, "Brutal Professional Audit of CryptoCat Published." Your former company did an audit for a customer, then you posted it to HN calling it "Brutal"? Really?
- deleted 13y ago[deleted]
- marshray 13y agoFormer employee adds adjective to HN submission title, film at 11.
- fakedavidthiel 13y agoI was certainly not comfortable with that either; however, Alex is his own man now, free to use whatever inflammatory adjectives he chooses. Not much we can do about it, but also not unusual for him to follow the news on a company he put so many years into. I also do wish that the original post had pointed at CryptoCat's blog post - aside from CryptoCat's context about problems and resolutions, people should also be aware of the separate report by Zooko's team.
- Wintamute 13y ago@secalex Dumb post man. Way to de-contextualise something, cause a drama and damage reputations needlessly.
- ris 13y agoReputations? What reputations, exactly? The only reputation CryptoCat developers have is for writing incredibly poorly thought out software full of holes. Combined with their taste for publicity I would rather call them a public menace.
- zooko_LeastAuth 13y agoHere is our blog post about our audit of Cryptocat, which was also announced today: https://leastauthority.com/blog/ https://leastauthority.com/blog/
- jnbiche 13y agoWow, remind me to never have an audit done by iSec. "Extremely thorough" would have been tough but appropriate, but "brutal" seems just gratuitously provocative. Was that what you were going for? Edit: OK, apologies to iSec for my mistake. I thought he was still affiliated with them. In any case, it looks like a top-notch report, so it would have been a shame to detract from that accomplishment.
- tptacek 13y agoAlex (the submitter) doesn't work for iSEC; he's the CISO of Yahoo now. He left iSEC last year to start Artemis. I agree that the title is bad, and (belatedly) asked 'dang to revert it.
- oafitupa 13y agoThe good thing about CryptoCat is that everyone wants to trash it, so it's becoming better.
- marshray 13y agoThe wisest words I've ever seen office administrative staff post over the FAX machine are: Everyone's life has a purpose. Consider the possibility that yours is to serve as a warning to others.
- fakedavidthiel 13y agoPrecisely. I'd also like to reiterate that CryptoCat had no special obligation to release the report, and doing so was remarkably transparent; Nadim was also super open and responsive throughout the whole audit process. Regardless of anyone's opinion of the findings, the issues are out in the open, discussable and resolvable, which can be nothing but beneficial. Kudos to Nadim and OTF for releasing the info and starting these discussions, as inevitably arduous as they are.
- secfirstmd 13y agoI for one think that Nadim and the Open Tech Fund are to be applauded for opening up their review to the public. I have to wonder how many other commercial and non-profit organisations would ever consider doing this? (Especially those which are in the field of communications and are relied on by people for their lives). Many people on HN seem to be reading the review without actually looking at Nadim's response on the Cryptocat blog - which I urge everyone to read first before commenting. https://blog.crypto.cat/2014/04/recent-audits-and-coming-improvements/ https://blog.crypto.cat/2014/04/recent-audits-and-coming-imp... As far as I understand the username for Nadim (Kaeporan) was also blocked from HN last night so probably he isn't able to continue responding to the comments up here.