4 ms·

I actually think it is. Journalists covering sensitive topics in sensitive areas -must- care about these questions. If you are using something anonymous, full

• tptacek 8y ago
  None of this makes any sense. This reads like an argument from a parallel universe where the only sane option for end-users is Linux on the Desktop. In fact: a huge portion of commercial vulnerability research and assessment work is done on binaries of closed-source products, and it has never been easier to do that kind of work (we're coming up on a decade into the era of IR lifters). Meanwhile, the types of vulnerabilities we're discussing here --- cryptographic exfiltration --- are difficult to identify in source, and source inspection has a terrible track record of finding them. There's no expertise evident in the strident arguments you've made all over this thread (network security work is not the same as shrink-wrap software security work, which is in turn not the same as cryptographic security work --- these are all well-defined specialties) and it concerns me that people might be taking these arguments seriously. No. This isn't how things actually work; it's just how advocates on message boards want them to.
  • lrvick 8y ago
    You can of course do network analysis and say "this looks TLS encrypted with X algo" then dive into those packets and then say "this looks end to end encrypted with X algo". You can also look into a binary application and study jumps, where the private key gets loaded into memory etc etc. I am with you on this, there is tons of low hanging fruit you can do on a binary application, and I do a lot of this sort of thing myself. Making sure an application at a high level seems to do what it says on the tin is a great phase one audit of something as a baseline, and should be done even on open source projects. Trouble with closed source tools is you either have to do this over and over again on every new binary released, -or- you can look at the diffs from one release to next and see if any of those critical codepaths have changed. You also -much- more easily evaluate bad entropy in key generation if you have the source code. People think they are clever all the time using silly approaches along the lines of random.seed(time.now()+"somestring") and it is much less time consuming to spot those types of flaws if you have the code. This is why I argue you need both whitebox and blackbox testing when it comes to mission critical security tools, and a clear public auditable log of changes made to a codebase so review can continually happen, instead of only at specific checkpoints. Again, some people may not care about that level of rigor and just want to share memes with their friends instead of journalists communicating in the feild. I tried to be pretty comprehensive with the list, and welcome recommendations for new objective criteria to include. It is currently roughly sorted by security. Usability is subjective and people can work down the list to the most secure tool they feel they can comfortably use.
    • tptacek 8y ago
      No, I don't think you're following me. You don't tcpdump and look at the packets (that would be silly; even horribly encrypted messages will look fine in a pcap). You don't even "study jumps, where private keys get loaded into memory". You take the binary and lift it to an IR and audit that. People who do a lot of binary reversing vuln research were doing a version of this long before the we had modern tooling; you don't look at each load and store, but rather the control flow graph. People were auditing C programs, by manual inspection, in IDA Pro before there even was a market for secure messaging applications. And that's just manual static analysis! Professional software assessors do a lot more than that; for instance, Trail has been talking about and even open-sourcing symbolic and concolic evaluation tools for years, and, obviously, people have been iterating and refining fuzzers and fault injectors for at least a decade before that. This isn't "low hanging fruit" analysis where we find out, like, the messenger application is logging secret keys to a globally accessible log file, or storing task-switcher screenshots with message content. This is verification of program logic --- and, I'd argue even more importantly, of cryptographic protocol security. "Bad entropy in key generation" is a funny example. Straightforward binary analysis will tell you what the "entropy source" is. If it's not the platform secure random number generator, you already have a vulnerability. The problem with your chart is that there is no real rigor to it. It's just a list of bullets. It's obviously catnip to nerds. We love things like this: break a bunch of things into a grid, with columns for all sorts of make-me-look-smart check-marks! But, for instance, there's no way to tell in this chart whether a messenger "audit" was done by JP Aumasson for Kudelski, or some random Python programmer who tcpdumps the application and looks for local-XSS. You can't even neatly break these audits down by "did they do crypto or not". The Electron messengers got bit a few months ago because they'd been audited by non-Electron specialists --- that's right, your app needed to be audited by someone who actually understood Electron internals, or it almost might as well not have been audited. Given that: what's the point of this spreadsheet? It makes superficial details look important and super-important details look like they don't exist. You didn't even capture forward secrecy. People who pay a lot of attention to cryptographic software security took a hard bite on this problem 5 years ago, with the EFF Secure Messaging Scorecard. EFF almost did a second version of it (I saw a draft). They gave up, because their experts convinced them it was a bad idea. I see LVH here trying to tell you the same thing, in detail, and I see you shrugging him off and at one point even accusing him of commenting in bad faith. Why should people trust your analysis?
• pvg 8y ago
  My point is not that this isn't of interest or important (especially to people who routinely handle sensitive information) but that your methodology is poor and it's poor in ways that have been reasonably comprehensively examined. As I said, it's still a worthwhile exercise and there's no law of physics that says you have to agree with existing consensus-y thought but your effort would be more serious if you're familiar with the arguments. You, on the other hand, just reinvented a threat model that has its very own funny paper.
• throwawaymath 8y ago
  > Not everyone cares about this sort of thing though, and there are 70+ other options listed with various tradeoffs. This is precisely pvg's point. The problem with your methodology is a systemic one that emerges in every crowdsourced threat modeling exercise. You've enumerated every possible security attribute and security feature of every software in a specific category, then tossed them all into a matrix of boolean values. But that does not result in a threat model users can competently assess, for several reasons: 1. You're treating all features as equal - if not in intention, then at least in presentation. Even if you don't intend it, the sea of green at the top is a loud proclamation of safety; likewise the sea of red at the bottom is a siren of insecurity. 2. You're not allowing any nuance in assessment feature or attributes. Boolean flags cannot capture all the nuance inherent in cryptographic security. Which specific party was responsible for an assessment? What are their credentials? What did they find? 3. You're including features which most users don't and shouldn't care about just because some minority might. Moreover you're not being opinionated enough, which is something that comes with expertise - for many of the "features" you listed, the minority that cares probably shouldn't if they only care because of a vague notion of security. 4. You're leaving out important features which should absolutely be considered for security. Where is forward secrecy? Where is authenticated encryption? Where is consideration of specific algorithms or primitives? Where is nonce misuse resistance? 5. Most importantly: you do not have any explicitly called out methodology that allows someone to audit what you've done. If you begin by pre-supposing that a given feature is worthy of inclusion because it's an important security metric, your conclusion is just going to end up magnifying that bias. Therefore it's paramount that you call out methodologies explicitly and early. We see this time and time again in security. People try to first principles the security of an entire category of software by being exhaustive about every type of threat and security feature they can think of. But they invariably leave out important threats/features, underestimate the importance of some and overestimate the importance of others. Exercises which attempt to give users the world almost always end up "empowering" them to boil the ocean. People looking at this spreadsheet are approximately all unqualified to make an informed decision based on a critical assessment of all those features, which means they're likely to just go with the most green option (or worse, proselytize the most green option as the most secure one to their friends and coworkers).