6 ms·

> Allo, whatsapp and other closed systems that give you no resaon to trust them other than faith in the people shilling them. Their claims can't be verified so

by lvh 8y ago

> Allo, whatsapp and other closed systems that give you no resaon to trust them other than faith in the people shilling them. Their claims can't be verified so they can only be marked as just that, claims. I have repeatedly refuted that point and you have repeatedly ignored it. > Sure, plenty of obvious flaws can be spotted without source code access, but many are hard to find even if you -do- have source code to the point they would probably have never been found were they not open to allow the right set of eyes to eventually read the right section of code (Heartbleed etc). You are implying that no source code prevents you from finding obvious flaws, and I have given you two counterexamples in a messaging service that came up in this thread that I had never heard of before. That was casual peeking, not even a serious audit. > I found random number generation flaws in Terraform I would of -never- found without source code access. Do you professionally audit software? The fact that you can't find a bug without source does not mean that no-one can. Project Zero finds bugs in Windows and Edge every other day that are a lot more complicated than figuring out what RNG Terraform uses and how Terraform uses it. > Security is -hard- and anyone that thinks they can get it right with a SPOF closed source approach is more interested in marketshare than security. Since you've impugned my motives in two separate places in that post (referring to me as a "shill"), I am no longer interested in discussing this with you. I'm sure people can make up their minds from the thread.

lrvick 8y ago
I changed the word "shill" in an edit right after I posted as that was unfair/unhelpful and realized you might think it was directed at you. It was not. I welcome this type of debate personally. > Do you professionally audit software? I do as a matter of fact. I'll be honest it is normally much easier in closed products as I know what to look for. It is generally much harder to find flaws in popular open source systems as someone else has usally long beaten me to the low hanging fruit in critical codepaths. I have been burned and seen others burned so many times by closed software and teams that ship security regressions that I now personally use only open tools I can audit at some level, or can audit the auditing and reproducible build process. I have seen "security" companies cut corners too many times in order to feature farm to trust anything that won't let me see the source code. I have even professionally audited multiple systems on the spreadsheet myself, some of which I am aware of vulnerabilities for currently under embargo. So far you have cited examples which I not only didn't ignore but responded to in the form of counter examples. This is however turning into a open source vs closed source debate with subjective evidence, which in the end always boils down to if you have blind faith in a small group of people or not. In the spreadsheet I tried to only include things that could be fairly objective, but when we get into concepts of trust of binaries and their authors, it gets muddy to be sure, and we will end up choosing paths based on our own threat profiles and experience. This is why I tried to include -everything- in the spreadsheet I am aware of, so people with threat profiles different than mine can make informed choices.
- tptacek 8y ago
  You professionally audit secure messaging applications? For what kinds of vulnerabilities? I have turned down invitations to audit some of these applications because I didn't feel qualified to render an assessment (I've been doing professional software security assessment, of closed/open source applications, since 1996). Did you take money for those assessments? Which ones did you do? I'd like to take a closer look at some of them sometime.
  - lrvick 8y ago
    I audit tools myself, my employers, or partners consider and if I find obvious flaws then I file bugs for those products and generally we either don't use them or limit their use. I feel qualified only to certify that a tool is obviously insecure, but I don't think -anyone- will ever be qualified to solo certify something as totally secure. (but sadly that is how clean audits are generally read). I have in the past hired 3-4 audit firms and each would find flaws the others did not, and that mine did not, but fail to spot flaws that mine did. No one has the full picture. The only relevant one not under embargo atm was my recent casual 2-3 hour audit of Lifesize which I found right away had a number of alarming issues the company would not address. Some of these may of been found to be non issues under further inspection, but there was more than enough to assert security was not a major focus of the platform. This cursory look was all it took for me to feel confident in ending any consideration of that product to protect the interests of the entity considering it. I reported all these to the company and there was no response for over 90 days and made my findings public after due warning. They are available here: https://gist.github.com/lrvick/6e600d8484cfb415d1e2b06e8b3452c2 https://gist.github.com/lrvick/6e600d8484cfb415d1e2b06e8b345... (reminder: this was only 2-3 hours of work and by all means grain of salt)
    - tptacek 8y ago
      I think you probably understand where I am coming from at this point. When I think about "audits" for secure messengers, I think about professional, specialized, contracted assessments with dedicated teams of experts (I'm a little biased, since that used to be my line of work). And my two objections to this discussion are: 1. I think LVH is right to point out that "audit" doesn't have much meaning in the document you've produced; it gloms together assessments of wildly differing depth and quality and condenses them all to a single pass/fail. 2. Secure messaging security is hard, much harder than secure transports (which you alluded to in the other subthread when you mentioned TLS). There are in fact not that many people in the world who can do a proper cryptographic assessment of a messenger at this point (not because it's prohibitively difficult; it should be well within the reach of everyone with a graduate degree in cryptography who enjoys coding --- rather, just because it's a specialized skill set that not many people have an opportunity to get good at). Like I said: I wouldn't say I'm qualified to perform such an assessment (LVH, different story). And all that "just" gets you the cryptography! If your messenger gets popular, the framework it's built on becomes one of the most important targets on the Internet! So I get itchy when people say things that amount to "I've audited things, I have an authoritative opinion about this stuff". Probably not? Maybe, but, like, if we were going to place a bet...? You can turn that logic around on me. But I'm not the one making an extraordinary claim. You are. So: what really bugs me is the idea that this is a problem domain you can reduce to a Wikipedia-style comparison chart. That kind of chart will get people hurt.

by lvh 8y ago

lrvick 8y ago

I changed the word "shill" in an edit right after I posted as that was unfair/unhelpful and realized you might think it was directed at you. It was not. I welcome this type of debate personally. > Do you professionally audit software? I do as a matter of fact. I'll be honest it is normally much easier in closed products as I know what to look for. It is generally much harder to find flaws in popular open source systems as someone else has usally long beaten me to the low hanging fruit in critical codepaths. I have been burned and seen others burned so many times by closed software and teams that ship security regressions that I now personally use only open tools I can audit at some level, or can audit the auditing and reproducible build process. I have seen "security" companies cut corners too many times in order to feature farm to trust anything that won't let me see the source code. I have even professionally audited multiple systems on the spreadsheet myself, some of which I am aware of vulnerabilities for currently under embargo. So far you have cited examples which I not only didn't ignore but responded to in the form of counter examples. This is however turning into a open source vs closed source debate with subjective evidence, which in the end always boils down to if you have blind faith in a small group of people or not. In the spreadsheet I tried to only include things that could be fairly objective, but when we get into concepts of trust of binaries and their authors, it gets muddy to be sure, and we will end up choosing paths based on our own threat profiles and experience. This is why I tried to include -everything- in the spreadsheet I am aware of, so people with threat profiles different than mine can make informed choices.

tptacek 8y ago
You professionally audit secure messaging applications? For what kinds of vulnerabilities? I have turned down invitations to audit some of these applications because I didn't feel qualified to render an assessment (I've been doing professional software security assessment, of closed/open source applications, since 1996). Did you take money for those assessments? Which ones did you do? I'd like to take a closer look at some of them sometime.
- lrvick 8y ago
  I audit tools myself, my employers, or partners consider and if I find obvious flaws then I file bugs for those products and generally we either don't use them or limit their use. I feel qualified only to certify that a tool is obviously insecure, but I don't think -anyone- will ever be qualified to solo certify something as totally secure. (but sadly that is how clean audits are generally read). I have in the past hired 3-4 audit firms and each would find flaws the others did not, and that mine did not, but fail to spot flaws that mine did. No one has the full picture. The only relevant one not under embargo atm was my recent casual 2-3 hour audit of Lifesize which I found right away had a number of alarming issues the company would not address. Some of these may of been found to be non issues under further inspection, but there was more than enough to assert security was not a major focus of the platform. This cursory look was all it took for me to feel confident in ending any consideration of that product to protect the interests of the entity considering it. I reported all these to the company and there was no response for over 90 days and made my findings public after due warning. They are available here: https://gist.github.com/lrvick/6e600d8484cfb415d1e2b06e8b3452c2 https://gist.github.com/lrvick/6e600d8484cfb415d1e2b06e8b345... (reminder: this was only 2-3 hours of work and by all means grain of salt)
  - tptacek 8y ago
    I think you probably understand where I am coming from at this point. When I think about "audits" for secure messengers, I think about professional, specialized, contracted assessments with dedicated teams of experts (I'm a little biased, since that used to be my line of work). And my two objections to this discussion are: 1. I think LVH is right to point out that "audit" doesn't have much meaning in the document you've produced; it gloms together assessments of wildly differing depth and quality and condenses them all to a single pass/fail. 2. Secure messaging security is hard, much harder than secure transports (which you alluded to in the other subthread when you mentioned TLS). There are in fact not that many people in the world who can do a proper cryptographic assessment of a messenger at this point (not because it's prohibitively difficult; it should be well within the reach of everyone with a graduate degree in cryptography who enjoys coding --- rather, just because it's a specialized skill set that not many people have an opportunity to get good at). Like I said: I wouldn't say I'm qualified to perform such an assessment (LVH, different story). And all that "just" gets you the cryptography! If your messenger gets popular, the framework it's built on becomes one of the most important targets on the Internet! So I get itchy when people say things that amount to "I've audited things, I have an authoritative opinion about this stuff". Probably not? Maybe, but, like, if we were going to place a bet...? You can turn that logic around on me. But I'm not the one making an extraordinary claim. You are. So: what really bugs me is the idea that this is a problem domain you can reduce to a Wikipedia-style comparison chart. That kind of chart will get people hurt.