7 ms·

Declaring it secure after an audit is like writing 100% coverage tests and saying it's bug-free. You can't prove absence, only presence. This title is the defi

Declaring it secure after an audit is like writing 100% coverage tests and saying it's bug-free. You can't prove absence, only presence. This title is the definition of sensationalism and only by reading the article do you find the truth: "Their tests uncovered no major issues or security vulnerabilities". This is a bad look for them and I'm wary of their company now...

capableweb 5y ago
I agree with you that the title is a bit sensationalist. But if independent security audits with no major issues uncovered cannot make you claim something is secure, when can you claim something as secure? Or are you of the opinion that nothing ever can be claimed to be secure as there can always be holes that could be uncovered in the future? Using openssh as an example, would you say it's secure when you're using public keys for the authentication? Their track record seems pretty good for the last years, but there might still be uncovered vulnerabilities, could it still claim to be secure?
- sodality2 5y ago
  > But if independent security audits with no major issues uncovered cannot make you claim something is secure, when can you claim something as secure? Nothing at all; it's a broken model. The server can at any time start serving malicious payloads [0]. The server hosts your mail but they also serve the webapp. The clientside decrypts the mail, but the server hosts the client code... It's a fundamentally flawed idea, trying to retrofit encryption into email in this way, when the server essentially has to hold all of your mail. In this case, the only thing that would make me feel secure in using it, is a third-party OSS client that downloads the mail without using the webapp, using only client-side code. And even then, of course, the mail can simply just be not encrypted when being ingested by Proton. So even then I wouldn't really trust it without external encryption like PGP. In which case, why bother? To be clear I do use private email services (protonmail, tutanota) but I am simply not going to fall for the illusion of guaranteed privacy; I just trust that they are what they say they are. They are still a better option IMO than something like Gmail, but I don't think they're a silver bullet. [0]: If you think this is unlikely, see this: https://news.ycombinator.com/item?id=25337507 https://news.ycombinator.com/item?id=25337507
  - upofadown 5y ago
    It appears the audit was applied to the Android and iOS apps. So no comment is being made here about the security of the webapp.
    - sodality2 5y ago
      I thought that was just a wrapper around the website, to be honest. But regardless, the app still loads JS from the website and executes it, so it's still a backdoor possibility.
  - prophesi 5y ago
    This is why I'm excited for 9elements and Mullvad's System Transparency[0][1] project. The goal is to let the end users know that your server is indeed running the code it says it's running. Of course, open firmware and hardware also plays a role in this as a server running binary blobs still has the potential for malicious code to be unknowingly run. [0] https://mullvad.net/en/blog/2019/6/3/system-transparency-future/ https://mullvad.net/en/blog/2019/6/3/system-transparency-fut... [1] https://www.system-transparency.org/ https://www.system-transparency.org/
  - uselpa 5y ago
    Secure email is snake oil; no amount of cruft can make it both reasonable secure and useful (as in federated). Other protocols better fill that space because they were designed for security needs.
    - sodality2 5y ago
      End to end secure email definitely is, I agree. No matter what you have to trust the server. (Maybe a Tor-based email system would be better, where each user is their own server? reminds me of `cables`). If you do trust the server then it can be acceptably secure.
    - gruez 5y ago
      I agree that I wouldn't use Protonmail if Signal was an option, but there are many situations where Signal isn't an option (eg. transactional email, people who don't use signal). In those contexts it's still better to use encrypted mail rather than ad-supported mail (eg. gmail, outlook), or even commercial mail (eg. fastmail), which are both unencrypted.
- yakak 5y ago
  There is a framework for this: https://en.wikipedia.org/wiki/Evaluation_Assurance_Level https://en.wikipedia.org/wiki/Evaluation_Assurance_Level Proton is claiming something similar to EAL4, which is not secure, there is an assurance that not all trained reviewers can find a vulnerability. Openssh is a little less secure than that formally, but has more trained reviewers informally, which probably cover some parts extremely well and other parts sparsely.
- jsjohnst 5y ago
  > when can you claim something as secure? here’s a maybe wild take, uh, never?
  - capableweb 5y ago
    But you'd agree with me that some things are more secure than others right?
    - jsjohnst 5y ago
      I’d be willing to say “there are things with known exploits, there are things which employ techniques that are known not to be safe making them less secure, and the rest is more akin to Schrödinger's cat as far as secureness goes.”
- deleted 5y ago
  [deleted]
- cascom 5y ago
  I think if it it more like having a fire Marshall inspect a building, doesn’t mean it can’t catch fire the next day, but at least you know that it’s not death trap (smoke alarms, fire doors, sprinklers etc all check out)
- tailspin2019 5y ago
  I think it's partly the phrase "Security experts declare" which is likely to be what's rubbing people up the wrong way. The security auditors themselves would never actually word it like that in their reports, because the statement implies a degree of certainty that cannot really exist. Here's an example of what the auditor's actually said: "Auditors identified two low-severity vulnerabilities. Additionally, five general recommendations were reported. At the same time, we confirm that no important security issues were identified during the pentest." There's a reason that audit reports will never say outright that something is "secure". They may say something like "strong and effective security measures are in place", but that's a very different kind of statement. I think the article itself is great but the headline just falls on the wrong side of being a bit hyperbolic and seems to be optimised for marketing impact over accuracy.
- josh2600 5y ago
  Here’s how I feel about this: You can write all of the tests, you can get all of the audits, but there’s nothing that’s going to stop a 13-year old polish kid from mucking about in the guts of your tech. Security isn’t a promise you can make in absolutes, at some point you have to ship and you hope that you did everything well enough that there’s no low-hanging fruit. There’s no such thing as a secure system, only systems which are more expensive to compromise.
- Beldin 5y ago
  > when can you claim something as secure? Typically: don't do that. Nevertheless, if you insist: you can claim a certain abstraction of a system guarantees certain mathematically expressed requirements cannot be violated by a certain attacker model once you've formally proved that. Of course, all implementations have implementation details which violate the abstraction, your mathematically expressed requirements may not fully capture your intentions, and in practice, an attacker may have additional options that your model doesn't consider. But hey, now you can truthfully claim that "the system" is "secure" - for some values of "system" and "secure".
- MattPalmer1086 5y ago
  You can't claim anything is secure just because problems haven't been found. It does build confidence in the security to perform security audits.
LinuxBender 5y ago
You beat me to it. I have sat with hundreds of software and network auditors. Never once have I heard them say something was secure. At best they might say they didn't find any high severity issues in this particular audit. I am curious who their statement was intended for.
WhyNotHugo 5y ago
Considering we live in world where Google and Facebook publicly state "We care about your privacy", I think this title pretty close to reality. Have they proved the non-existence of bugs? Nope. But the title is also not the complete opposite of reality, which is what their competition seems to doing.

capableweb 5y ago

I agree with you that the title is a bit sensationalist. But if independent security audits with no major issues uncovered cannot make you claim something is secure, when can you claim something as secure? Or are you of the opinion that nothing ever can be claimed to be secure as there can always be holes that could be uncovered in the future? Using openssh as an example, would you say it's secure when you're using public keys for the authentication? Their track record seems pretty good for the last years, but there might still be uncovered vulnerabilities, could it still claim to be secure?

sodality2 5y ago
> But if independent security audits with no major issues uncovered cannot make you claim something is secure, when can you claim something as secure? Nothing at all; it's a broken model. The server can at any time start serving malicious payloads [0]. The server hosts your mail but they also serve the webapp. The clientside decrypts the mail, but the server hosts the client code... It's a fundamentally flawed idea, trying to retrofit encryption into email in this way, when the server essentially has to hold all of your mail. In this case, the only thing that would make me feel secure in using it, is a third-party OSS client that downloads the mail without using the webapp, using only client-side code. And even then, of course, the mail can simply just be not encrypted when being ingested by Proton. So even then I wouldn't really trust it without external encryption like PGP. In which case, why bother? To be clear I do use private email services (protonmail, tutanota) but I am simply not going to fall for the illusion of guaranteed privacy; I just trust that they are what they say they are. They are still a better option IMO than something like Gmail, but I don't think they're a silver bullet. [0]: If you think this is unlikely, see this: https://news.ycombinator.com/item?id=25337507 https://news.ycombinator.com/item?id=25337507
- upofadown 5y ago
  It appears the audit was applied to the Android and iOS apps. So no comment is being made here about the security of the webapp.
  - sodality2 5y ago
    I thought that was just a wrapper around the website, to be honest. But regardless, the app still loads JS from the website and executes it, so it's still a backdoor possibility.
- prophesi 5y ago
  This is why I'm excited for 9elements and Mullvad's System Transparency[0][1] project. The goal is to let the end users know that your server is indeed running the code it says it's running. Of course, open firmware and hardware also plays a role in this as a server running binary blobs still has the potential for malicious code to be unknowingly run. [0] https://mullvad.net/en/blog/2019/6/3/system-transparency-future/ https://mullvad.net/en/blog/2019/6/3/system-transparency-fut... [1] https://www.system-transparency.org/ https://www.system-transparency.org/
- uselpa 5y ago
  Secure email is snake oil; no amount of cruft can make it both reasonable secure and useful (as in federated). Other protocols better fill that space because they were designed for security needs.
  - sodality2 5y ago
    End to end secure email definitely is, I agree. No matter what you have to trust the server. (Maybe a Tor-based email system would be better, where each user is their own server? reminds me of `cables`). If you do trust the server then it can be acceptably secure.
  - gruez 5y ago
    I agree that I wouldn't use Protonmail if Signal was an option, but there are many situations where Signal isn't an option (eg. transactional email, people who don't use signal). In those contexts it's still better to use encrypted mail rather than ad-supported mail (eg. gmail, outlook), or even commercial mail (eg. fastmail), which are both unencrypted.
yakak 5y ago
There is a framework for this: https://en.wikipedia.org/wiki/Evaluation_Assurance_Level https://en.wikipedia.org/wiki/Evaluation_Assurance_Level Proton is claiming something similar to EAL4, which is not secure, there is an assurance that not all trained reviewers can find a vulnerability. Openssh is a little less secure than that formally, but has more trained reviewers informally, which probably cover some parts extremely well and other parts sparsely.
jsjohnst 5y ago
> when can you claim something as secure? here’s a maybe wild take, uh, never?
- capableweb 5y ago
  But you'd agree with me that some things are more secure than others right?
  - jsjohnst 5y ago
    I’d be willing to say “there are things with known exploits, there are things which employ techniques that are known not to be safe making them less secure, and the rest is more akin to Schrödinger's cat as far as secureness goes.”
deleted 5y ago
[deleted]
cascom 5y ago
I think if it it more like having a fire Marshall inspect a building, doesn’t mean it can’t catch fire the next day, but at least you know that it’s not death trap (smoke alarms, fire doors, sprinklers etc all check out)
tailspin2019 5y ago
I think it's partly the phrase "Security experts declare" which is likely to be what's rubbing people up the wrong way. The security auditors themselves would never actually word it like that in their reports, because the statement implies a degree of certainty that cannot really exist. Here's an example of what the auditor's actually said: "Auditors identified two low-severity vulnerabilities. Additionally, five general recommendations were reported. At the same time, we confirm that no important security issues were identified during the pentest." There's a reason that audit reports will never say outright that something is "secure". They may say something like "strong and effective security measures are in place", but that's a very different kind of statement. I think the article itself is great but the headline just falls on the wrong side of being a bit hyperbolic and seems to be optimised for marketing impact over accuracy.
josh2600 5y ago
Here’s how I feel about this: You can write all of the tests, you can get all of the audits, but there’s nothing that’s going to stop a 13-year old polish kid from mucking about in the guts of your tech. Security isn’t a promise you can make in absolutes, at some point you have to ship and you hope that you did everything well enough that there’s no low-hanging fruit. There’s no such thing as a secure system, only systems which are more expensive to compromise.
Beldin 5y ago
> when can you claim something as secure? Typically: don't do that. Nevertheless, if you insist: you can claim a certain abstraction of a system guarantees certain mathematically expressed requirements cannot be violated by a certain attacker model once you've formally proved that. Of course, all implementations have implementation details which violate the abstraction, your mathematically expressed requirements may not fully capture your intentions, and in practice, an attacker may have additional options that your model doesn't consider. But hey, now you can truthfully claim that "the system" is "secure" - for some values of "system" and "secure".
MattPalmer1086 5y ago
You can't claim anything is secure just because problems haven't been found. It does build confidence in the security to perform security audits.

LinuxBender 5y ago

You beat me to it. I have sat with hundreds of software and network auditors. Never once have I heard them say something was secure. At best they might say they didn't find any high severity issues in this particular audit. I am curious who their statement was intended for.

WhyNotHugo 5y ago

Considering we live in world where Google and Facebook publicly state "We care about your privacy", I think this title pretty close to reality. Have they proved the non-existence of bugs? Nope. But the title is also not the complete opposite of reality, which is what their competition seems to doing.