6 ms·

Tarsnap: No heartbleed here

222 points by cperciva 13y ago

orthecreedence 13y ago
This is a plus to distributing your own app...you can just bundle the public key(s) for your servers with the app and then there's never any reason to verify through CAs or any such nonsense. Although in Tarsnap's case, an SSL leak wouldn't be terrible because everything's encrypted before even going over the wire, at least to my understanding.
- cperciva 13y ago
  This is a plus to distributing your own app...you can just bundle the public key(s) for your servers with the app Yes, this is one of my standard recommendations: If you can, distribute your own damn keys. in Tarsnap's case, an SSL leak wouldn't be terrible because everything's encrypted before even going over the wire Correct, and requests are authenticated using signatures, so even if Tarsnap was using SSL, there would be no tokens disclosed which could be stolen. Breaking Tarsnap's client-server encryption/authentication would allow an attacker to identify the names of blocks of data, though -- so he could see e.g., that someone is deleting today the data which was uploaded yesterday.
casca 13y ago
For anyone else interested in the protocols that tarsnap uses - https://www.tarsnap.com/crypto.html https://www.tarsnap.com/crypto.html
zck 13y ago
I'm no security expert, so this is an honest question. Why was Tarsnap running a version of OpenSSL that's at least two years old^1 ? My understanding is that, while you might not want to upgrade immediately upon release (because there may be new bugs), neither should you never upgrade (because releases fix old bugs). Is it because the website itself doesn't do much -- it's not the business; backup is -- so not much time is put into it? [1] OpenSSL 1.0.1, the first version of OpenSSL with the bug, was released in March 2012: http://heartbleed.com/ http://heartbleed.com/
- revelation 13y ago
  Security fixes are usually backported for affected versions, and presumably Colin installed these updates. This depends on the distribution you choose and if you are running a release where they have guaranteed critical updates (i.e. Ubuntu LTS releases). What you do miss out on is new protocol support, which for TLS can be a huge problem when secure and compatible crypto options run out. (Sadly, people will usually err on the side of compatibility, which in the past boils down to the terrible RC4)
- cperciva 13y ago
  The Tarsnap website is running on FreeBSD 9, which gets OpenSSL bug fixes backported.
- tptacek 13y ago
  OpenSSL frequently hosts vulnerabilities that don't affect all users of TLS. If you pay attention to the updates (as Colin surely does), you can filter down to the updates that matter. It's a risky thing to do if you're not willing to own those judgement calls; I wouldn't recommend that most shops do that. Among other things, you need to eyeball the diffs.
  - X-Istence 13y ago
    Or he is running an older version of FreeBSD that contains an older version of OpenSSL in the base.
    - tptacek 13y ago
      For what it's worth: he's the former FreeBSD security officer, so I wouldn't worry too much about which version of FreeBSD he's using.
      - deleted 13y ago
        [deleted]
        13y ago
soundsop 13y ago
The older posting that is mentioned, which is called Securing an HTTPS Server is a good read: http://www.daemonology.net/blog/2009-09-28-securing-https.html http://www.daemonology.net/blog/2009-09-28-securing-https.ht...
rst 13y ago
Title's correct: there is no heartbleed in Tarsnap, because the only thing that was using OpenSSL was using an old, unbugged version. That said, he goes on to claim that even if he had been using a buggy SSL stack, his stunnel setup "keeps the bug away from anything sensitive." But his prior blogpost on the stunnel setup[1] itself acknowledges some limits: "if someone compromises OpenSSL, they will still be ... able to steal the SSL certificate, to be sure, and also able to intercept other HTTPS connections". Meaning, I think, they'd have access to the net traffic in cleartext --- including such goodies as usernames and passwords, which might be shared with other sites. (Colin might respond, "use a password manager". He very likely does. Most likely, though, a lot of his clients don't.) So, if they (well, he) had been running a vulnerable OpenSSL, he might still have things to worry about. [1] http://www.daemonology.net/blog/2009-09-28-securing-https.html http://www.daemonology.net/blog/2009-09-28-securing-https.ht...
- jmileham 13y ago
  I don't think he'd refute that he'd have more to worry about had he not been lucky enough to be on an unaffected version of OpenSSL. I believe his point was that the use of stunnel to terminate SSL connections mitigates some of the attack vectors that could've been used to recover customer information in the event of having been compromised at the OpenSSL layer, and that the architecture of Tarsnap itself absolutely precludes recovery of customer backups in any event. And that these facts aren't an accident. The important takeaway from this post is that it pays to employ layers of security when building software systems.
  - ams6110 13y ago
    If I understand correctly, he's decoupled the SSL connection handling from the http server. That seems to have all kinds of advantages to me, for example if a vulnerability in your SSL library is found you could quickly swap in SSL termination based on another library (e.g. gnutls or nss) or if there was a vulerablity in stunnel you could change it out for stud or even apache or nginx in a jail (assuming you had any or all of these things ready to go). Should also make for more flexibility with load balancing. Brilliant engineering.
  - 13y ago
atmosx 13y ago
I think the juice is in the comments[1]. [1] http://www.daemonology.net/blog/2014-04-09-tarsnap-no-heartbleed-here.html#comment-1327391638 http://www.daemonology.net/blog/2014-04-09-tarsnap-no-heartb...
- pseut 13y ago
  And, just to go back to patio11's article[1] "replace OpenSSL for free" is something that Tarsnap could afford to do as a company, and might make business sense, if it were substantially more profitable. </nag> https://news.ycombinator.com/item?id=7523953https://news.ycombinator.com/item?id=7523953 https://news.ycombinator.com/item?id=7523953https://news.yco...
Oculus 13y ago
I read the blog post earlier and this one line really resonated with me: we don't assess the structure of bridges by asking "has it collapsed yet?" Security is one of those notoriously hard fields to get right for precisely this reason: you don't know you're doing a bad job until it's too late.
- BrandonM 13y ago
  I like that. I think by that measure, though, OpenSSL is failing badly. It's like an old rusted-metal and rotten-wood bridge that should have a sign saying, "No vehicles over 2 tons," but instead it's carrying highway traffic. I'm starting to get frustrated by the security experts basically chasing everyone away from cryptography. Am I the only one getting tired of the experts (many with misaligned incentives) telling us, "This stuff is really hard... Just trust us"?
  - 001spartan 13y ago
    The fact remains that crypto _is_ very hard. I don't see why telling people that equates to "chasing people away" from cryptography. It would be even more dangerous to not mention that fact when people try to roll their own encryption systems. It's just not something that can be safely done by a hobbyist/amateur, at least not for critical applications.
    - BrandonM 13y ago
      In many other areas of CS (e.g., gaming, language development, DB design), the experts are simply more willing to discuss or enumerate the requirements of the problems they are solving. They make an effort to educate those who are interested in learning. In security, you have a cabal of self-appointed experts who write inscrutable code and chastise you if you try to improve on it. What does that accomplish other than chasing people away? These security experts make their livelihoods based on their credentials; they have every incentive to keep others out. At what point do we stop giving them the benefit of the doubt and start educating ourselves? Trust can only go so far.
tedchs 13y ago
Does Heartbleed affect OpenSSH? I seem to recall it uses OpenSSL but I haven't seen it mentioned.
- twic 13y ago
  No. SSH is a completely different protocol to TLS/SSL: http://tools.ietf.org/html/rfc4253 http://tools.ietf.org/html/rfc4253
yp_master 13y ago
I prefer SSL termination on the client-side (my computer, my data only). I like to have the ability to view my SSL traffic in plain text. Should the user be able to see what her computer is sending out? I think she should. And encrypted traffic should not be some special exception. Installing someone else's "MITM" software to decrypt SSL seems unnecessary. It is much simpler to generate and install your own "fake" certificates that you control. stunnel is one option. There are others. socat, Pound, etc. It should be the user who has the final decision over which certificates to trust. Users are the real "Certificate Authorities". They should have full control over encryption and decryption should they want to exercise it. Is it wise to irrevocably delegate the decision to trust/not trust to website owners and browser authors? Perhaps those promoting solutions like "TACK" should give this more thought.
- jlgaddis 13y ago
  > It should be the user who has the final decision over which certificates to trust. Users are the real "Certificate Authorities". They should have full control over encryption and decryption should they want to exercise it. And they do. I'm not sure what you're getting at here. You and I and my mother all have the ability to edit the root CA certificates on our computers and add our own, if we wish.
  - yp_master 13y ago
    Indeed, that has been my solution. But I'm seeing more and more authentication information being incorporated ("baked in", pre-installed, whatever) into browsers, whether it is lists of "valid" TLD's, certificates for "approved" CA's, or chosen individual website certificates. Personally, I think this information should be cleanly separated from the software that may use it rather than pre-installed and "hidden from the user".

222 points by cperciva 13y ago

This is a plus to distributing your own app...you can just bundle the public key(s) for your servers with the app and then there's never any reason to verify through CAs or any such nonsense. Although in Tarsnap's case, an SSL leak wouldn't be terrible because everything's encrypted before even going over the wire, at least to my understanding.

cperciva 13y ago
This is a plus to distributing your own app...you can just bundle the public key(s) for your servers with the app Yes, this is one of my standard recommendations: If you can, distribute your own damn keys. in Tarsnap's case, an SSL leak wouldn't be terrible because everything's encrypted before even going over the wire Correct, and requests are authenticated using signatures, so even if Tarsnap was using SSL, there would be no tokens disclosed which could be stolen. Breaking Tarsnap's client-server encryption/authentication would allow an attacker to identify the names of blocks of data, though -- so he could see e.g., that someone is deleting today the data which was uploaded yesterday.

casca 13y ago

For anyone else interested in the protocols that tarsnap uses - https://www.tarsnap.com/crypto.html https://www.tarsnap.com/crypto.html

zck 13y ago

I'm no security expert, so this is an honest question. Why was Tarsnap running a version of OpenSSL that's at least two years old^1 ? My understanding is that, while you might not want to upgrade immediately upon release (because there may be new bugs), neither should you never upgrade (because releases fix old bugs). Is it because the website itself doesn't do much -- it's not the business; backup is -- so not much time is put into it? [1] OpenSSL 1.0.1, the first version of OpenSSL with the bug, was released in March 2012: http://heartbleed.com/ http://heartbleed.com/

revelation 13y ago
Security fixes are usually backported for affected versions, and presumably Colin installed these updates. This depends on the distribution you choose and if you are running a release where they have guaranteed critical updates (i.e. Ubuntu LTS releases). What you do miss out on is new protocol support, which for TLS can be a huge problem when secure and compatible crypto options run out. (Sadly, people will usually err on the side of compatibility, which in the past boils down to the terrible RC4)
cperciva 13y ago
The Tarsnap website is running on FreeBSD 9, which gets OpenSSL bug fixes backported.
tptacek 13y ago
OpenSSL frequently hosts vulnerabilities that don't affect all users of TLS. If you pay attention to the updates (as Colin surely does), you can filter down to the updates that matter. It's a risky thing to do if you're not willing to own those judgement calls; I wouldn't recommend that most shops do that. Among other things, you need to eyeball the diffs.
- X-Istence 13y ago
  Or he is running an older version of FreeBSD that contains an older version of OpenSSL in the base.
  - tptacek 13y ago
    For what it's worth: he's the former FreeBSD security officer, so I wouldn't worry too much about which version of FreeBSD he's using.
    - deleted 13y ago
      [deleted]
      - 13y ago

soundsop 13y ago

The older posting that is mentioned, which is called Securing an HTTPS Server is a good read: http://www.daemonology.net/blog/2009-09-28-securing-https.html http://www.daemonology.net/blog/2009-09-28-securing-https.ht...

rst 13y ago

Title's correct: there is no heartbleed in Tarsnap, because the only thing that was using OpenSSL was using an old, unbugged version. That said, he goes on to claim that even if he had been using a buggy SSL stack, his stunnel setup "keeps the bug away from anything sensitive." But his prior blogpost on the stunnel setup[1] itself acknowledges some limits: "if someone compromises OpenSSL, they will still be ... able to steal the SSL certificate, to be sure, and also able to intercept other HTTPS connections". Meaning, I think, they'd have access to the net traffic in cleartext --- including such goodies as usernames and passwords, which might be shared with other sites. (Colin might respond, "use a password manager". He very likely does. Most likely, though, a lot of his clients don't.) So, if they (well, he) had been running a vulnerable OpenSSL, he might still have things to worry about. [1] http://www.daemonology.net/blog/2009-09-28-securing-https.html http://www.daemonology.net/blog/2009-09-28-securing-https.ht...

jmileham 13y ago
I don't think he'd refute that he'd have more to worry about had he not been lucky enough to be on an unaffected version of OpenSSL. I believe his point was that the use of stunnel to terminate SSL connections mitigates some of the attack vectors that could've been used to recover customer information in the event of having been compromised at the OpenSSL layer, and that the architecture of Tarsnap itself absolutely precludes recovery of customer backups in any event. And that these facts aren't an accident. The important takeaway from this post is that it pays to employ layers of security when building software systems.
- ams6110 13y ago
  If I understand correctly, he's decoupled the SSL connection handling from the http server. That seems to have all kinds of advantages to me, for example if a vulnerability in your SSL library is found you could quickly swap in SSL termination based on another library (e.g. gnutls or nss) or if there was a vulerablity in stunnel you could change it out for stud or even apache or nginx in a jail (assuming you had any or all of these things ready to go). Should also make for more flexibility with load balancing. Brilliant engineering.
- 13y ago

atmosx 13y ago

I think the juice is in the comments[1]. [1] http://www.daemonology.net/blog/2014-04-09-tarsnap-no-heartbleed-here.html#comment-1327391638 http://www.daemonology.net/blog/2014-04-09-tarsnap-no-heartb...

pseut 13y ago
And, just to go back to patio11's article[1] "replace OpenSSL for free" is something that Tarsnap could afford to do as a company, and might make business sense, if it were substantially more profitable. </nag> https://news.ycombinator.com/item?id=7523953https://news.ycombinator.com/item?id=7523953 https://news.ycombinator.com/item?id=7523953https://news.yco...

Oculus 13y ago

I read the blog post earlier and this one line really resonated with me: we don't assess the structure of bridges by asking "has it collapsed yet?" Security is one of those notoriously hard fields to get right for precisely this reason: you don't know you're doing a bad job until it's too late.

BrandonM 13y ago
I like that. I think by that measure, though, OpenSSL is failing badly. It's like an old rusted-metal and rotten-wood bridge that should have a sign saying, "No vehicles over 2 tons," but instead it's carrying highway traffic. I'm starting to get frustrated by the security experts basically chasing everyone away from cryptography. Am I the only one getting tired of the experts (many with misaligned incentives) telling us, "This stuff is really hard... Just trust us"?
- 001spartan 13y ago
  The fact remains that crypto _is_ very hard. I don't see why telling people that equates to "chasing people away" from cryptography. It would be even more dangerous to not mention that fact when people try to roll their own encryption systems. It's just not something that can be safely done by a hobbyist/amateur, at least not for critical applications.
  - BrandonM 13y ago
    In many other areas of CS (e.g., gaming, language development, DB design), the experts are simply more willing to discuss or enumerate the requirements of the problems they are solving. They make an effort to educate those who are interested in learning. In security, you have a cabal of self-appointed experts who write inscrutable code and chastise you if you try to improve on it. What does that accomplish other than chasing people away? These security experts make their livelihoods based on their credentials; they have every incentive to keep others out. At what point do we stop giving them the benefit of the doubt and start educating ourselves? Trust can only go so far.

tedchs 13y ago

Does Heartbleed affect OpenSSH? I seem to recall it uses OpenSSL but I haven't seen it mentioned.

twic 13y ago
No. SSH is a completely different protocol to TLS/SSL: http://tools.ietf.org/html/rfc4253 http://tools.ietf.org/html/rfc4253

yp_master 13y ago

I prefer SSL termination on the client-side (my computer, my data only). I like to have the ability to view my SSL traffic in plain text. Should the user be able to see what her computer is sending out? I think she should. And encrypted traffic should not be some special exception. Installing someone else's "MITM" software to decrypt SSL seems unnecessary. It is much simpler to generate and install your own "fake" certificates that you control. stunnel is one option. There are others. socat, Pound, etc. It should be the user who has the final decision over which certificates to trust. Users are the real "Certificate Authorities". They should have full control over encryption and decryption should they want to exercise it. Is it wise to irrevocably delegate the decision to trust/not trust to website owners and browser authors? Perhaps those promoting solutions like "TACK" should give this more thought.

jlgaddis 13y ago
> It should be the user who has the final decision over which certificates to trust. Users are the real "Certificate Authorities". They should have full control over encryption and decryption should they want to exercise it. And they do. I'm not sure what you're getting at here. You and I and my mother all have the ability to edit the root CA certificates on our computers and add our own, if we wish.
- yp_master 13y ago
  Indeed, that has been my solution. But I'm seeing more and more authentication information being incorporated ("baked in", pre-installed, whatever) into browsers, whether it is lists of "valid" TLD's, certificates for "approved" CA's, or chosen individual website certificates. Personally, I think this information should be cleanly separated from the software that may use it rather than pre-installed and "hidden from the user".