9 ms·

HTTPS Interception Weakens TLS Security

215 points by mlindner 10y ago

mlindner 10y ago
The US government has basically declared "HTTPS/TLS Interception Considered Harmful". This is going to be interesting as all the major security load blanacer/appliances out there offer this as a standard service at this point.
- userbinator 10y ago
  Watch them change their opinion when the next leak of government information is determined to have happened over an unintercepted TLS connection...
- saurik 10y ago
  As far as I can tell this is targetted at what I would expect: "antivirus" and compliance tools built into firewalls, not reverse proxies built into load balancers.
  - hueving 10y ago
    There is limited distinction there. Both terminate and re-establish TLS sessions.
    - saurik 10y ago
      I strongly disagree: who is in control, how it is implemented, and the purpose of the product are all both extremely important and fundamentally different between the two use cases. An "antivirus" filter ends up implemented as a man-in-the-middle attack between software I wrote and servers I contract and control, wherein protections that were actively put in place are stripped away in order to increase the attack surface. A reverse proxy is something that I actively implemented as part of my infrastructure, and I am contracting for it and paying for it like anything else I rent. It became "extra cool" to hate on reverse proxies recently while everyone hates CloudFlare (and to be clear: I've hated CloudFlare since before it was cool to hate on CloudFlare, and you should check my comment and submission history to verify if you don't believe ;P), but what people are essentially complaining about are "it encourages you to do something insecure", which I think is actually largely true only of CloudFlare, and you have to remember that the vast majority of developers who are using CloudFlare are doing so because they don't know much about network administration. But like, think about it this way: Apache is also an SSL termination point. The code in your website is likely 99% "transform a URL into SQL, then transform the result into HTML": it is just a dumb encoder/decoder, and pretty exactly analogous to an SSL termination point. If you really believe in true end-to-end encryption, then having this webserver in the middle is a bad idea: you should route requests directly to database servers and ensure that at no point does any non-database computer have a decrypted copy of the information that will eventually be rendered to the user. Another thing to think about: if you are heavily concerned that the CDN or reverse proxy service you are using has access to your traffic, realize that they are no different than using a hosted server. We don't even need to invoke "the cloud" for this one: unless I built the machine myself, I have no way of knowing that the server I am renting to run my database server on isn't running under a hypervisor that is allowing dumps of my decrypted memory. A reverse proxy is just another piece of rented infrastructure that someone else is in control of, and I don't see many people (though if you are, that's great, and then I'd be really happy ;P) insisting that everyone build their own computers and host them at their office, to make sure decrypted data never touches a computer owned and operated by someone else. The risk profile is also just so fundamentally different, in no small part due to these differences of "who is in control and how is it implemented". Let's say that CDNetworks is failing to check SSL certificates correctly: how exactly are you going to attack that against my website? I mean, sure: if you are a state-level actor, that is going to be easy, but otherwise, that's going to be extremely rough. An "antivirus" filter that, as described in this alert, is failing to correctly check SSL certificates... well, that opens you up to attack by amateur attackers that are "close to you", such as a stalker or an ex-spouse or a corporate competitor. You can't just claim "well, if I abstract away all of the details, they look the same"... you have to look at why you care and verify that you didn't strip away anything required and relevant.
      - nickik 10y ago
        > The code in your website is likely 99% "transform a URL into SQL, then transform the result into HTML" My live seems more trivil now.
      - JshWright 10y ago
        > If you really believe in true end-to-end encryption, then having this webserver in the middle is a bad idea: you should route requests directly to database servers and ensure that at no point does any non-database computer have a decrypted copy of the information that will eventually be rendered to the user. I can't say I haven't considered abusing PL/pgSQL to that point... ;)
        _jal 10y ago
        I've actually seen that done - HTML straight out of a stored procedure. Really awesome <TR BGCOLOR=...><TD> stuff. I've mostly recovered from the experience, thanks, although I do have occasional flashbacks.
      - lmm 10y ago
        >An "antivirus" filter ends up implemented as a man-in-the-middle attack between software I wrote and servers I contract and control, wherein protections that were actively put in place are stripped away in order to increase the attack surface. A reverse proxy is something that I actively implemented as part of my infrastructure, and I am contracting for it and paying for it like anything else I rent. Those can equally happen the other way around. You might put an antivirus filter in place on your own network. Your hosting provider might set up their own reverse proxy. > But like, think about it this way: Apache is also an SSL termination point. The code in your website is likely 99% "transform a URL into SQL, then transform the result into HTML": it is just a dumb encoder/decoder, and pretty exactly analogous to an SSL termination point. If you really believe in true end-to-end encryption, then having this webserver in the middle is a bad idea: you should route requests directly to database servers and ensure that at no point does any non-database computer have a decrypted copy of the information that will eventually be rendered to the user. I agree with that - best to have the database and the web server on the same server, communicating via local socket or shared memory. Better still to compile the web layer and the data layer into the same unikernel. Current best practice (e.g. PCI-DSS) seems to accept having an (encrypted) network connection between distinct web and database servers, but only within a single physically secured rack. You're right that if you do the same thing with a load balancer (have it in the same physically secured rack as your web servers, encrypt connections between it and them) then it's no less secure, but I don't think people would want to be using a load balancer within a single rack anyway, since most of the point of a load balancer is availability and you want separate racks for that. Certainly a third-party reverse-proxy is a substantial weakening of security. > I don't see many people (though if you are, that's great, and then I'd be really happy ;P) insisting that everyone build their own computers and host them at their office, to make sure decrypted data never touches a computer owned and operated by someone else. I wouldn't go that far - AIUI PCI-DSS etc. require you have physically secured/access-controlled servers but allow you to rent space/power/cooling from a datacenter operator (though they don't get access to your rack, and possibly have to also be audited themselves). > Let's say that CDNetworks is failing to check SSL certificates correctly: how exactly are you going to attack that against my website? I mean, sure: if you are a state-level actor, that is going to be easy, but otherwise, that's going to be extremely rough. An "antivirus" filter that, as described in this alert, is failing to correctly check SSL certificates... well, that opens you up to attack by amateur attackers that are "close to you", such as a stalker or an ex-spouse or a corporate competitor. It's a different risk profile to be sure. Are you more worried about someone making a large-scale attack against the remote service you were connecting to, or just against the Internet in general? Or someone making a small-scale local attack? I think both can be serious; it's not just state-level actors but also large criminal rings or even political groups that would potentially have the resources to execute an attack against a particular service (e.g. a medical provider doing something politically contentious, or just anything that accepts credit cards / personal information).
    - deleted 10y ago
      [deleted]
    - drdaeman 10y ago
      When running on the same machine, reverse TLS-terminating proxy isn't different from built-in TLS implementation, except for done in "microservice-like" way. Most certainly, there's no difference whatsoever (or almost so, if the reverse proxy is "smart" and has some application logic in it) if your webapp doesn't speak HTTP natively, but talks FastCGI/uWSGI/SCGI protocols instead. Which is quite common. I mean, the actual interface (passing buffer pointers in the local process or passing same data via local UNIX sockets) shouldn't matter much, right? As long as interfaces have similar security properties and all the necessary data is passed around properly. A sufficiently privileged user can introspect both (with ltrace or tcpdump), and insufficiently privileged one can't do anything anyway.
    - jupp0r 10y ago
      Not necessarily. A common pattern for load balancers is terminating TLS and communicating with backends unencrypted/unauthenticated.
- deleted 10y ago
  [deleted]
- micaksica 10y ago
  I don't think that's the correct interpretation of this document. A better title would be "Most TLS Proxies Considered Harmful", not TLS interception as a whole. US-CERT is explicitly calling out that many appliances and applications often are weakening security vs. the common TLS clients, by way of not properly validating certificate chains by proxy for the client and otherwise masking SSL/TLS issues from being surfaced to the end user. I believe this is CERT trying to get software companies to get their act together on solving security-detrimental UX/UI issues vs. a blanket statement not to add TLS interception proxies to a corporate network.
  - hannob 10y ago
    > I don't think that's the correct interpretation of this document. A better title would be "Most TLS Proxies Considered Harmful", not TLS interception as a whole. This is the usual reaction to this I see everywhere. It doesn't match with reality. Read the complete paper by Halderman et al [1]. It says pretty clearly that all existing products lower the security in one way or another: "Our grading scale focuses on the security of the TLS handshake and does not account for the additional HTTPS validation checks present in many browsers, such as HSTS, HPKP, OneCRL/CRLSets, certificate transparency validation, and OCSP must-staple. None of the products we tested supported these features." TLS Interception that doesn't degrade security is a myth that exists only in some parallel world fantasy. It's also not very surprising. Browser vendors have extremely qualified security teams that are involved in many ways in the development of stronger TLS protections. Nothing like that can be said about any of the vendors of interception products. [1] https://jhalderm.com/pub/papers/interception-ndss17.pdf https://jhalderm.com/pub/papers/interception-ndss17.pdf
    - mathw 10y ago
      Is better TLS interception actually possible? Is it just that it's usually badly implemented, or is it a fundamental weakening? Other than the unavoidable weakening of having another system seeing the unencrypted data, which is inherent in using one of these systems and can never go away.
      - 10y ago
- Spooky23 10y ago
  No, they haven't. They say pretty plainly that you need to ensure that such products provide correct certificate validation, since the client cannot do so reliably.
  - Ajedi32 10y ago
    Well, it does also say: > In general, organizations considering the use of HTTPS inspection should carefully consider the pros and cons of such products before implementing So I think there's at least some element of "don't do this unless you have to" in there. But for the most part yeah, the alert seems to be more about ensuring the interception is done securely than about not doing it at all.
  - jamiesonbecker 10y ago
    >> The US government has basically declared "HTTPS/TLS Interception Considered Harmful". > No, they haven't. The original report headline from US-CERT: "HTTPS Interception Weakens TLS Security" That seems pretty unambiguous, regardless of your personal feelings regarding CERT or whether it speaks for the broader US government.
    - Spooky23 10y ago
      Don't analyze a book by its cover!
- deleted 10y ago
  [deleted]
- tptacek 10y ago
  US-CERT has very little influence, even in regulated industries. As usual, CERT is saying something formally that has been axiomatic among practitioners for many years. The only conclusion I'd draw from this is about CERT. Obviously: what they're saying is broadly true. I also agree with the other commenters here who are pointing out that this is largely a response to endpoint TLS interception, and that the point is to focus attention on tools that intercept but don't validate certificates --- CERT is not telling every Fortune 500 company and every federal agency to stop deploying BlueCoat boxes. But the more important thing to push back on in your comment is the notion that a proclamation from CERT has any real meaning. It doesn't, even within FedGov IT.
jusob 10y ago
It does not have to be. Done correctly, SSL interception can pass through all the errors to the client: * certificate issues (expiration, domain mismatch, etc.) * OCSP/CRL verification * validation of HPKP header I understand that few vendors may be doing it (I know one which does at least the first 2). Probably the worst offense is choosing the weakest TLS version + cipher to save resources, like using TLS 1.0 because it take less resources to decode/encode than TLS 1.2 + elliptic curve.
- userbinator 10y ago
  On the other hand, a MITM proxy can also do upgrade "attacks"(?!), communicating with remote servers over the Internet using a stronger protocol than the clients on the LAN behind it support. In fact it seems to me that having the validation happening in one place may potentially be easier to maintain than across many different clients' software.
  - hackcasual 10y ago
    My experience with browser companies vs. proxy software companies is that the browser vendors give a much bigger shit about end user security.
    - andrewflnr 10y ago
      You're assuming people on the LAN can upgrade to recent browsers. A lot of them are stuck on Windows XP. I realize that's a "you have bigger problems" type of scenario, but you also have to play the hand you're dealt.
      - Laforet 10y ago
        If an organisation has a large number of networked computers on Windows XP they are going to have more issues than that - having no SNI support is one.
        10y ago
evilDagmar 10y ago
TL;DR: If your organization is going to do HTTPS interception, don't screw it up.
- jaimex2 10y ago
  or make sure your appliance/software vendor isn't screwing up.
- angry_octet 10y ago
  I think the take away is that your mitm box will eventually screw up, and you will be liable for whatever results, from brokenness (e.g. the 40,000 chromebooks going offline) to complete disclosure of company-breaking information.
synicalx 10y ago
It might weaken TLS, but it also stops the 3000 head of cattle I managed from being able to watch porn (6 incidents) and torrent movies (47 copyright notices). If their internet banking, which they're not supposed to be doing at work, gets compromised then I really couldn't care less.
- plugger 10y ago
  Instead couldn't you just whitelist the online banking sites and intercept everything else?
  - jaimex2 10y ago
    he just said he doesn't care :P
  - synicalx 10y ago
    That would defeat the whole point of a proxy, if I can't see what they're doing and report on people who are doing the wrong thing (according to their employment contract), then why bother?
    - mulmen 10y ago
      Because if you block it then they can't do it and you have nothing to report. Everyone wins.
      - jupp0r 10y ago
        That's one way to see it. In my experience, those proxies in schools are a great way to motivate students to self-educate about how the internet works and about online privacy tools like TOR...
        mulmen 10y ago
        Why is a proxy necessary for a student to self-educate? Are you just saying this is a lesson they learn the hard way once they have been spied on?
jaimex2 10y ago
Reading the title ... no shit.
- jupp0r 10y ago
  Exactly my idea, but after reading the article I'm severely disappointed that the whole practice isn't declared bad. Instead it only points out that some mitm boxes are even more broken than I though possible and mess up cert validation.
  - jaimex2 10y ago
    It's a terrible practice, the article doesn't even go into some other things like how easy is it to actually break into the intercepting software and steal the private cert so you can mitm the network for free! and I'm a dev at one of these companies! We really didn't want to add the feature but we were bleeding customer's who for some reason had to have it. ( Yes, we do validate the certificates ) We decided to bite the bullet when it looked like even Google was "supporting" being mitmed as a customer pointed out. Reading this article... https://support.google.com/a/answer/1668854?hl=en https://support.google.com/a/answer/1668854?hl=en was my biggest "fuck you" moment.
Keverw 10y ago
A while back I remember seeing on HN there was a issue with a certain vendor and ChromeBooks because Chrome used a newer TLS(And the mitm vendor vendor was noticed in advance too, and didn't update their product). I wonder how schools and banks plan to react to this... Apparently financial firms have to record everything their employees do for some regulations. To me, schools doing this sort of thing is wrong. I wouldn't be surprised if the principle would grab people's passwords and login to their accounts even. I know some schools even went as far to demand students hand over their passwords to social media when they report bullying... Which if the school blocks social networks anyways, I don't see how it's a school issue for what happens outside of school... If this sort of thing really needs to be done, at-least people should be warned and aware they are being monitored. If it's for a bank and it's only company equipment everything is being monitored it seems a bit more okay to do if everyone is well aware. "You are only to use work computers for official business." sort of policy.
- JumpCrisscross 10y ago
  > financial firms have to record everything their employees do for some regulations Phone calls, emails, instant messages and other forms of client contact, yes. Internet browsing history? No. Rest assured, many firms do this. But it's because they decided to, not because of regulation.
  - Mandatum 10y ago
    Worked for Big 4. They did not log my phone calls or sniff my TLS traffic.
    - newjersey 10y ago
      Why isn't it enough to do the tracking/audit log in the server side? I can't imagine even the JP Morgan Chase CEO lives under surveillance of JP Morgan Chase Bank 24/7. Just require them to use centralized services and keep audit log of that? Sounds simple enough. I don't get the need to monitor all traffic. Surely the sensitive servers have audit log independent of network?
    - 10y ago
tofflos 10y ago
So how do I test if my workplace is doing a good job of this? The article mentions badssl.com. Do I just click all the red links in the certificate section and verify that my browser is refusing to display the pages?
- j_s 10y ago
  Just wanted to say thanks for attempting to point the conversation in a useful direction; I hope someone who can help is willing to share the answer.
- tgasson 10y ago
  You want to do that both on your intercepted LAN and with a direct connection and compare the results. If they both give the same results you can be happy your MitM proxy is at least treating insecure sites equivalently. You still lose the advantage that you browser will tell you why the site is insecure
- mleonhard 10y ago
  Click the Dashboard link at the top-left. https://badssl.com/dashboard/ https://badssl.com/dashboard/
Animats 10y ago
We need MITM detection in the browser. Yes, it's possible. The crypto bits the host is sending are different from the crypto bits the client is receiving. There are several ways to compare those, despite what the MITM box is doing. Out of band channels, timing, and order of data can be used. I sometimes refer to HTTPS Everywhere as "Security Theater Everywhere". Before the mania for HTTPS, many sites only used HTTPS only for crucial transactions such as logins and credit cards. Those were infrequent enough that they didn't have to go through a CDN. Now, with HTTPS Everywhere, there's no distinction between the stuff that has to be hidden from observers, and the stuff which only needs something like Subresource Integrity to make sure it hasn't been messed with. So now the secure channel over which credit card numbers and logins are passed is exposed at the CDN.
- deleted 10y ago
  [deleted]
- averagewall 10y ago
  Then those organizations using MITM boxes will install a patch or special modified browser on their members' computers instead of installing a certificate. What's the difference? They still control the computer and can bypass whatever security it has.
- ancarda 10y ago
  It's not safe to do only logins over HTTPS because if the initial site is served over HTTP, an attacker could rewrite the URL or inject JavaScript to leak your password. Also, once you're signed in, every request sends a cookie which is also valuable to steal as it's an active session. Every request needs to be encrypted. We haven't made TLS worse by deploying it everywhere.
  - 4ad 10y ago
    Logins? 99% of the websites I visit don't have user accounts.
    - icebraining 10y ago
philbert 10y ago
I still find it unfortunately shallow analysis. I'm currently fighting a battle in a company in the middle of rolling out Blue Coat ProxySG. I only became aware of it because it began causing interruptions to our work since none of the development tools get the necessary root cert to validate the certs that the proxy is rewriting. The root cert is only installed into the Windows credentials to make browsers work and it's left up to every client to fix whatever other problems they have. One of the common arguments I've encountered from people is along the lines of "if the company trusts the security people, then I trust the security people". However this ignores the fact that the company has legally enforceable agreements in the form od employment contracts, NDAs, and what-have-you that protect the company against a breach of that trust. But what kind of protection do you have as an individual if, for what ever reason, one of the security people decides to single you out while they have unsupervised access to you social media logins, personal email and credit cards numbers and financial details (and lets face the fact that everyone does all of these things to some degree on a work computer). The answer is that you more than likely have no protection at all (likely not even in workplace law). Even if you suspected someone had inappropriately accessed your personal details, the company almost certainly has an IT policy saying that you shouldn't be using your computer for personal use. You're screwed. However these inspection boxes drastically change the situation to what it was before, because most people outside the IT department won't even know about this drastic change in capability. And likely they try hard not to think about until (if) they discover something has happened. I'm sure everyone here is aware of the abuses of NSA employees spying on their wifes, or ex-es [0]. Such is human nature. Sadly I suspect that all these arguments will do little to change the situation. It seems more likely that the companies who deploy these systems are only going to listen to arguments specifically about how it is increasing work, causing delays of deliveries and affecting project costs. [0] http://www.reuters.com/article/us-usa-surveillance-watchdog-idUSBRE98Q14G20130927 http://www.reuters.com/article/us-usa-surveillance-watchdog-...
- rkeene2 10y ago
  TLS MITM devices also break required components of TLS, such as TLS client certificates. If you want to have TLS sessions with mutual authentication it can't go through the MITM. A way forward might be to start requiring TLS client certificates for business products. :-) For us, we use TLS client certificate authentication to authenticate almost all our users so any TLS MITM breaks that and is unacceptable. This is really common and tens of millions of people use TLS client certificates this way everyday -- mostly with client certificates on smartcards.
tinus_hn 10y ago
And the answer is going to be custom unspecified encryption inside the encrypted channel. HTTPS mitm proxying is a dumb idea that can't work right but is easy to sell to executives. It will be defeated until we're at the state of a few years ago, where the products that really matter, like banking, have tough security and the rest has less.
seasonalgrit 10y ago
any recommendations on tutorials/guides for better understanding the world of TLS, certificates, and so on? i don't feel like i have a healthy mastery of the ideas discussed in articles like this one.
- kbart 10y ago
  HTTPS is basically HTTP over TLS, so start with RFCs: - TLS v1.2 (https://tools.ietf.org/html/rfc5246 https://tools.ietf.org/html/rfc5246) - Certificate validation (https://tools.ietf.org/html/rfc6960 https://tools.ietf.org/html/rfc6960) - PKI is a little more tricky, as there's no single standard that defines it, though to get basic idea of what are certificates read RFC 5280 (https://tools.ietf.org/html/rfc5280 https://tools.ietf.org/html/rfc5280). Sure, you can dig as deep as you want, but these resources should give you a good starting point.
  - rkeene2 10y ago
    Don't forget RFC 2818 (HTTPS) which Chromium (and thus Chrome) recently broke mandatory features of: https://bugs.chromium.org/p/chromium/issues/detail?id=308330#c39 https://bugs.chromium.org/p/chromium/issues/detail?id=308330...
    - NetStrikeForce 10y ago
      Is this a remake of that "embrace, extend, ..." movie of the 90s or this time it's cool?
      - rkeene2 10y ago
        It's really very similar. This is a case where the Chromium developers want to do something so they are going to do it, standards be dammed...
- nikcub 10y ago
  Ivan Ristic's "Bulletproof SSL and TLS"[0] [0] https://www.feistyduck.com/books/bulletproof-ssl-and-tls/reviewerKit.html https://www.feistyduck.com/books/bulletproof-ssl-and-tls/rev...
omribahumi 10y ago
This is what squid has been trying to solve with their server first bumping[0] and peek-and-splice[1] [0] http://wiki.squid-cache.org/Features/BumpSslServerFirst http://wiki.squid-cache.org/Features/BumpSslServerFirst [1] http://wiki.squid-cache.org/Features/SslPeekAndSplice http://wiki.squid-cache.org/Features/SslPeekAndSplice edit: reference peek-and-slice too
Khaine 10y ago
One important reason to do ssl inspection is to detect the exfiltration of corporate data, particularly if your organisation is likely to be attacked by governments or sophisticated crime gangs. I know my organisation do SSL inspection, but they have whitelisted common personal websites like internet banking to protect the privacy of staff. As far as I'm concerned, if your risk profile requires you to do SSL inspection then this is the right way to do it.

215 points by mlindner 10y ago

The US government has basically declared "HTTPS/TLS Interception Considered Harmful". This is going to be interesting as all the major security load blanacer/appliances out there offer this as a standard service at this point.

userbinator 10y ago
Watch them change their opinion when the next leak of government information is determined to have happened over an unintercepted TLS connection...
saurik 10y ago
As far as I can tell this is targetted at what I would expect: "antivirus" and compliance tools built into firewalls, not reverse proxies built into load balancers.
- hueving 10y ago
  There is limited distinction there. Both terminate and re-establish TLS sessions.
  - saurik 10y ago
    I strongly disagree: who is in control, how it is implemented, and the purpose of the product are all both extremely important and fundamentally different between the two use cases. An "antivirus" filter ends up implemented as a man-in-the-middle attack between software I wrote and servers I contract and control, wherein protections that were actively put in place are stripped away in order to increase the attack surface. A reverse proxy is something that I actively implemented as part of my infrastructure, and I am contracting for it and paying for it like anything else I rent. It became "extra cool" to hate on reverse proxies recently while everyone hates CloudFlare (and to be clear: I've hated CloudFlare since before it was cool to hate on CloudFlare, and you should check my comment and submission history to verify if you don't believe ;P), but what people are essentially complaining about are "it encourages you to do something insecure", which I think is actually largely true only of CloudFlare, and you have to remember that the vast majority of developers who are using CloudFlare are doing so because they don't know much about network administration. But like, think about it this way: Apache is also an SSL termination point. The code in your website is likely 99% "transform a URL into SQL, then transform the result into HTML": it is just a dumb encoder/decoder, and pretty exactly analogous to an SSL termination point. If you really believe in true end-to-end encryption, then having this webserver in the middle is a bad idea: you should route requests directly to database servers and ensure that at no point does any non-database computer have a decrypted copy of the information that will eventually be rendered to the user. Another thing to think about: if you are heavily concerned that the CDN or reverse proxy service you are using has access to your traffic, realize that they are no different than using a hosted server. We don't even need to invoke "the cloud" for this one: unless I built the machine myself, I have no way of knowing that the server I am renting to run my database server on isn't running under a hypervisor that is allowing dumps of my decrypted memory. A reverse proxy is just another piece of rented infrastructure that someone else is in control of, and I don't see many people (though if you are, that's great, and then I'd be really happy ;P) insisting that everyone build their own computers and host them at their office, to make sure decrypted data never touches a computer owned and operated by someone else. The risk profile is also just so fundamentally different, in no small part due to these differences of "who is in control and how is it implemented". Let's say that CDNetworks is failing to check SSL certificates correctly: how exactly are you going to attack that against my website? I mean, sure: if you are a state-level actor, that is going to be easy, but otherwise, that's going to be extremely rough. An "antivirus" filter that, as described in this alert, is failing to correctly check SSL certificates... well, that opens you up to attack by amateur attackers that are "close to you", such as a stalker or an ex-spouse or a corporate competitor. You can't just claim "well, if I abstract away all of the details, they look the same"... you have to look at why you care and verify that you didn't strip away anything required and relevant.
    - nickik 10y ago
      > The code in your website is likely 99% "transform a URL into SQL, then transform the result into HTML" My live seems more trivil now.
    - JshWright 10y ago
      > If you really believe in true end-to-end encryption, then having this webserver in the middle is a bad idea: you should route requests directly to database servers and ensure that at no point does any non-database computer have a decrypted copy of the information that will eventually be rendered to the user. I can't say I haven't considered abusing PL/pgSQL to that point... ;)
      - _jal 10y ago
        I've actually seen that done - HTML straight out of a stored procedure. Really awesome <TR BGCOLOR=...><TD> stuff. I've mostly recovered from the experience, thanks, although I do have occasional flashbacks.
    - lmm 10y ago
      >An "antivirus" filter ends up implemented as a man-in-the-middle attack between software I wrote and servers I contract and control, wherein protections that were actively put in place are stripped away in order to increase the attack surface. A reverse proxy is something that I actively implemented as part of my infrastructure, and I am contracting for it and paying for it like anything else I rent. Those can equally happen the other way around. You might put an antivirus filter in place on your own network. Your hosting provider might set up their own reverse proxy. > But like, think about it this way: Apache is also an SSL termination point. The code in your website is likely 99% "transform a URL into SQL, then transform the result into HTML": it is just a dumb encoder/decoder, and pretty exactly analogous to an SSL termination point. If you really believe in true end-to-end encryption, then having this webserver in the middle is a bad idea: you should route requests directly to database servers and ensure that at no point does any non-database computer have a decrypted copy of the information that will eventually be rendered to the user. I agree with that - best to have the database and the web server on the same server, communicating via local socket or shared memory. Better still to compile the web layer and the data layer into the same unikernel. Current best practice (e.g. PCI-DSS) seems to accept having an (encrypted) network connection between distinct web and database servers, but only within a single physically secured rack. You're right that if you do the same thing with a load balancer (have it in the same physically secured rack as your web servers, encrypt connections between it and them) then it's no less secure, but I don't think people would want to be using a load balancer within a single rack anyway, since most of the point of a load balancer is availability and you want separate racks for that. Certainly a third-party reverse-proxy is a substantial weakening of security. > I don't see many people (though if you are, that's great, and then I'd be really happy ;P) insisting that everyone build their own computers and host them at their office, to make sure decrypted data never touches a computer owned and operated by someone else. I wouldn't go that far - AIUI PCI-DSS etc. require you have physically secured/access-controlled servers but allow you to rent space/power/cooling from a datacenter operator (though they don't get access to your rack, and possibly have to also be audited themselves). > Let's say that CDNetworks is failing to check SSL certificates correctly: how exactly are you going to attack that against my website? I mean, sure: if you are a state-level actor, that is going to be easy, but otherwise, that's going to be extremely rough. An "antivirus" filter that, as described in this alert, is failing to correctly check SSL certificates... well, that opens you up to attack by amateur attackers that are "close to you", such as a stalker or an ex-spouse or a corporate competitor. It's a different risk profile to be sure. Are you more worried about someone making a large-scale attack against the remote service you were connecting to, or just against the Internet in general? Or someone making a small-scale local attack? I think both can be serious; it's not just state-level actors but also large criminal rings or even political groups that would potentially have the resources to execute an attack against a particular service (e.g. a medical provider doing something politically contentious, or just anything that accepts credit cards / personal information).
  - deleted 10y ago
    [deleted]
  - drdaeman 10y ago
    When running on the same machine, reverse TLS-terminating proxy isn't different from built-in TLS implementation, except for done in "microservice-like" way. Most certainly, there's no difference whatsoever (or almost so, if the reverse proxy is "smart" and has some application logic in it) if your webapp doesn't speak HTTP natively, but talks FastCGI/uWSGI/SCGI protocols instead. Which is quite common. I mean, the actual interface (passing buffer pointers in the local process or passing same data via local UNIX sockets) shouldn't matter much, right? As long as interfaces have similar security properties and all the necessary data is passed around properly. A sufficiently privileged user can introspect both (with ltrace or tcpdump), and insufficiently privileged one can't do anything anyway.
  - jupp0r 10y ago
    Not necessarily. A common pattern for load balancers is terminating TLS and communicating with backends unencrypted/unauthenticated.
deleted 10y ago
[deleted]
micaksica 10y ago
I don't think that's the correct interpretation of this document. A better title would be "Most TLS Proxies Considered Harmful", not TLS interception as a whole. US-CERT is explicitly calling out that many appliances and applications often are weakening security vs. the common TLS clients, by way of not properly validating certificate chains by proxy for the client and otherwise masking SSL/TLS issues from being surfaced to the end user. I believe this is CERT trying to get software companies to get their act together on solving security-detrimental UX/UI issues vs. a blanket statement not to add TLS interception proxies to a corporate network.
- hannob 10y ago
  > I don't think that's the correct interpretation of this document. A better title would be "Most TLS Proxies Considered Harmful", not TLS interception as a whole. This is the usual reaction to this I see everywhere. It doesn't match with reality. Read the complete paper by Halderman et al [1]. It says pretty clearly that all existing products lower the security in one way or another: "Our grading scale focuses on the security of the TLS handshake and does not account for the additional HTTPS validation checks present in many browsers, such as HSTS, HPKP, OneCRL/CRLSets, certificate transparency validation, and OCSP must-staple. None of the products we tested supported these features." TLS Interception that doesn't degrade security is a myth that exists only in some parallel world fantasy. It's also not very surprising. Browser vendors have extremely qualified security teams that are involved in many ways in the development of stronger TLS protections. Nothing like that can be said about any of the vendors of interception products. [1] https://jhalderm.com/pub/papers/interception-ndss17.pdf https://jhalderm.com/pub/papers/interception-ndss17.pdf
  - mathw 10y ago
    Is better TLS interception actually possible? Is it just that it's usually badly implemented, or is it a fundamental weakening? Other than the unavoidable weakening of having another system seeing the unencrypted data, which is inherent in using one of these systems and can never go away.
    - 10y ago
Spooky23 10y ago
No, they haven't. They say pretty plainly that you need to ensure that such products provide correct certificate validation, since the client cannot do so reliably.
- Ajedi32 10y ago
  Well, it does also say: > In general, organizations considering the use of HTTPS inspection should carefully consider the pros and cons of such products before implementing So I think there's at least some element of "don't do this unless you have to" in there. But for the most part yeah, the alert seems to be more about ensuring the interception is done securely than about not doing it at all.
- jamiesonbecker 10y ago
  >> The US government has basically declared "HTTPS/TLS Interception Considered Harmful". > No, they haven't. The original report headline from US-CERT: "HTTPS Interception Weakens TLS Security" That seems pretty unambiguous, regardless of your personal feelings regarding CERT or whether it speaks for the broader US government.
  - Spooky23 10y ago
    Don't analyze a book by its cover!
deleted 10y ago
[deleted]
tptacek 10y ago
US-CERT has very little influence, even in regulated industries. As usual, CERT is saying something formally that has been axiomatic among practitioners for many years. The only conclusion I'd draw from this is about CERT. Obviously: what they're saying is broadly true. I also agree with the other commenters here who are pointing out that this is largely a response to endpoint TLS interception, and that the point is to focus attention on tools that intercept but don't validate certificates --- CERT is not telling every Fortune 500 company and every federal agency to stop deploying BlueCoat boxes. But the more important thing to push back on in your comment is the notion that a proclamation from CERT has any real meaning. It doesn't, even within FedGov IT.

jusob 10y ago

It does not have to be. Done correctly, SSL interception can pass through all the errors to the client: * certificate issues (expiration, domain mismatch, etc.) * OCSP/CRL verification * validation of HPKP header I understand that few vendors may be doing it (I know one which does at least the first 2). Probably the worst offense is choosing the weakest TLS version + cipher to save resources, like using TLS 1.0 because it take less resources to decode/encode than TLS 1.2 + elliptic curve.

userbinator 10y ago
On the other hand, a MITM proxy can also do upgrade "attacks"(?!), communicating with remote servers over the Internet using a stronger protocol than the clients on the LAN behind it support. In fact it seems to me that having the validation happening in one place may potentially be easier to maintain than across many different clients' software.
- hackcasual 10y ago
  My experience with browser companies vs. proxy software companies is that the browser vendors give a much bigger shit about end user security.
  - andrewflnr 10y ago
    You're assuming people on the LAN can upgrade to recent browsers. A lot of them are stuck on Windows XP. I realize that's a "you have bigger problems" type of scenario, but you also have to play the hand you're dealt.
    - Laforet 10y ago
      If an organisation has a large number of networked computers on Windows XP they are going to have more issues than that - having no SNI support is one.
      - 10y ago

evilDagmar 10y ago

TL;DR: If your organization is going to do HTTPS interception, don't screw it up.

jaimex2 10y ago
or make sure your appliance/software vendor isn't screwing up.
angry_octet 10y ago
I think the take away is that your mitm box will eventually screw up, and you will be liable for whatever results, from brokenness (e.g. the 40,000 chromebooks going offline) to complete disclosure of company-breaking information.

synicalx 10y ago

It might weaken TLS, but it also stops the 3000 head of cattle I managed from being able to watch porn (6 incidents) and torrent movies (47 copyright notices). If their internet banking, which they're not supposed to be doing at work, gets compromised then I really couldn't care less.

plugger 10y ago
Instead couldn't you just whitelist the online banking sites and intercept everything else?
- jaimex2 10y ago
  he just said he doesn't care :P
- synicalx 10y ago
  That would defeat the whole point of a proxy, if I can't see what they're doing and report on people who are doing the wrong thing (according to their employment contract), then why bother?
  - mulmen 10y ago
    Because if you block it then they can't do it and you have nothing to report. Everyone wins.
    - jupp0r 10y ago
      That's one way to see it. In my experience, those proxies in schools are a great way to motivate students to self-educate about how the internet works and about online privacy tools like TOR...
      - mulmen 10y ago
        Why is a proxy necessary for a student to self-educate? Are you just saying this is a lesson they learn the hard way once they have been spied on?

jaimex2 10y ago

Reading the title ... no shit.

jupp0r 10y ago
Exactly my idea, but after reading the article I'm severely disappointed that the whole practice isn't declared bad. Instead it only points out that some mitm boxes are even more broken than I though possible and mess up cert validation.
- jaimex2 10y ago
  It's a terrible practice, the article doesn't even go into some other things like how easy is it to actually break into the intercepting software and steal the private cert so you can mitm the network for free! and I'm a dev at one of these companies! We really didn't want to add the feature but we were bleeding customer's who for some reason had to have it. ( Yes, we do validate the certificates ) We decided to bite the bullet when it looked like even Google was "supporting" being mitmed as a customer pointed out. Reading this article... https://support.google.com/a/answer/1668854?hl=en https://support.google.com/a/answer/1668854?hl=en was my biggest "fuck you" moment.

Keverw 10y ago

A while back I remember seeing on HN there was a issue with a certain vendor and ChromeBooks because Chrome used a newer TLS(And the mitm vendor vendor was noticed in advance too, and didn't update their product). I wonder how schools and banks plan to react to this... Apparently financial firms have to record everything their employees do for some regulations. To me, schools doing this sort of thing is wrong. I wouldn't be surprised if the principle would grab people's passwords and login to their accounts even. I know some schools even went as far to demand students hand over their passwords to social media when they report bullying... Which if the school blocks social networks anyways, I don't see how it's a school issue for what happens outside of school... If this sort of thing really needs to be done, at-least people should be warned and aware they are being monitored. If it's for a bank and it's only company equipment everything is being monitored it seems a bit more okay to do if everyone is well aware. "You are only to use work computers for official business." sort of policy.

JumpCrisscross 10y ago
> financial firms have to record everything their employees do for some regulations Phone calls, emails, instant messages and other forms of client contact, yes. Internet browsing history? No. Rest assured, many firms do this. But it's because they decided to, not because of regulation.
- Mandatum 10y ago
  Worked for Big 4. They did not log my phone calls or sniff my TLS traffic.
  - newjersey 10y ago
    Why isn't it enough to do the tracking/audit log in the server side? I can't imagine even the JP Morgan Chase CEO lives under surveillance of JP Morgan Chase Bank 24/7. Just require them to use centralized services and keep audit log of that? Sounds simple enough. I don't get the need to monitor all traffic. Surely the sensitive servers have audit log independent of network?
  - 10y ago

tofflos 10y ago

So how do I test if my workplace is doing a good job of this? The article mentions badssl.com. Do I just click all the red links in the certificate section and verify that my browser is refusing to display the pages?

j_s 10y ago
Just wanted to say thanks for attempting to point the conversation in a useful direction; I hope someone who can help is willing to share the answer.
tgasson 10y ago
You want to do that both on your intercepted LAN and with a direct connection and compare the results. If they both give the same results you can be happy your MitM proxy is at least treating insecure sites equivalently. You still lose the advantage that you browser will tell you why the site is insecure
mleonhard 10y ago
Click the Dashboard link at the top-left. https://badssl.com/dashboard/ https://badssl.com/dashboard/

Animats 10y ago

We need MITM detection in the browser. Yes, it's possible. The crypto bits the host is sending are different from the crypto bits the client is receiving. There are several ways to compare those, despite what the MITM box is doing. Out of band channels, timing, and order of data can be used. I sometimes refer to HTTPS Everywhere as "Security Theater Everywhere". Before the mania for HTTPS, many sites only used HTTPS only for crucial transactions such as logins and credit cards. Those were infrequent enough that they didn't have to go through a CDN. Now, with HTTPS Everywhere, there's no distinction between the stuff that has to be hidden from observers, and the stuff which only needs something like Subresource Integrity to make sure it hasn't been messed with. So now the secure channel over which credit card numbers and logins are passed is exposed at the CDN.

deleted 10y ago
[deleted]
averagewall 10y ago
Then those organizations using MITM boxes will install a patch or special modified browser on their members' computers instead of installing a certificate. What's the difference? They still control the computer and can bypass whatever security it has.
ancarda 10y ago
It's not safe to do only logins over HTTPS because if the initial site is served over HTTP, an attacker could rewrite the URL or inject JavaScript to leak your password. Also, once you're signed in, every request sends a cookie which is also valuable to steal as it's an active session. Every request needs to be encrypted. We haven't made TLS worse by deploying it everywhere.
- 4ad 10y ago
  Logins? 99% of the websites I visit don't have user accounts.
  - icebraining 10y ago

philbert 10y ago

I still find it unfortunately shallow analysis. I'm currently fighting a battle in a company in the middle of rolling out Blue Coat ProxySG. I only became aware of it because it began causing interruptions to our work since none of the development tools get the necessary root cert to validate the certs that the proxy is rewriting. The root cert is only installed into the Windows credentials to make browsers work and it's left up to every client to fix whatever other problems they have. One of the common arguments I've encountered from people is along the lines of "if the company trusts the security people, then I trust the security people". However this ignores the fact that the company has legally enforceable agreements in the form od employment contracts, NDAs, and what-have-you that protect the company against a breach of that trust. But what kind of protection do you have as an individual if, for what ever reason, one of the security people decides to single you out while they have unsupervised access to you social media logins, personal email and credit cards numbers and financial details (and lets face the fact that everyone does all of these things to some degree on a work computer). The answer is that you more than likely have no protection at all (likely not even in workplace law). Even if you suspected someone had inappropriately accessed your personal details, the company almost certainly has an IT policy saying that you shouldn't be using your computer for personal use. You're screwed. However these inspection boxes drastically change the situation to what it was before, because most people outside the IT department won't even know about this drastic change in capability. And likely they try hard not to think about until (if) they discover something has happened. I'm sure everyone here is aware of the abuses of NSA employees spying on their wifes, or ex-es [0]. Such is human nature. Sadly I suspect that all these arguments will do little to change the situation. It seems more likely that the companies who deploy these systems are only going to listen to arguments specifically about how it is increasing work, causing delays of deliveries and affecting project costs. [0] http://www.reuters.com/article/us-usa-surveillance-watchdog-idUSBRE98Q14G20130927 http://www.reuters.com/article/us-usa-surveillance-watchdog-...

rkeene2 10y ago
TLS MITM devices also break required components of TLS, such as TLS client certificates. If you want to have TLS sessions with mutual authentication it can't go through the MITM. A way forward might be to start requiring TLS client certificates for business products. :-) For us, we use TLS client certificate authentication to authenticate almost all our users so any TLS MITM breaks that and is unacceptable. This is really common and tens of millions of people use TLS client certificates this way everyday -- mostly with client certificates on smartcards.

tinus_hn 10y ago

And the answer is going to be custom unspecified encryption inside the encrypted channel. HTTPS mitm proxying is a dumb idea that can't work right but is easy to sell to executives. It will be defeated until we're at the state of a few years ago, where the products that really matter, like banking, have tough security and the rest has less.

seasonalgrit 10y ago

any recommendations on tutorials/guides for better understanding the world of TLS, certificates, and so on? i don't feel like i have a healthy mastery of the ideas discussed in articles like this one.

kbart 10y ago
HTTPS is basically HTTP over TLS, so start with RFCs: - TLS v1.2 (https://tools.ietf.org/html/rfc5246 https://tools.ietf.org/html/rfc5246) - Certificate validation (https://tools.ietf.org/html/rfc6960 https://tools.ietf.org/html/rfc6960) - PKI is a little more tricky, as there's no single standard that defines it, though to get basic idea of what are certificates read RFC 5280 (https://tools.ietf.org/html/rfc5280 https://tools.ietf.org/html/rfc5280). Sure, you can dig as deep as you want, but these resources should give you a good starting point.
- rkeene2 10y ago
  Don't forget RFC 2818 (HTTPS) which Chromium (and thus Chrome) recently broke mandatory features of: https://bugs.chromium.org/p/chromium/issues/detail?id=308330#c39 https://bugs.chromium.org/p/chromium/issues/detail?id=308330...
  - NetStrikeForce 10y ago
    Is this a remake of that "embrace, extend, ..." movie of the 90s or this time it's cool?
    - rkeene2 10y ago
      It's really very similar. This is a case where the Chromium developers want to do something so they are going to do it, standards be dammed...
nikcub 10y ago
Ivan Ristic's "Bulletproof SSL and TLS"[0] [0] https://www.feistyduck.com/books/bulletproof-ssl-and-tls/reviewerKit.html https://www.feistyduck.com/books/bulletproof-ssl-and-tls/rev...

omribahumi 10y ago

This is what squid has been trying to solve with their server first bumping[0] and peek-and-splice[1] [0] http://wiki.squid-cache.org/Features/BumpSslServerFirst http://wiki.squid-cache.org/Features/BumpSslServerFirst [1] http://wiki.squid-cache.org/Features/SslPeekAndSplice http://wiki.squid-cache.org/Features/SslPeekAndSplice edit: reference peek-and-slice too

Khaine 10y ago

One important reason to do ssl inspection is to detect the exfiltration of corporate data, particularly if your organisation is likely to be attacked by governments or sophisticated crime gangs. I know my organisation do SSL inspection, but they have whitelisted common personal websites like internet banking to protect the privacy of staff. As far as I'm concerned, if your risk profile requires you to do SSL inspection then this is the right way to do it.