5 ms·

Direct TLS can speed up your connections

91 points by tanelpoder 1y ago

ronsor 1y ago
> When the client sends a hello, the firewall says “that looks like a TLS hello”, and then waits for the server’s response. It inspects the certificate and then applies any rules. This kind of stuff is exactly why TLS 1.3 encrypts everything now.
- kevvok 1y ago
  Years of middleboxes tampering with TCP and barfing on anything they didn’t recognize is why QUIC encrypts everything as well
  - silverwind 1y ago
    > QUIC encrypts everything QUIC is built on top of TLS 1.3 where client hello encryption is not mandatory, so this is not strictly true.
  - OptionOfT 1y ago
    Many companies block outgoing traffic to 443/UDP, or block UDP completely.
- rkagerer 1y ago
  In TLS 1.3, the server’s certificate is encrypted, which means the firewall can’t do this anymore. ... The firewall will open a second connection and do a TLS 1.2 handshake to retrieve the certificate in plaintext. If the rules allow the connection, the firewall then allow the connection to proceed. Sounds like a cat & mouse game. How long before we get servers that respond with different certificates based on who"s asking or other cues from the connection, in efforts to bypass the firewall restrictions.
  - tialaramex 1y ago
    Sure, this already happens today as in practice pre-standard ECH is deployed, so client says "Hi I'm calling some.boring.example [encrypted: Actually clown-porn.example]" and this useless middlebox cannot decrypt the inner message so it retrieves the some.boring.example certificate but the client's encrypted connection reaches clown-porn.example which may have a different certificate. For many years, long before TLS 1.3 the RFC explains how to do this properly, obviating the problem, but that's very expensive because you need to TLS proxy every single connection, you can impose oversight by - literally - adding an actual oversight layer, which your users can also see you did. The popular middlebox products say they're doing two things, they're cheaper (maybe you buy their $100k product instead of a $100M solution) and they're less "intrusive" (ie you needn't tell your users that you're spying on them) In reality they're ineffective, which is why the RFC says not to do this - but they can either outright lie or hide this fact in an asterisked disclaimer somewhere, and most of their customers don't care whether it actually works, they want to tick a box.
    - fragmede 1y ago
      CISO is such a thankless job.
  - FuriouslyAdrift 1y ago
    In the infosec space, I am seeing rumblings of moving to compliant devices where the network tap is at the OS level and/or application level using software agents, AV hooks, or fake virtual network devices.
- tialaramex 1y ago
  Technically TLS 1.3 doesn't encrypt the Client Hello, and indeed the first stanza of the Server Hello isn't encrypted either, although it deliberately doesn't say anything you could use as a third party to make decisions and the rest of a TLS 1.3 Server Hello is encrypted. The in-progress Encrypted Client Hello (was Encrypted Server Name Indication hence esni in the name of the document) fixes that and you probably have software which uses it though the work to tie up all the loose ends up and publish a document might take some time yet. Because the middlebox vendors are _so_ incompetent not only does TLS 1.3 need to work around their nonsense, thus proving that it was never useful security (an attacker could always have just done what TLS 1.3 does and it would have bypassed this worthless garbage) some of them screwed up badly enough that the anti-downgrade feature trips, to their credit Google refused to ship a permanent workaround for this, the workaround they shipped in Chrome sunset in about a year & required an explicit key setting, so basically "I acknowledge that I have defective middleboxes and must remove or upgrade them soon" by the local administrators. But yes, it's noticeable that it was much easier to sell some engineers on "Thanks to TLS 1.3 now this stupid middlebox product won't be able to make your service slower" than say, "Thanks to TLS 1.3 now this stupid middlebox product won't report to the government if you read a Wikipedia article which contradicts its dogma".
  - yardstick 1y ago
    Encrypted SNI was never about hiding which specific articles in Wikipedia you are looking at. It was about hiding the fact you are looking at Wikipedia at all. There are valid use cases for TLS middleboxes. Anyone having to secure a networks outbound access to only essential services has run into the “I have to allow all of AWS/GCP/Azure/CloudFlare/…” for some critical tool to work. Options are: - Allow all out (nope, not secure) - Allow all to the cloud providers IP range (still terrible) - Filter on SNI but don’t inspect the payload (better than no security, and doesn’t require plaintext access). - Full MITM TLS proxy (performance bottleneck, and now we have plaintext access to all your data, which we really don’t want and didn’t previously require to do the filtering). - Try convince the third party service to run on a handful of static IPs that aren’t behind a global load balancer with access to the rest of the cloud providers customer domains. (Yeah right) See: Hospitals. Payment Networks. IoT networks.
- hypeatei 1y ago
  Encrypted client hello can be stripped easily by modern firewalls[0]. I know because my employers network enables this functionality and it forces a downgrade so that it can see what you're trying to access. [0] https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-block-TLS-1-3-Encrypted-Client-Hello-ECH-in/ta-p/328324 https://community.fortinet.com/t5/FortiGate/Technical-Tip-Ho...
  - tialaramex 1y ago
    "easily" in the sense that in the proxy mode, where the Fortigate gets to make any arbitrary change it wants, it can indeed make this change too. I guess "Use the American spelling for words" would be a slightly more invasive change than "strip ECH" in this context, but it's not that different. This is a full proxy, your client has (likely because it was configured by corporate IT) agreed that all your communication will run through a proxy, probably somebody will swear they're definitely not logging everything you do† but obviously it could do absolutely anything. † Actually if you work in trading they might tell you it does log everything, the regulators for these industries want records of everything because they don't trust traders as far as they can throw them, and having met a few traders I know why - they tend to make Boris Johnson look like Forest Gump.
- josephcsible 1y ago
  What it does isn't good enough, since it basically leaves itself open to downgrade attacks on purpose. I wish the world would decide on a flag day after which if both endpoints support TLS ECH, the connection will hard-fail instead of retrying without it if a middlebox breaks it.
  - stavros 1y ago
    Wouldn't the middlebox just lie to each party that the other doesn't support ECH?
    - ronsor 1y ago
      TLS 1.3 prevents tampering with the handshake. The transcripts won't match and the connection will be aborted.
nemoniac 1y ago
Direct TLS can speed up your postgreSQL connection
- deleted 1y ago
  [deleted]
- aflukasz 1y ago
  And mostly if you are behind CISCO firewall during TLS Server Identity Discovery or some equivalent setup. 3 seconds mentioned in the article were coming mostly from that. From the text itself it's not clear how much gains come from sslnegotiation=direct itself (if we assume no other factors like those present in this case).
  - aflukasz 1y ago
    For those interested - I've checked and observed a difference of 0.2ms on average across 1000 connection attempts on localhost.
    - yencabulator 1y ago
      Localhost is the least interesting place to measure a roundtrip delay.
      - aflukasz 1y ago
        Fair point, but it's a place with some useful properties. All the rest of the effect will depend on the specifics of your network. Observing the impact on localhost shows scale of the effect that does not come from the network (more or less) and puts a lower bound on its size one can expect in more realistic conditions.
  - yencabulator 1y ago
    Even without Cisco meddling, sslnegotiation=direct saves a roundtrip.
- prussian 1y ago
smallpipe 1y ago
"Having a dumb firewall can slow down your connections"
mrb 1y ago
Yeah, my take away is more "yet another example of a lousy firewall causing dumb issues".
- mixermachine 1y ago
  yes, that should be the answer. The firewall can't analyse the connection thus cancels the connection attempt.
sylware 1y ago
Is there a conflict or significant redundancy between QUIC TLS and DTLS?
- tialaramex 1y ago
  There will be (is? under standards development?) a DTLS replacement built on QUIC but today these are orthogonal technologies. QUIC makes you a TCP-but-encrypted while DTLS is more TLS-but-for-UDP In a world with no legacy technology rusted in place, QUIC would be a new IP protocol, like TCP and UDP, but there's stuff rusted in place which can barely manage UDP and would not understand how there can possibly be a new protocol even though the entire network was designed to allow that, so that's why QUIC is spelled as UDP data.
  - immibis 1y ago
    It's a shame we didn't put ports at the IP layer. With IPv6, on most networks, you can use as many addresses as you like, but it's inefficient because each one has to be individually resolved.
    - sylware 1y ago
      And if ISPs properly let enough bits to client in order to randomize IPv6 addresses (a reasonable amount per domestic line) for their internet client-server sessions in the IPv6 prefixes they provide, it will make those toxic 'global internet scanners' near useless. I can do that with my domestic internet line, and that has been the case for more than a decade. Now, my sight is on mobile IPv6 internet: my current ISP is actually generating that randomized IPv6 address (unless it is what is actually done by my modem and I don't see it). In theory, the upper 64bits IPv6 should be enough, but mobile ISP could give a bit less to client: a 96bits prefix, with 32bits to choose addresses from. The real hard part is to make that prefix uniq to each mobile line, that globally namely what they actually do with phone number roaming. As I said, it will make those toxic "global IP scanners" near useless, and even recording the IPv6 won't matter much with good randomization, that in the case of classic client-server (the web for instance). For p2p contact-based communication, like audio/video IP phones, the client OS will have to choose a stable IPv6 addresses in order for the contacts to be able to use those very addresses for communication. The nice part, could have a few of them (for filters), and it is ez to move to new addresses, the tough part is to tell selected contacts of the new addresses upon phasing out some addresses.
deleted 1y ago
[deleted]

91 points by tanelpoder 1y ago

> When the client sends a hello, the firewall says “that looks like a TLS hello”, and then waits for the server’s response. It inspects the certificate and then applies any rules. This kind of stuff is exactly why TLS 1.3 encrypts everything now.

kevvok 1y ago
Years of middleboxes tampering with TCP and barfing on anything they didn’t recognize is why QUIC encrypts everything as well
- silverwind 1y ago
  > QUIC encrypts everything QUIC is built on top of TLS 1.3 where client hello encryption is not mandatory, so this is not strictly true.
- OptionOfT 1y ago
  Many companies block outgoing traffic to 443/UDP, or block UDP completely.
rkagerer 1y ago
In TLS 1.3, the server’s certificate is encrypted, which means the firewall can’t do this anymore. ... The firewall will open a second connection and do a TLS 1.2 handshake to retrieve the certificate in plaintext. If the rules allow the connection, the firewall then allow the connection to proceed. Sounds like a cat & mouse game. How long before we get servers that respond with different certificates based on who"s asking or other cues from the connection, in efforts to bypass the firewall restrictions.
- tialaramex 1y ago
  Sure, this already happens today as in practice pre-standard ECH is deployed, so client says "Hi I'm calling some.boring.example [encrypted: Actually clown-porn.example]" and this useless middlebox cannot decrypt the inner message so it retrieves the some.boring.example certificate but the client's encrypted connection reaches clown-porn.example which may have a different certificate. For many years, long before TLS 1.3 the RFC explains how to do this properly, obviating the problem, but that's very expensive because you need to TLS proxy every single connection, you can impose oversight by - literally - adding an actual oversight layer, which your users can also see you did. The popular middlebox products say they're doing two things, they're cheaper (maybe you buy their $100k product instead of a $100M solution) and they're less "intrusive" (ie you needn't tell your users that you're spying on them) In reality they're ineffective, which is why the RFC says not to do this - but they can either outright lie or hide this fact in an asterisked disclaimer somewhere, and most of their customers don't care whether it actually works, they want to tick a box.
  - fragmede 1y ago
    CISO is such a thankless job.
- FuriouslyAdrift 1y ago
  In the infosec space, I am seeing rumblings of moving to compliant devices where the network tap is at the OS level and/or application level using software agents, AV hooks, or fake virtual network devices.
tialaramex 1y ago
Technically TLS 1.3 doesn't encrypt the Client Hello, and indeed the first stanza of the Server Hello isn't encrypted either, although it deliberately doesn't say anything you could use as a third party to make decisions and the rest of a TLS 1.3 Server Hello is encrypted. The in-progress Encrypted Client Hello (was Encrypted Server Name Indication hence esni in the name of the document) fixes that and you probably have software which uses it though the work to tie up all the loose ends up and publish a document might take some time yet. Because the middlebox vendors are _so_ incompetent not only does TLS 1.3 need to work around their nonsense, thus proving that it was never useful security (an attacker could always have just done what TLS 1.3 does and it would have bypassed this worthless garbage) some of them screwed up badly enough that the anti-downgrade feature trips, to their credit Google refused to ship a permanent workaround for this, the workaround they shipped in Chrome sunset in about a year & required an explicit key setting, so basically "I acknowledge that I have defective middleboxes and must remove or upgrade them soon" by the local administrators. But yes, it's noticeable that it was much easier to sell some engineers on "Thanks to TLS 1.3 now this stupid middlebox product won't be able to make your service slower" than say, "Thanks to TLS 1.3 now this stupid middlebox product won't report to the government if you read a Wikipedia article which contradicts its dogma".
- yardstick 1y ago
  Encrypted SNI was never about hiding which specific articles in Wikipedia you are looking at. It was about hiding the fact you are looking at Wikipedia at all. There are valid use cases for TLS middleboxes. Anyone having to secure a networks outbound access to only essential services has run into the “I have to allow all of AWS/GCP/Azure/CloudFlare/…” for some critical tool to work. Options are: - Allow all out (nope, not secure) - Allow all to the cloud providers IP range (still terrible) - Filter on SNI but don’t inspect the payload (better than no security, and doesn’t require plaintext access). - Full MITM TLS proxy (performance bottleneck, and now we have plaintext access to all your data, which we really don’t want and didn’t previously require to do the filtering). - Try convince the third party service to run on a handful of static IPs that aren’t behind a global load balancer with access to the rest of the cloud providers customer domains. (Yeah right) See: Hospitals. Payment Networks. IoT networks.
hypeatei 1y ago
Encrypted client hello can be stripped easily by modern firewalls[0]. I know because my employers network enables this functionality and it forces a downgrade so that it can see what you're trying to access. [0] https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-block-TLS-1-3-Encrypted-Client-Hello-ECH-in/ta-p/328324 https://community.fortinet.com/t5/FortiGate/Technical-Tip-Ho...
- tialaramex 1y ago
  "easily" in the sense that in the proxy mode, where the Fortigate gets to make any arbitrary change it wants, it can indeed make this change too. I guess "Use the American spelling for words" would be a slightly more invasive change than "strip ECH" in this context, but it's not that different. This is a full proxy, your client has (likely because it was configured by corporate IT) agreed that all your communication will run through a proxy, probably somebody will swear they're definitely not logging everything you do† but obviously it could do absolutely anything. † Actually if you work in trading they might tell you it does log everything, the regulators for these industries want records of everything because they don't trust traders as far as they can throw them, and having met a few traders I know why - they tend to make Boris Johnson look like Forest Gump.
josephcsible 1y ago
What it does isn't good enough, since it basically leaves itself open to downgrade attacks on purpose. I wish the world would decide on a flag day after which if both endpoints support TLS ECH, the connection will hard-fail instead of retrying without it if a middlebox breaks it.
- stavros 1y ago
  Wouldn't the middlebox just lie to each party that the other doesn't support ECH?
  - ronsor 1y ago
    TLS 1.3 prevents tampering with the handshake. The transcripts won't match and the connection will be aborted.

nemoniac 1y ago

Direct TLS can speed up your postgreSQL connection

deleted 1y ago
[deleted]
aflukasz 1y ago
And mostly if you are behind CISCO firewall during TLS Server Identity Discovery or some equivalent setup. 3 seconds mentioned in the article were coming mostly from that. From the text itself it's not clear how much gains come from sslnegotiation=direct itself (if we assume no other factors like those present in this case).
- aflukasz 1y ago
  For those interested - I've checked and observed a difference of 0.2ms on average across 1000 connection attempts on localhost.
  - yencabulator 1y ago
    Localhost is the least interesting place to measure a roundtrip delay.
    - aflukasz 1y ago
      Fair point, but it's a place with some useful properties. All the rest of the effect will depend on the specifics of your network. Observing the impact on localhost shows scale of the effect that does not come from the network (more or less) and puts a lower bound on its size one can expect in more realistic conditions.
- yencabulator 1y ago
  Even without Cisco meddling, sslnegotiation=direct saves a roundtrip.
prussian 1y ago

smallpipe 1y ago

"Having a dumb firewall can slow down your connections"

mrb 1y ago

Yeah, my take away is more "yet another example of a lousy firewall causing dumb issues".

mixermachine 1y ago
yes, that should be the answer. The firewall can't analyse the connection thus cancels the connection attempt.

sylware 1y ago

Is there a conflict or significant redundancy between QUIC TLS and DTLS?

tialaramex 1y ago
There will be (is? under standards development?) a DTLS replacement built on QUIC but today these are orthogonal technologies. QUIC makes you a TCP-but-encrypted while DTLS is more TLS-but-for-UDP In a world with no legacy technology rusted in place, QUIC would be a new IP protocol, like TCP and UDP, but there's stuff rusted in place which can barely manage UDP and would not understand how there can possibly be a new protocol even though the entire network was designed to allow that, so that's why QUIC is spelled as UDP data.
- immibis 1y ago
  It's a shame we didn't put ports at the IP layer. With IPv6, on most networks, you can use as many addresses as you like, but it's inefficient because each one has to be individually resolved.
  - sylware 1y ago
    And if ISPs properly let enough bits to client in order to randomize IPv6 addresses (a reasonable amount per domestic line) for their internet client-server sessions in the IPv6 prefixes they provide, it will make those toxic 'global internet scanners' near useless. I can do that with my domestic internet line, and that has been the case for more than a decade. Now, my sight is on mobile IPv6 internet: my current ISP is actually generating that randomized IPv6 address (unless it is what is actually done by my modem and I don't see it). In theory, the upper 64bits IPv6 should be enough, but mobile ISP could give a bit less to client: a 96bits prefix, with 32bits to choose addresses from. The real hard part is to make that prefix uniq to each mobile line, that globally namely what they actually do with phone number roaming. As I said, it will make those toxic "global IP scanners" near useless, and even recording the IPv6 won't matter much with good randomization, that in the case of classic client-server (the web for instance). For p2p contact-based communication, like audio/video IP phones, the client OS will have to choose a stable IPv6 addresses in order for the contacts to be able to use those very addresses for communication. The nice part, could have a few of them (for filters), and it is ez to move to new addresses, the tough part is to tell selected contacts of the new addresses upon phasing out some addresses.

deleted 1y ago

[deleted]