3 ms·

One issue is that there’s no “one true list of intermediates” for a server to include in the handshake, as it’s not possible to always know what roots a client

One issue is that there’s no “one true list of intermediates” for a server to include in the handshake, as it’s not possible to always know what roots a client trusts. The client authors (of browsers) tend to care about making their software easy to use and avoid errors. And there’s a pretty small set of clients. TLS servers on the other hand often just serve whatever you put in a config file. A chain that works for them on an up to date Firefox may not work for an older android phone, but the server administrator may not be aware of the trade off they’ve made. My personal opinion is that the tools for debugging for server authors aren’t good enough yet. I’m one of the authors of the certigo tool mentioned in the article, which was written out of frustration from a lack of other tools. Even then, I think it’s not good enough and I hope this class of issue is one we can illustrate better. SSL Labs probably does the best here.

hamburglar 4y ago
> One issue is that there’s no “one true list of intermediates” for a server to include in the handshake This doesn't make sense. Given a cert that was issued from an intermediate, there is exactly one intermediate that issued it, and that intermediate must be in the handshake. If that intermediate was issued from another intermediate, that next intermediate must also be in the handshake. And so on. There's no ambiguity here. Your server is required to send a fully-formed cert chain whose final cert is signed by one of the roots the client trusts. It's entirely possible to run a PKI where the set of intermediates is known by nobody and is unbounded and ephemeral, so long as everyone who is given a cert is given the intermediate chain that issued it. The burden is on the bearer of the cert to include it in the handshake. All problems here have arisen from a misunderstanding that causes people to think the intermediates should be well-known or obtainable publicly. That is simply not true. > it’s not possible to always know what roots a client trusts It doesn't matter. You send the chain that issued your cert, not what you expect the client to trust. If the client trusts it, you're good. If not, you're not.
- hamburglar 4y ago
  TFA complains about browser intermediate cache behavior but doesn't seem to realize that if you're relying on browser intermediate caching at all, your server is misconfigured. The lesson should be "fix your server", not "those darn browsers."
  - jomar 4y ago
    TFA is a war story about how infelicities such as caching in tools and “those darn browsers” contributed to the identification of the cause of this issue taking longer than it should have. Once the missing intermediate certificate issue had been identified, the fix (to the server) was indeed clear. But it's not stated whether the author is in charge of the server in question (it's only mentioned as “couldn't connect to a particular HTTPS site”); I suspect this was some external web site not under their control.
- toast0 4y ago
  Your end entity cert was signed by some specific intermediate CA yes. And that intermediate is often (always?) only signed by one CA. But that CA may have signed certificates from multiple other CAs (generally this is known as cross signing). If your client doesn't trust the CA that signed the intermediate that signed your cert, you'll need a cross signed CA cert that works for your clients, but sometimes it's not possible to make a single chain that supports all the clients you want. And it's difficult to differentiate (and even if you can figure out how to differentiate, you're probably going to be patching your TLS server to do it, because it's not widely supported). When you consider poor client behavior, you get situations like client A doesn't have your CA, but does have another you can cross chain to; client B has your CA and not that other one and has poor verification logic, so if you include the cross chain, verification will fail; you can't make both clients work with the same chain (unless you switch to a different CA). My personal favorite though is when the client helpfully supplies a cached cross chain, but the certificate expired, so it fails validation... even though you didn't send that cert, and you have a path it would validate if it didn't have the cached certificate. I'm exceedingly glad I don't have to deal with this anymore. But at least TLS1.3 changed the spec to require(i think) clients to handle out of order or extraneous certs which strongly suggests clients should try all possible paths and pass the validation if any are valid. Still no way to present multiple end entity certificates though (end entity must be the first certificate, you couldn't send two full chains)
  - hamburglar 4y ago
    The issue from TFA was a simple matter of not including an intermediate at all, not confusion about which intermediate you should include when cross-signing (the article says the intermediate wasn't included, and there is no mention of cross-signing at all). Either way, the proper solution is "include all the intermediates," which applies to cross-signing scenarios. The client will find the path from leaf to any root it trusts. If you had some kind of crazy number of alternate paths for a given cert, this might become onerous, but in reality, the number of alternate paths is basically always either zero or one.
    - cipherboy 4y ago
NovemberWhiskey 4y ago
>One issue is that there’s no “one true list of intermediates” for a server to include in the handshake The one true list of intermediates is "the intermediates between the leaf certificate and the root". Cross-signing is a corner case. Technically, TLS 1.2 cannot support cross-signing due to the requirement that each certificate in the chain certify the immediately preceding one (which is obviously not possible with cross-signing, because the cross-signing intermediates break the chain into two paths). Practically speaking, it works anyway (and TLS 1.3 fixes the problem with the specification) and you should send any intermediates you have and expect the client to construct a chain to a root that it trusts. It's absolutely not that hard, but my experience is there's a lot of cargo cult engineering around TLS configuration, as well as a degree of deploying-it-until-it-works-for-me. My personal low was experiencing issues with a server that, upon investigations, was sending 22 certificates including all of the current- and next-generation CAs roots for our internal PKI as well as all of the issuing and intermediate CAs. The TLS handshake was about 200KB on the wire and would fail for some implementations that gave up trying to put together a trusted path.

hamburglar 4y ago

> One issue is that there’s no “one true list of intermediates” for a server to include in the handshake This doesn't make sense. Given a cert that was issued from an intermediate, there is exactly one intermediate that issued it, and that intermediate must be in the handshake. If that intermediate was issued from another intermediate, that next intermediate must also be in the handshake. And so on. There's no ambiguity here. Your server is required to send a fully-formed cert chain whose final cert is signed by one of the roots the client trusts. It's entirely possible to run a PKI where the set of intermediates is known by nobody and is unbounded and ephemeral, so long as everyone who is given a cert is given the intermediate chain that issued it. The burden is on the bearer of the cert to include it in the handshake. All problems here have arisen from a misunderstanding that causes people to think the intermediates should be well-known or obtainable publicly. That is simply not true. > it’s not possible to always know what roots a client trusts It doesn't matter. You send the chain that issued your cert, not what you expect the client to trust. If the client trusts it, you're good. If not, you're not.

hamburglar 4y ago
TFA complains about browser intermediate cache behavior but doesn't seem to realize that if you're relying on browser intermediate caching at all, your server is misconfigured. The lesson should be "fix your server", not "those darn browsers."
- jomar 4y ago
  TFA is a war story about how infelicities such as caching in tools and “those darn browsers” contributed to the identification of the cause of this issue taking longer than it should have. Once the missing intermediate certificate issue had been identified, the fix (to the server) was indeed clear. But it's not stated whether the author is in charge of the server in question (it's only mentioned as “couldn't connect to a particular HTTPS site”); I suspect this was some external web site not under their control.
toast0 4y ago
Your end entity cert was signed by some specific intermediate CA yes. And that intermediate is often (always?) only signed by one CA. But that CA may have signed certificates from multiple other CAs (generally this is known as cross signing). If your client doesn't trust the CA that signed the intermediate that signed your cert, you'll need a cross signed CA cert that works for your clients, but sometimes it's not possible to make a single chain that supports all the clients you want. And it's difficult to differentiate (and even if you can figure out how to differentiate, you're probably going to be patching your TLS server to do it, because it's not widely supported). When you consider poor client behavior, you get situations like client A doesn't have your CA, but does have another you can cross chain to; client B has your CA and not that other one and has poor verification logic, so if you include the cross chain, verification will fail; you can't make both clients work with the same chain (unless you switch to a different CA). My personal favorite though is when the client helpfully supplies a cached cross chain, but the certificate expired, so it fails validation... even though you didn't send that cert, and you have a path it would validate if it didn't have the cached certificate. I'm exceedingly glad I don't have to deal with this anymore. But at least TLS1.3 changed the spec to require(i think) clients to handle out of order or extraneous certs which strongly suggests clients should try all possible paths and pass the validation if any are valid. Still no way to present multiple end entity certificates though (end entity must be the first certificate, you couldn't send two full chains)
- hamburglar 4y ago
  The issue from TFA was a simple matter of not including an intermediate at all, not confusion about which intermediate you should include when cross-signing (the article says the intermediate wasn't included, and there is no mention of cross-signing at all). Either way, the proper solution is "include all the intermediates," which applies to cross-signing scenarios. The client will find the path from leaf to any root it trusts. If you had some kind of crazy number of alternate paths for a given cert, this might become onerous, but in reality, the number of alternate paths is basically always either zero or one.
  - cipherboy 4y ago

NovemberWhiskey 4y ago

>One issue is that there’s no “one true list of intermediates” for a server to include in the handshake The one true list of intermediates is "the intermediates between the leaf certificate and the root". Cross-signing is a corner case. Technically, TLS 1.2 cannot support cross-signing due to the requirement that each certificate in the chain certify the immediately preceding one (which is obviously not possible with cross-signing, because the cross-signing intermediates break the chain into two paths). Practically speaking, it works anyway (and TLS 1.3 fixes the problem with the specification) and you should send any intermediates you have and expect the client to construct a chain to a root that it trusts. It's absolutely not that hard, but my experience is there's a lot of cargo cult engineering around TLS configuration, as well as a degree of deploying-it-until-it-works-for-me. My personal low was experiencing issues with a server that, upon investigations, was sending 22 certificates including all of the current- and next-generation CAs roots for our internal PKI as well as all of the issuing and intermediate CAs. The TLS handshake was about 200KB on the wire and would fail for some implementations that gave up trying to put together a trusted path.