9 ms·
We build X.509 chains so you don't have to
- deleted 3y ago[deleted]
- caeruleus 3y agoCongratulations to the authors, this was a feature that was dearly missing from pyca/cryptography. It took a long time to get right. For the history: https://github.com/pyca/cryptography/issues/2381 https://github.com/pyca/cryptography/issues/2381
- reaperhulk 3y agoThanks for posting! I had forgotten just how long a road this was, but I'm very pleased that the requirements we talked about years ago have been met. :) That's not to say this is done! There's a set of features we still want to add and we'd like to gain some confidence in the APIs as structured before we mark them as stabilized.
- caeruleus 3y agoLooking forward! Thanks for maintaining such an awesome library btw, most of my internal PKI (X.509 and OpenSSH) is managed with it. It's a bliss to work with, even more so compared to other crypto libs. :)
- asylteltine 3y ago[flagged]
- efitz 3y agoYes. Python was designed from the beginning to be compatible with C; it can make calls directly into C modules. There is an entire portion of the Python ecosystem dedicated to C interoperability. Many of the most popular packages are written in C (e.g. numpy, pandas, etc.) and many other packages have the most performance-sensitive code written in C (scipy et. al.) [ed] s/pansas/pandas/g
- asylteltine 3y ago[flagged]
- Etheryte 3y agoA considerable number of common packages in the Python ecosystem are backed at least in part by another language. This is not a shortcoming, it's a feature. Give end users flexibility while figuring out performance and other beasts internally.
- CogitoCogito 3y ago> Is it really “Python” if the underlying stuff is written in another language? Given that "python" is basically synonymous with "cpython" and cpython has always been implemented in C, I guess the answer to your question is "yes".
- Zambyte 3y agoA language is distinct from its implementation language. Else everything is machine code.
- CogitoCogito 3y agoI never argued otherwise. But how is using a package which is implemented in another language not "python"? How is "list(range(10))" python whereas "numpy.arange(10)" is not?
- dang 3y agoWe've asked you more than once to stop posting unsubstantive comments and flamebait. You've continued to do it, for example in the parent comment, https://news.ycombinator.com/item?id=39108095 https://news.ycombinator.com/item?id=39108095, and https://news.ycombinator.com/item?id=39103476 https://news.ycombinator.com/item?id=39103476. You've even posted outright slurs like https://news.ycombinator.com/item?id=38832516 https://news.ycombinator.com/item?id=38832516, which is seriously not ok. Since you've continued to break HN's rules like this, I've banned the account. If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html.
- nindalf 3y agoHoly fucking shit. I've never seen anything like that last comment on HN before.
- dang 3y agoThat's good, because they certainly get posted. Such things are inevitable on any sufficiently large public forum. If you haven't seen them, it must be because they're mostly getting flagged and/or killed by users and/or moderators, which is the desired outcome.
- EGreg 3y agoDo we have this in PHP? Javascript ?
- tob_scott_a 3y agoI think getting it into cURL would be more impactful, since that's already tapped into by many OSS projects.
- hlieberman 3y agoI'm curious -- what was the inputs to the decision to redo the logic from scratch, rather than wrap something like the `webpki` crate?
- woodruffw 3y agoThis is a great question! The answer is twofold: 1. Compatibility: PyCA Cryptography has strict compatibility requirements, including being able to build with older versions of Rust that crates like webpki and rustls-webpki may not be interested in supporting. 2. Generality: PyCA Cryptography is a general purpose cryptographic toolbox, and the plan for X.509 validation is not intrinsically linked to just the Web PKI profiles. In the future, they may wish to make their APIs more flexible than a strictly CABF validator would require, meaning that they would either need to carry patches against `webpki` or get such flexibility upstreamed (to the detriment of the crate's single purpose). (I am not a maintainer of Cryptography; this is my understanding of the reasons as one of the implementors of this feature.)
- ivanr 3y agoGenuine question: does the `webpki` crate provide a good path building implementation?
- tob_scott_a 3y ago> Carcinize existing C and C++ X.509 users. This could be game-changing for a lot of open source software. I spent years avoiding X.509 (and ASN.1, for that matter) in my designs because every time someone I trust poked it, a remotely exploitable bug fell out. Most often, it was a Denial of Service issue rather than Remote Code Execution. Moving to Rust would demonstrably improve the security of the entire Internet. You might be tempted to ask, "What about BouncyCastle?" (or similar queries). Sure, you're not overwriting the EIP in most Java X.509 bugs, but check the release notes for X.509 and ASN.1 mentions: https://www.bouncycastle.org/releasenotes.html https://www.bouncycastle.org/releasenotes.html When I worked for Amazon, we disclosed a few X.509-related vulnerabilities to projects that we almost found by accident.
- otabdeveloper4 3y ago> Moving to Rust would ... ... do absolutely nothing to fix denial of service attacks.
- tob_scott_a 3y agoBy itself? No. The other details covered in the blog post, however, would absolutely do something to fix denial of service attacks. To wit: x509-limbo
- woodruffw 3y agoI don't think this is true. Rust cannot prevent all possible forms of denial of service, but there are plenty of underlying DoS causes that Rust either outright eliminates (such as memory corruption without further control) or mitigates through stronger types. A recent example of this is CVE-2024-0567 in GnuTLS: an invariant that otherwise would likely have been noticed at the type level is instead checked with an assert, leading to a remotely trigger-able DoS.
- saurik 3y agoNor the other myriad of logic and parsing bugs that led to incorrect behavior (more than just denial of service) in the Java library that was somehow not as good as Rust :/.
- e12e 3y agoFantastic! Curious about revocations though - the post doesn't mention it?
- woodruffw 3y agoThanks for calling this out. Revocations are not currently supported by the public API; we made a decision early in the MVP planning process to exclude them for complexity reasons. (This is consistent with Go's crypto/x509, which does not do revocation checking as part of the `Verify` API.)
- colechristensen 3y agoNot planned for MVP or not planned ever?
- woodruffw 3y agoNot planned for the MVP, but not necessarily excluded in the future. I think it'd require a substantial design period, but that's my impression as an external contributor.
- ivanr 3y agoDo you think that it's possible to implement a good path building algorithm without support for revocation? For example, without revocation checking you may select a chain with a revoked certificate in it, only to discover that later. Then what? :) On the other hand, if you check the revocation status as part of the process, you can pivot to another certificate.
- woodruffw 3y agoI think so -- I would consider crypto/x509's implementation good, and they similarly don't support revocation. More bluntly: it's not clear that revocation really works at all in the Web PKI; I've heard it described more as "homeopathic" than an actual security mechanism. What makes a path building implementation good is (IMO) its adherence to the rules Ryan Sleevi enumerates[1]: as long as you conceptualize the path building process as a dynamic search problem rather than a static one with a "single" result, any implementation you build will probably have the right "primitives" for future constraints (like revocations). [1]: https://medium.com/@sleevi_/path-building-vs-path-verifying-the-chain-of-pain-9fbab861d7d6 https://medium.com/@sleevi_/path-building-vs-path-verifying-...
- ak217 3y agoThis is an absolute game changer for the ability to write secure PKI-based software in Python. Before now, you had to use pyOpenSSL (ignoring its warning to not use it for anything other than an SSL connection) and avoid the many footguns associated with building the certificate chain on your own, or leave Python entirely and talk to a separate Rust or Go process. Thanks to Trail of Bits for pulling this off.
- teddyh 3y agoNote: If your application does not absolutely need it, there are alternatives to X.509 certificates in TLS, like raw public keys (RFC 7250).
- thadt 3y agoWhile the contribution to Python Cryptography is incredibly awesome, I'd suggest that perhaps the lede is slightly buried. Having a large suite of well documented test vectors, including both positive and negative examples, raises the bar for the entire ecosystem. Go and OpenSSL are mentioned, but any other environment implementing X.509 now has another very well developed test suite that can be plugged in directly. While I'm a huge fan of formal methods for developing secure code, IMHO well developed suites such as x509-limbo[1] with common, good, problematic, and outright sadistically wrong vectors are the way forward for developing robust cross platform protocols. These are the kinds of unit tests a spec should come with. [1] https://x509-limbo.com https://x509-limbo.com
- ivanr 3y agoAgreed. I don't see NIST's test suites [1] on the list, though. Maybe they're embedded somewhere and I've missed it? [1] https://csrc.nist.gov/projects/pki-testing https://csrc.nist.gov/projects/pki-testing EDIT If the authors are reading, are any of the test cases taken from Chrome's codebase?
- woodruffw 3y ago(I'm one of the authors). I looked at NIST's PKITS while building the test suite. We ultimately decided to punt on it for the first round of tests, primarily because it fell slightly out of scope of our MVP on Python Cryptography (being strictly a RFC 3280 suite). We instead chose to integrate the BetterTLS suite[1] (warning: very large page) for the MVP. With that being said: x509-limbo is open source, and I would be overjoyed if someone added a new `nist::pkits` or similar namespace for these :-). I can't promise that it's an immediate priority for us to add ourselves, however. > are any of the test cases taken from Chrome's codebase? Nope. That would also be a valuable thing to have here. [1]: https://x509-limbo.com/testcases/bettertls/ https://x509-limbo.com/testcases/bettertls/
- ivanr 3y agoFor reference: https://github.com/chromium/chromium/tree/main/net/data/ssl https://github.com/chromium/chromium/tree/main/net/data/ssl It's been a long time since I looked at them, not sure what's in there exactly any more. Ask Ryan perhaps :)
- thriftwy 3y agoMeanwhile, JVM ecosystem looks down on the choice between OpenSSL and pure-Rust Python chain, because it have been using a managed, pure-Java X.509 chain for 25 years now. I believe it also had various implementations under common API.
- notaluddite 3y agoGiven the ambiguities and degrees of freedom different TLS clients can exercise during chain validation (mentioned in this post), we wrote a paper proposing that validation policies should be expressed as interchangeable Prolog programs: https://dl.acm.org/doi/pdf/10.1145/3548606.3560594 https://dl.acm.org/doi/pdf/10.1145/3548606.3560594
- flumpcakes 3y agoAren't revocations a pretty important part of certificates, considering the whole point of this is security? I don't understand the rationale for skipping.
- woodruffw 3y agoRevocations are a great idea. In practice, they're both operationally complicated and an unreliable source of truth about certificate validity. In practice, a lot of implementations (including Go's) simply ignore them. 'mcpherrim posted a comment below[1] with some additional information, including changes to the Web PKI and CRL distribution techniques that may eventually make them more useful. [1]: https://news.ycombinator.com/item?id=39132430 https://news.ycombinator.com/item?id=39132430
- ivanr 3y agoAny chance you could expand on "In practice, they're both operationally complicated and an unreliable source of truth about certificate validity."?
- woodruffw 3y agoI'm thinking about CRL update windows: a lot of CAs run weekly updates, meaning that CRL changes don't propagate to end users for up to a week at a time. That's well into "security theater" territory for me, given that a compromised CA or EE only needs to fool me for a matter of seconds, once.
- baby 3y agocan OpenSSL use this as well? This would be amazing
- im3w1l 3y agoI'm sure it was even worse before, but I'm not entirely happy with the look of that. * I shouldn't have to care where the certificates are stored. Just load the os default ones without asking me. * I shouldn't have to know what a pem is, and I shouldn't have to open() one. * I don't want a PolicyBuilder. Just give me the normal policy. * I shouldn't have to construct some verifier object specific to a given dns. I don't want to verify a billion certificates for one domain, just one. * I shouldn't have to know what an untrusted intermediary is. Here's what I think it should look like verifier = Verifier() # constructs a verifier with sensible defaults, overrides are possible though verifier.verify(chain, "cryptography.io")
- paragon_init 3y ago> * I shouldn't have to care where the certificates are stored. Just load the os default ones without asking me. > * I shouldn't have to know what a pem is, and I shouldn't have to open() one. Agreed, but what you're requesting is separate from the work being discussed in this blog post, and both are actually compatible. For the PHP community, we made Certainty - https://github.com/paragonie/certainty https://github.com/paragonie/certainty You can just... <?php use ParagonIE\Certainty\RemoteFetch; // cURL boilerplate $ch = curl_init(); curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true); // Fetch the latest CACert bundle, verify its authenticity, save it locally $fetcher = new RemoteFetch('/path/to/certainty/data'); $latestCACertBundle = $fetcher->getLatestBundle(); curl_setopt($ch, CURLOPT_CAINFO, $latestCACertBundle->getFilePath()); Writing a Python client should be relatively straightforward, should anyone want to. For Python specifically, there may be some value in storing the relevant PEM files into something like SigStore. That could be an easier proposal for the PyCA team to consider.
- RustyRussell 3y agoThis looks great! I wanted to bundle a chain of certs, check that the leaf was <some domain> and its key signed <some data>, and that the bundle provided a path to a locally-trusted root. This allows third-party serving of signed data, rather than having to trade your IP address for validation. Doing so with existing tools was so terrible that I was pretty sure my result was grossly insecure :( I look forward to the promised lower-level APIs!