6 ms·

There was advocacy and pressure to adopt pyca/cryptography in other popular python packages. Don't roll your own crypto, don't use old unmaintained crypto libra

There was advocacy and pressure to adopt pyca/cryptography in other popular python packages. Don't roll your own crypto, don't use old unmaintained crypto libraries (pycrypto ... but these days there's pycryptodome) etc. If pyca/cryptography was instead advertised as "python crypto library for rust lovers" and other python developers knew that a _gigantic_ and _obnoxious_ rust toolchain dependency was involved, I think many would have avoided cryptography, or made it optional. This was a bait-and-switch. This is obnoxious for people who want to start with a tiny standard distro image with just a C toolchain, and build their app and all dependencies from source. It is also obnoxious for distros that want to build all packages from source. rust depends on llvm, itself a huge complicated finicky package ... but not just that, it depends on a custom-patched llvm! ...but not just that, it also depends on an extremely recent version of rust! this problem is recursive! And there are many other niche cases ... and "niche cases" of "unauthorized" porting/modification/substitution is how much of the open source ecosystem we love got its start. This is really about issues of paranoia and control in the "security" realm. They consider it irresponsible to enable users to do anything they can't guarantee "safe". "Your organization should be paying for a $$$ corporate support contract if you want to do anything we don't officialy support." Maybe you could have fixed a portability bug in some python or C code (I've fixed a couple when bringing up a little-endian mips64 system a decade ago), but no you'll need rust+llvm experts to port that whole crazy toolchain. It's for your own good.

tptacek 6y ago
Here you seem to be demanding that the authors and maintainers of their own cryptography library lighten up about security, because you find the way they manage their own package to be controlling. You see some irony in that, right?
- will4274 6y ago
  I don't read /u/ploxiln as demanding anything. They seem frustrated with the changing dependency profile of the dependency, and a little frustrated with the way some people like to shout "security" as if it's a trump card, ignoring that security is a spectrum and that the only perfectly secure software does nothing (perfectly).
  - tptacek 6y ago
    This is a security library. In fact, it's a security library that was created specifically to harden and user-proof less secure alternatives. If you don't care as much as they do about security, use a different library (or just keep using the pre-Rust version of this one).
aw1621107 6y ago
> If pyca/cryptography was instead advertised as "python crypto library for rust lovers" I feel like this really only makes sense if pyca/cryptography had planned on adding the Rust dependency from the very beginning (or from very early on). Is there any indication that was the case? > but not just that, it depends on a custom-patched llvm! This doesn't seem to be true [0, 1]. [0]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.... [1]: https://news.ycombinator.com/item?id=26217182 https://news.ycombinator.com/item?id=26217182
- ploxiln 6y ago
  >> Strongly prefer to upstream all patches to LLVM before including them in rustc. > That is, this is already the case. We don't like maintaining a fork. We try to upstream as much as we can. > But, at the same time, even when you do this, it takes tons of time. A contributor was talking about exactly this on Twitter earlier today, and estimated that, even if the patch was written and accepted as fast as possible, it would still take roughly a year for that patch to make it into the release used by Rust. This is the opposite side of the whole "move slow and only use old versions of things" tradeoff: that would take even longer to get into, say, the LLVM in Debian stable, as suggested in another comment chain. > So our approach is, upstream the patches, keep them in our fork, and then remove them when they inevitably make it back downstream. ... so there are always some outstanding patches rust applies to the llvm codebase.
  - aw1621107 6y ago
    For Rust's LLVM fork, sure. But as Steve Klabnik noted in the first comment I linked, unmodified LLVM is supported. In addition, later down in the comment chain: cycloptic: > Can't there be a build option to not use the LLVM submodule, and instead use the system LLVM? steveklabnik: > There is. We even test that it builds as part of our CI, to make sure it works, IIRC. For a more concrete example, Fedora supports choosing between system and bundled LLVM when building Rust [0, 1]. [0]: https://news.ycombinator.com/item?id=26222190 https://news.ycombinator.com/item?id=26222190 [1]: https://src.fedoraproject.org/rpms/rust//blob/rawhide/f/rust.spec https://src.fedoraproject.org/rpms/rust//blob/rawhide/f/rust...
tsimionescu 6y ago
What you're saying makes sense in any other domain except cryptographical primitives. With those, the basic correctness of the code will depend on details of the compiler and processor architecture, and it is extremely likely that the code will be fundamentally incorrect when used on other architectures or compilers. And this is not just a matter of fixing some little-endian assumption, it's a matter of understanding microoptimizations, instruction reordering, prefetch behavior, branch prediction etc. Ensuring code has predictable runtime in the presence of an optimizing compiler and out of order processor is extremely difficult. It's probably just as smart to roll your own crypto instead of trying to use a known library on a new platform, because the hardest parts of rolling your own will have to be done anyway.
mlindner 6y ago
> rust depends on llvm, itself a huge complicated finicky package gcc is also a "huge complicated finicky package"... Any decently sized compiler is.

tptacek 6y ago

Here you seem to be demanding that the authors and maintainers of their own cryptography library lighten up about security, because you find the way they manage their own package to be controlling. You see some irony in that, right?

will4274 6y ago
I don't read /u/ploxiln as demanding anything. They seem frustrated with the changing dependency profile of the dependency, and a little frustrated with the way some people like to shout "security" as if it's a trump card, ignoring that security is a spectrum and that the only perfectly secure software does nothing (perfectly).
- tptacek 6y ago
  This is a security library. In fact, it's a security library that was created specifically to harden and user-proof less secure alternatives. If you don't care as much as they do about security, use a different library (or just keep using the pre-Rust version of this one).

aw1621107 6y ago

> If pyca/cryptography was instead advertised as "python crypto library for rust lovers" I feel like this really only makes sense if pyca/cryptography had planned on adding the Rust dependency from the very beginning (or from very early on). Is there any indication that was the case? > but not just that, it depends on a custom-patched llvm! This doesn't seem to be true [0, 1]. [0]: https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.html https://rustc-dev-guide.rust-lang.org/backend/updating-llvm.... [1]: https://news.ycombinator.com/item?id=26217182 https://news.ycombinator.com/item?id=26217182

ploxiln 6y ago
>> Strongly prefer to upstream all patches to LLVM before including them in rustc. > That is, this is already the case. We don't like maintaining a fork. We try to upstream as much as we can. > But, at the same time, even when you do this, it takes tons of time. A contributor was talking about exactly this on Twitter earlier today, and estimated that, even if the patch was written and accepted as fast as possible, it would still take roughly a year for that patch to make it into the release used by Rust. This is the opposite side of the whole "move slow and only use old versions of things" tradeoff: that would take even longer to get into, say, the LLVM in Debian stable, as suggested in another comment chain. > So our approach is, upstream the patches, keep them in our fork, and then remove them when they inevitably make it back downstream. ... so there are always some outstanding patches rust applies to the llvm codebase.
- aw1621107 6y ago
  For Rust's LLVM fork, sure. But as Steve Klabnik noted in the first comment I linked, unmodified LLVM is supported. In addition, later down in the comment chain: cycloptic: > Can't there be a build option to not use the LLVM submodule, and instead use the system LLVM? steveklabnik: > There is. We even test that it builds as part of our CI, to make sure it works, IIRC. For a more concrete example, Fedora supports choosing between system and bundled LLVM when building Rust [0, 1]. [0]: https://news.ycombinator.com/item?id=26222190 https://news.ycombinator.com/item?id=26222190 [1]: https://src.fedoraproject.org/rpms/rust//blob/rawhide/f/rust.spec https://src.fedoraproject.org/rpms/rust//blob/rawhide/f/rust...

tsimionescu 6y ago

What you're saying makes sense in any other domain except cryptographical primitives. With those, the basic correctness of the code will depend on details of the compiler and processor architecture, and it is extremely likely that the code will be fundamentally incorrect when used on other architectures or compilers. And this is not just a matter of fixing some little-endian assumption, it's a matter of understanding microoptimizations, instruction reordering, prefetch behavior, branch prediction etc. Ensuring code has predictable runtime in the presence of an optimizing compiler and out of order processor is extremely difficult. It's probably just as smart to roll your own crypto instead of trying to use a known library on a new platform, because the hardest parts of rolling your own will have to be done anyway.

mlindner 6y ago

> rust depends on llvm, itself a huge complicated finicky package gcc is also a "huge complicated finicky package"... Any decently sized compiler is.