3 ms·

I was rather shocked to find that python didn’t have a full-featured crypto library included in its standard lib. The alternatives all ended up being unmaintain

by kentm 7y ago

I was rather shocked to find that python didn’t have a full-featured crypto library included in its standard lib. The alternatives all ended up being unmaintained or maintained by small groups (which makes trust in the soundness difficult). I tapped our security team, who were in disbelief, but ultimately they gave up to and I wrote the software in go instead.

tptacek 7y ago
You want pyca/cryptography. The last thing in the world you want is a standard crypto library that no experts are enthusiastic about maintaining. Golang had an unfair advantage here, because the language team included cryptography engineers. It would be weird if most languages had the same kind of crypto in their stdlibs. I think there is in general nothing wrong with a language ecosystem where key parts of the whole platform are in well-maintained third-party libraries rather than the standard library. Which is also something Amber Brown is saying here.
- jteppinette 7y ago
  When you find a coworker trying to use this module.. > cryptography/src/cryptography/hazmat/__init__.py Hazardous Materials This is a "Hazardous Materials" module. You should ONLY use it if you're 100% absolutely sure that you know what you're doing because this module is full of land mines, dragons, and dinosaurs with laser guns.
- geofft 7y ago
  Also I expect that a) pyca/cryptography is maintained by as many people as Go's cryptography libraries are, so the worry about maintenance by small groups doesn't argue in favor of Go here b) the number of people maintaining any particular module in the Python standard library is very small (e.g., IIRC there's one maintainer for `ssl`). I would be surprised if either 'kentm or their security team wanted the rest of the Python standard library developers (or the Go ones, for that matter) to mess with cryptography modules when it's not their area of expertise.
  - tptacek 7y ago
    I think we're saying basically the same thing.
- TheOtherHobbes 7y ago
  The problem with third-party libraries is that it's impossible to know which ones are "standard" and which ones are copycat hobby projects in various states of disrepair. And in the worst case you can pick a library, get a day or two into using it, and discover it has a show-stopper limitation. I recently hacked together some Python to process MIDI files. Should I have used this: https://pypi.org/project/MIDIUtil/ https://pypi.org/project/MIDIUtil/ or this: https://pypi.org/project/mxm.midifile/ https://pypi.org/project/mxm.midifile/ or this: https://mido.readthedocs.io/en/latest/ https://mido.readthedocs.io/en/latest/ or this? https://github.com/vishnubob/python-midi https://github.com/vishnubob/python-midi A well-maintained and thoughtfully curated stdlib makes these choices for you - which is one less thing to worry about, and can be a significant time saver.
  - tptacek 7y ago
    In a perfect world everything would be in the stdlib, everything would be well-integrated, everything would serve every use case, and everything would be well-maintained by engaged and motivated maintainers. For some kinds of libraries, you can sacrifice a whole bunch of those constraints and still have it make sense to host it in the stdlib. I'm fine if the JSON library is slow or inflexible if it covers a bunch of use cases and doesn't impede people from writing better ones. You can see this to some extent with the state of "router" HTTP libraries in Go. But for some things, most notably cryptography, it's worse to have a suboptimal version in the stdlib than to have none at all.
    - Rapzid 7y ago
      To add to net/http not being very competitive performance wise within Go's language peer group, the stdlib file system walking methods took liberties in baking in assumptions without escape hatches that nerf the performance unnecessarily for a lot of heavy-lift use cases. Go definitely has a few shortcomings in the otherwise fantastically useful stdlib.
  - roca 7y ago
    You can have curation outside the standard library, without incurring all the costs of moving libraries into the standard library.
- ddebernardy 7y ago
  Sure, but in Python's case, what's wrong with bundling a light wrapper around a well maintained C library?
  - tptacek 7y ago
    That it would be inferior to pyca/cryptography?
    - ddebernardy 7y ago
      Honestly, I've no idea. For all I know some Python devs are even more sophisticated about security than the C developers who are maintaining libraries for Linux and *BSD distros.

by kentm 7y ago

tptacek 7y ago

You want pyca/cryptography. The last thing in the world you want is a standard crypto library that no experts are enthusiastic about maintaining. Golang had an unfair advantage here, because the language team included cryptography engineers. It would be weird if most languages had the same kind of crypto in their stdlibs. I think there is in general nothing wrong with a language ecosystem where key parts of the whole platform are in well-maintained third-party libraries rather than the standard library. Which is also something Amber Brown is saying here.

jteppinette 7y ago
When you find a coworker trying to use this module.. > cryptography/src/cryptography/hazmat/__init__.py Hazardous Materials This is a "Hazardous Materials" module. You should ONLY use it if you're 100% absolutely sure that you know what you're doing because this module is full of land mines, dragons, and dinosaurs with laser guns.
geofft 7y ago
Also I expect that a) pyca/cryptography is maintained by as many people as Go's cryptography libraries are, so the worry about maintenance by small groups doesn't argue in favor of Go here b) the number of people maintaining any particular module in the Python standard library is very small (e.g., IIRC there's one maintainer for `ssl`). I would be surprised if either 'kentm or their security team wanted the rest of the Python standard library developers (or the Go ones, for that matter) to mess with cryptography modules when it's not their area of expertise.
- tptacek 7y ago
  I think we're saying basically the same thing.
TheOtherHobbes 7y ago
The problem with third-party libraries is that it's impossible to know which ones are "standard" and which ones are copycat hobby projects in various states of disrepair. And in the worst case you can pick a library, get a day or two into using it, and discover it has a show-stopper limitation. I recently hacked together some Python to process MIDI files. Should I have used this: https://pypi.org/project/MIDIUtil/ https://pypi.org/project/MIDIUtil/ or this: https://pypi.org/project/mxm.midifile/ https://pypi.org/project/mxm.midifile/ or this: https://mido.readthedocs.io/en/latest/ https://mido.readthedocs.io/en/latest/ or this? https://github.com/vishnubob/python-midi https://github.com/vishnubob/python-midi A well-maintained and thoughtfully curated stdlib makes these choices for you - which is one less thing to worry about, and can be a significant time saver.
- tptacek 7y ago
  In a perfect world everything would be in the stdlib, everything would be well-integrated, everything would serve every use case, and everything would be well-maintained by engaged and motivated maintainers. For some kinds of libraries, you can sacrifice a whole bunch of those constraints and still have it make sense to host it in the stdlib. I'm fine if the JSON library is slow or inflexible if it covers a bunch of use cases and doesn't impede people from writing better ones. You can see this to some extent with the state of "router" HTTP libraries in Go. But for some things, most notably cryptography, it's worse to have a suboptimal version in the stdlib than to have none at all.
  - Rapzid 7y ago
    To add to net/http not being very competitive performance wise within Go's language peer group, the stdlib file system walking methods took liberties in baking in assumptions without escape hatches that nerf the performance unnecessarily for a lot of heavy-lift use cases. Go definitely has a few shortcomings in the otherwise fantastically useful stdlib.
- roca 7y ago
  You can have curation outside the standard library, without incurring all the costs of moving libraries into the standard library.
ddebernardy 7y ago
Sure, but in Python's case, what's wrong with bundling a light wrapper around a well maintained C library?
- tptacek 7y ago
  That it would be inferior to pyca/cryptography?
  - ddebernardy 7y ago
    Honestly, I've no idea. For all I know some Python devs are even more sophisticated about security than the C developers who are maintaining libraries for Linux and *BSD distros.