7 ms·

The Python standard library has been a huge help for me. Evaluating which third party packages to trust and handling updates is a hassle. (Would love a solution

The Python standard library has been a huge help for me. Evaluating which third party packages to trust and handling updates is a hassle. (Would love a solution for this. Does anyone have a curated version of PyPI?) I’m surprised that people want to slim it down other than for performance on a more constrained system. As an aside, why doesn’t the Python standard library extend/replace features with code from successful packages like Requests? Tried it and it didn’t work? Too much bloat? Already got too much on the to-do list?

llukas 7y ago
+100 > As an aside, why doesn’t the Python standard library extend/replace features with code from successful packages like Requests? It is possible (ie. asyncio was separate package). It is slow process though.
- quietbritishjim 7y ago
  pathlib is another example.
- misterdoubt 7y ago
  I'd bet that absorbing Requests into the standard library, no matter the particular method of absorption proposed, would present too much of a political challenge to overcome.
jonnycomputer 7y ago
I've come to use packages outside of the standard library very sparingly; been burned too many times to find that development of some package stopped or slowed down and backing out can be a real pita.
- falcor84 7y ago
  What is your argument here? Standard library module development is also extremely slow.
  - kilburn 7y ago
    There is one major difference: an abandoned external package may break with newer python versions, whereas you can always count on stdlib packages being updated for new versions.
    - jonnycomputer 7y ago
      exactly. and i'm in academic software. its even worse.
    - falcor84 7y ago
      Can you provide an example? To the best of my knowledge, minor version updates in Python 3 have been entirely backwards so far and shouldn't have broken any library code. And as for incompatible changes sick as the transition to 3, then these of course had interface changes in the standard library.
pdonis 7y ago
> Already got too much on the to-do list? That would be my guess. I would also add that getting through the process of adding a third-party set of modules to the standard library can take quite a while.
debatem1 7y ago
The quote about the stdlib being "where packages go to die" was, for a long time, considered a feature and not a bug. The theory was that once a package is in the stdlib its development should slow to prioritize stability over new features. This may somewhat explain why lots of successful packages are not in the stdlib: putting them in effectively killed future development until a few years ago. But today that argument is inconsistently applied and I'm not sure if it's a rule worth keeping.
- sitkack 7y ago
  That phrase never had a good connotation. I believe someone skilled in PR respun that meaning.
aasasd 7y ago
> Does anyone have a curated version of PyPI? Pypi have thrown out the downloads counter—a huge misservice to coders. Like I got all day to figure out the best libs for ten different features which I only need in passing, so my primary concern is to not pick complete garbage. So, my solution to that now is to look up Github pages for the libs and choose the one with most stars. As much as I dislike Github for its occasional typical proprietary behavior, Gitlab doesn't help in this case.
- j88439h84 7y ago
  Pypi still has a download counter, but it will tend to reflect which libraries are used in ci, so it's a biased estimate.
  - aasasd 7y ago
    Could you please point me to the location of that counter? Because I ain't seeing it anywhere.
    - detaro 7y ago
      afaik it's not in the UI, but the dataset is published and and access provided by other sites: https://pypistats.org/ https://pypistats.org/
  - jancsika 7y ago
    If only there were a way to filter machines on the internet by some kind of id number, and to subtract numbers from other numbers at scale.
    - orf 7y ago
      And what about caches and proxies?
- petre 7y ago
  Perl has stars on MetaCPAN and every version has a test counter. Download counter is unreliable.
simonh 7y ago
Requests depends on urllib3 which would also have to go into the stdlib. It also contains a CA bundle which the core devs don’t want to do. It also likely the internal implementation doesn’t follow core dev standards and practices, a common problem with integrating external libs. Finally there’s a risk it would slow or discourage new feature development by tying it to the core release cycle. A better approach might be to add a core of basic requests-like features built from the stdlib’s existing resources. That would be beneficial to many users and if they need more then there’s always Requests.
- stochastastic 7y ago
  That makes sense; if it makes any difference I don’t necessarily mean “take code from X and drop it in” so much as “if the consensus appears to be that the X api is better then add that to stdlib”. I like that notion of adding the X api, or parts thereof, And then having a third party X+ package. Maybe I just like the idea of something I built being “worthy” of stdlib.
chestervonwinch 7y ago
> Does anyone have a curated version of PyPI? I'm not entirely sure what you're looking for, but have you tried https://www.enthought.com/product/enthought-deployment-manager/ https://www.enthought.com/product/enthought-deployment-manag... ? Edit: example $ edm envs create tester36 --version 3.6 $ edm shell -e tester36 (tester36) $ edm install ipython matplotlib pyqt
- stochastastic 7y ago
  That’s interesting. Thanks!
misterdoubt 7y ago
Curating packages for quality across multiple versions and architectures is hard work. In addition to Enthought (mentioned earlier), Anaconda maintains a curated set.
lukka5 7y ago
> Does anyone have a curated version of PyPI? https://python.libhunt.com/ https://python.libhunt.com/ Not exactly curation but it does have rated libs for lots of categories.

llukas 7y ago

+100 > As an aside, why doesn’t the Python standard library extend/replace features with code from successful packages like Requests? It is possible (ie. asyncio was separate package). It is slow process though.

quietbritishjim 7y ago
pathlib is another example.
misterdoubt 7y ago
I'd bet that absorbing Requests into the standard library, no matter the particular method of absorption proposed, would present too much of a political challenge to overcome.

jonnycomputer 7y ago

I've come to use packages outside of the standard library very sparingly; been burned too many times to find that development of some package stopped or slowed down and backing out can be a real pita.

falcor84 7y ago
What is your argument here? Standard library module development is also extremely slow.
- kilburn 7y ago
  There is one major difference: an abandoned external package may break with newer python versions, whereas you can always count on stdlib packages being updated for new versions.
  - jonnycomputer 7y ago
    exactly. and i'm in academic software. its even worse.
  - falcor84 7y ago
    Can you provide an example? To the best of my knowledge, minor version updates in Python 3 have been entirely backwards so far and shouldn't have broken any library code. And as for incompatible changes sick as the transition to 3, then these of course had interface changes in the standard library.

pdonis 7y ago

> Already got too much on the to-do list? That would be my guess. I would also add that getting through the process of adding a third-party set of modules to the standard library can take quite a while.

debatem1 7y ago

The quote about the stdlib being "where packages go to die" was, for a long time, considered a feature and not a bug. The theory was that once a package is in the stdlib its development should slow to prioritize stability over new features. This may somewhat explain why lots of successful packages are not in the stdlib: putting them in effectively killed future development until a few years ago. But today that argument is inconsistently applied and I'm not sure if it's a rule worth keeping.

sitkack 7y ago
That phrase never had a good connotation. I believe someone skilled in PR respun that meaning.

aasasd 7y ago

> Does anyone have a curated version of PyPI? Pypi have thrown out the downloads counter—a huge misservice to coders. Like I got all day to figure out the best libs for ten different features which I only need in passing, so my primary concern is to not pick complete garbage. So, my solution to that now is to look up Github pages for the libs and choose the one with most stars. As much as I dislike Github for its occasional typical proprietary behavior, Gitlab doesn't help in this case.

j88439h84 7y ago
Pypi still has a download counter, but it will tend to reflect which libraries are used in ci, so it's a biased estimate.
- aasasd 7y ago
  Could you please point me to the location of that counter? Because I ain't seeing it anywhere.
  - detaro 7y ago
    afaik it's not in the UI, but the dataset is published and and access provided by other sites: https://pypistats.org/ https://pypistats.org/
- jancsika 7y ago
  If only there were a way to filter machines on the internet by some kind of id number, and to subtract numbers from other numbers at scale.
  - orf 7y ago
    And what about caches and proxies?
petre 7y ago
Perl has stars on MetaCPAN and every version has a test counter. Download counter is unreliable.

simonh 7y ago

Requests depends on urllib3 which would also have to go into the stdlib. It also contains a CA bundle which the core devs don’t want to do. It also likely the internal implementation doesn’t follow core dev standards and practices, a common problem with integrating external libs. Finally there’s a risk it would slow or discourage new feature development by tying it to the core release cycle. A better approach might be to add a core of basic requests-like features built from the stdlib’s existing resources. That would be beneficial to many users and if they need more then there’s always Requests.

stochastastic 7y ago
That makes sense; if it makes any difference I don’t necessarily mean “take code from X and drop it in” so much as “if the consensus appears to be that the X api is better then add that to stdlib”. I like that notion of adding the X api, or parts thereof, And then having a third party X+ package. Maybe I just like the idea of something I built being “worthy” of stdlib.

chestervonwinch 7y ago

> Does anyone have a curated version of PyPI? I'm not entirely sure what you're looking for, but have you tried https://www.enthought.com/product/enthought-deployment-manager/ https://www.enthought.com/product/enthought-deployment-manag... ? Edit: example $ edm envs create tester36 --version 3.6 $ edm shell -e tester36 (tester36) $ edm install ipython matplotlib pyqt

stochastastic 7y ago
That’s interesting. Thanks!

misterdoubt 7y ago

Curating packages for quality across multiple versions and architectures is hard work. In addition to Enthought (mentioned earlier), Anaconda maintains a curated set.

lukka5 7y ago

> Does anyone have a curated version of PyPI? https://python.libhunt.com/ https://python.libhunt.com/ Not exactly curation but it does have rated libs for lots of categories.