6 ms·

A Map of Python

175 points by fi-le 2y ago

fsdkfdsf 2y ago
[flagged]
WillAdams 2y ago
Are there any languages which have an hierarchical repository which is structured so that one can easily browse and find what one needs? Or, arriving where it ought to be, discover that it does not yet exist? CTAN seems pretty close, but is tightly focused on TeX, and CPAN is for Perl, not Python.
- bradrn 2y ago
  Hackage has a categories system which can be used to browse through all Haskell packages [0]. Unfortunately the categories are unstandardised, so it’s less useful than it could be — though the Flora project exists, which has improved the UI by standardising categories a bit [1]. [0] https://hackage.haskell.org/packages/ https://hackage.haskell.org/packages/ [1] https://flora.pm/categories https://flora.pm/categories
qwertox 2y ago
That is not an iteractive map. Google Chrome is the only piece of software which manages to bring my system to a halt, where the mouse then moves an 0.2 FPS. That map should have used a mapping server, possibly serving vector tiles.
- fi-le 2y ago
  You're right, evidently my programming wasn't quite up to the task here. Good to see though that there's demand, so I'll try and make a better version implementing something like you are describing.
- miohtama 2y ago
  Hacker News sends its best angry nerds (:
pluto_modadic 2y ago
This could be improved slightly to make it more performant. Possibly by slowly exposing data to the user?
leejoramo 2y ago
I worked with Zope/Plone for about 10 years in the 2000s. Without reading the story, I loaded the larger version of the graph and looked for a cluster for Plone. On my third click, I found it. Then I read the article which actually stated that Plone is one of distinct clusters. Pretty amazing for a 20+ year old technology
- BiteCode_dev 2y ago
  My first job was with plone. I hated it. Then I learned the twisted ecosystem and had a moment of crisis. Those are the devil pits of the python world. Thank god for django and asyncio.
  - leejoramo 2y ago
    Python programming Plone was painful. However, I learned so much from the entire system. * The CSS of the Plone theme was a Masterpiece. There is a very good reason why Wikipedia used a near direct copy Plone’s CSS for most of the 2000s. Using just a layer of CSS and minor changes to the templates, I could radically re-theme an entire site in a short amount of time. * Plone enforced semantic HTML and used XHTML. Regardless of what you think of the value of semantics and XHTML, it thought me how to create well structured HTML at a time when the web was full of very broken HTML4 * While programming was painful, Plone’s UX for content managers was first rate. I was invoked in testing Plone, Joomla, Drupal and WordPress. Plone got top marks by a large margin * Again too marks for Accessibility. In 2005, I built a Plone site for a nonprofit that worked with the blind. I remember users saying they could not believe how easy Plone was to use using the Jaws screen reader * Multi-lingual sites with workflows for translators. Last year I ran into a translator who used a Plone site I build 20 years ago. They lamented that none of the sites the work on today are as good as that old Plone site. * etc
    - BiteCode_dev 2y ago
      Yes it was a good tech from the user perspective. If you could make it render fast enough, that is. And twisted did make use cases possible in python that would have been near impossible otherwise. If you could debug it, that is.
andai 2y ago
The article also links to this gem, where you can fly through them like a galaxy (WASD controls): https://anvaka.github.io/pm/#/galaxy/python?cx=-2700&cy=377&cz=5622&lx=-0.0869&ly=-0.2315&lz=-0.0338&lw=0.9684&ml=150&s=1.75&l=1&v=2015-09-27T13-00-00Z https://anvaka.github.io/pm/#/galaxy/python?cx=-2700&cy=377&...
- mahouk 2y ago
  How can one build a visualisation like this? And does something like this exist but for the web?
  - dleeftink 2y ago
    Anvanka provides detailed instructions here[0]. Nowadays I would also consider using Cosmograph[1]. [0]: https://github.com/anvaka/pm?tab=readme-ov-file#your-own-graphs https://github.com/anvaka/pm?tab=readme-ov-file#your-own-gra... [1]: https://github.com/cosmograph-org https://github.com/cosmograph-org
- kwar13 2y ago
  That's some buttery smooth navigation.
- inkubus 2y ago
  Looks very cool on mobile!
wodenokoto 2y ago
It would also be interesting to have "transient" dependencies count towards "Used by metric" E.g., Request is used by more packages than URLlib3, even though requests requires urllib, so in a sense, urllib is used by many more than requests.
- HelloNurse 2y ago
  They are called transitive dependencies. Considering how packages ften have optional and unstated dependencies I'd expect an analysis of transitive dependency chains to become very vague, at best distinguishing between packages that are used mostly directly or, like urllib, mostly indirectly.
dirkc 2y ago
Nice! I like the look of the resulting map. I've visualized gentoo packages and their dependencies before [1], but the result is no where near as aesthetic as this. [1] - https://www.thebacklog.net/2011/04/06/high-resolution-dependency-graph/ https://www.thebacklog.net/2011/04/06/high-resolution-depend...
orf 2y ago
I maintain a project that publishes a SQLite file containing all package metadata, if you don’t want to use BigQuery or the API to do this kind of analysis https://github.com/pypi-data/pypi-json-data https://github.com/pypi-data/pypi-json-data
btreecat 2y ago
I truly despise Click and the ways you're supposed to test it
CraigJPerry 2y ago
>> pyyaml I keep being surprised at the popularity of this dependency. It has a great name for discovery but the API does not easily provide ways to override or customise behaviours. The continued popularity suggests it’s probably me holding the tool wrong.
- taeric 2y ago
  I'm curious what sort of behavior you are wanting to change? And, yeah, don't underestimate the effectiveness of a good name.
globular-toast 2y ago
Nitpick: It's PyPI (pronounced pie-pee-eye).
- codexb 2y ago
  I will always pronounce it pie-pie
  - globular-toast 2y ago
    PyPy is the jit interpreter, though. Thankfully it's usually clear from context.
codexb 2y ago
I would have expected requests to be more prevalent.

175 points by fi-le 2y ago

[flagged]

Are there any languages which have an hierarchical repository which is structured so that one can easily browse and find what one needs? Or, arriving where it ought to be, discover that it does not yet exist? CTAN seems pretty close, but is tightly focused on TeX, and CPAN is for Perl, not Python.

bradrn 2y ago
Hackage has a categories system which can be used to browse through all Haskell packages [0]. Unfortunately the categories are unstandardised, so it’s less useful than it could be — though the Flora project exists, which has improved the UI by standardising categories a bit [1]. [0] https://hackage.haskell.org/packages/ https://hackage.haskell.org/packages/ [1] https://flora.pm/categories https://flora.pm/categories

qwertox 2y ago

That is not an iteractive map. Google Chrome is the only piece of software which manages to bring my system to a halt, where the mouse then moves an 0.2 FPS. That map should have used a mapping server, possibly serving vector tiles.

fi-le 2y ago
You're right, evidently my programming wasn't quite up to the task here. Good to see though that there's demand, so I'll try and make a better version implementing something like you are describing.
miohtama 2y ago
Hacker News sends its best angry nerds (:

pluto_modadic 2y ago

This could be improved slightly to make it more performant. Possibly by slowly exposing data to the user?

leejoramo 2y ago

I worked with Zope/Plone for about 10 years in the 2000s. Without reading the story, I loaded the larger version of the graph and looked for a cluster for Plone. On my third click, I found it. Then I read the article which actually stated that Plone is one of distinct clusters. Pretty amazing for a 20+ year old technology

BiteCode_dev 2y ago
My first job was with plone. I hated it. Then I learned the twisted ecosystem and had a moment of crisis. Those are the devil pits of the python world. Thank god for django and asyncio.
- leejoramo 2y ago
  Python programming Plone was painful. However, I learned so much from the entire system. * The CSS of the Plone theme was a Masterpiece. There is a very good reason why Wikipedia used a near direct copy Plone’s CSS for most of the 2000s. Using just a layer of CSS and minor changes to the templates, I could radically re-theme an entire site in a short amount of time. * Plone enforced semantic HTML and used XHTML. Regardless of what you think of the value of semantics and XHTML, it thought me how to create well structured HTML at a time when the web was full of very broken HTML4 * While programming was painful, Plone’s UX for content managers was first rate. I was invoked in testing Plone, Joomla, Drupal and WordPress. Plone got top marks by a large margin * Again too marks for Accessibility. In 2005, I built a Plone site for a nonprofit that worked with the blind. I remember users saying they could not believe how easy Plone was to use using the Jaws screen reader * Multi-lingual sites with workflows for translators. Last year I ran into a translator who used a Plone site I build 20 years ago. They lamented that none of the sites the work on today are as good as that old Plone site. * etc
  - BiteCode_dev 2y ago
    Yes it was a good tech from the user perspective. If you could make it render fast enough, that is. And twisted did make use cases possible in python that would have been near impossible otherwise. If you could debug it, that is.

andai 2y ago

The article also links to this gem, where you can fly through them like a galaxy (WASD controls): https://anvaka.github.io/pm/#/galaxy/python?cx=-2700&cy=377&cz=5622&lx=-0.0869&ly=-0.2315&lz=-0.0338&lw=0.9684&ml=150&s=1.75&l=1&v=2015-09-27T13-00-00Z https://anvaka.github.io/pm/#/galaxy/python?cx=-2700&cy=377&...

mahouk 2y ago
How can one build a visualisation like this? And does something like this exist but for the web?
- dleeftink 2y ago
  Anvanka provides detailed instructions here[0]. Nowadays I would also consider using Cosmograph[1]. [0]: https://github.com/anvaka/pm?tab=readme-ov-file#your-own-graphs https://github.com/anvaka/pm?tab=readme-ov-file#your-own-gra... [1]: https://github.com/cosmograph-org https://github.com/cosmograph-org
kwar13 2y ago
That's some buttery smooth navigation.
inkubus 2y ago
Looks very cool on mobile!

wodenokoto 2y ago

It would also be interesting to have "transient" dependencies count towards "Used by metric" E.g., Request is used by more packages than URLlib3, even though requests requires urllib, so in a sense, urllib is used by many more than requests.

HelloNurse 2y ago
They are called transitive dependencies. Considering how packages ften have optional and unstated dependencies I'd expect an analysis of transitive dependency chains to become very vague, at best distinguishing between packages that are used mostly directly or, like urllib, mostly indirectly.

dirkc 2y ago

Nice! I like the look of the resulting map. I've visualized gentoo packages and their dependencies before [1], but the result is no where near as aesthetic as this. [1] - https://www.thebacklog.net/2011/04/06/high-resolution-dependency-graph/ https://www.thebacklog.net/2011/04/06/high-resolution-depend...

orf 2y ago

I maintain a project that publishes a SQLite file containing all package metadata, if you don’t want to use BigQuery or the API to do this kind of analysis https://github.com/pypi-data/pypi-json-data https://github.com/pypi-data/pypi-json-data

btreecat 2y ago

I truly despise Click and the ways you're supposed to test it

CraigJPerry 2y ago

>> pyyaml I keep being surprised at the popularity of this dependency. It has a great name for discovery but the API does not easily provide ways to override or customise behaviours. The continued popularity suggests it’s probably me holding the tool wrong.

taeric 2y ago
I'm curious what sort of behavior you are wanting to change? And, yeah, don't underestimate the effectiveness of a good name.

globular-toast 2y ago

Nitpick: It's PyPI (pronounced pie-pee-eye).

codexb 2y ago
I will always pronounce it pie-pie
- globular-toast 2y ago
  PyPy is the jit interpreter, though. Thankfully it's usually clear from context.

codexb 2y ago

I would have expected requests to be more prevalent.