4 ms·

We were talking tonight about CPAN modules, Python eggs, and PEAR... pears? We'd love to take what we've learned serving Rubygems and apply the knowledge to an

by defunkt 17y ago

We were talking tonight about CPAN modules, Python eggs, and PEAR... pears? We'd love to take what we've learned serving Rubygems and apply the knowledge to any other languages that have a solid packaging system.

jrockway 17y ago
Excellent. Please send me an email (jon AT jrock.us) and I can give you more detail (and help you implement). Basically, CPAN modules are just archives (usually tar.gz). There is a program called the "PAUSE indexer" that examines the archives, and generates a mapping of module name => archive file. The CPAN client fetches this file, and then knows what archive to download to install the named module. All github would need to do is maintain this index file; the rest is handled client-side. (The indexing algorithm isn't much more than this: http://github.com/jrockway/metacpan/blob/be78711a6f12afd3b71fa10ae137d70af6a785c9/lib/MetaCPAN/Distribution.pm#L211 http://github.com/jrockway/metacpan/blob/be78711a6f12afd3b71...)
ubernostrum 17y ago
Python may be a bit more difficult than you'd expect. The tools -- easy_install, pip, etc. -- are built around a core which, though it can use alternate sources to look for packages, happens to have things like heuristics for screen-scraping pypi.python.org hard-coded in.
- defunkt 17y ago
  Luckily the documentation explains clearly those heuristics and provides examples so you can appease them yourself :)
- simonw 17y ago
  Does anyone else find the idea of a package installer that /screen scrapes a website for download links/ deeply and fundamentally disturbing, or is it just me?
  - jrockway 17y ago
    It's obviously impossible to write bad code in Python, so clearly everyone else is wrong. Guido said so, and he works for Google!
tlrobinson 17y ago
I wish there was a standard package management protocol. Why does every language have to reinvent package management? They could still write the package manager clients (and servers) in their language, as they all seem to love to do. We (the ServerJS pseudo working group: https://wiki.mozilla.org/ServerJS https://wiki.mozilla.org/ServerJS) are currently looking at package management solutions for server-side JavaScript. Using Rubygems with other languages like JS is one possibility, then we would get the Github integration for free. And there's JSAN (http://openjsan.org/ http://openjsan.org/). Kris Kowal hacked together one that pulls zipped repos directly from GitHub (or potentially any HTTP server) (tentatively named "tusk", implemented on "narwhal": http://github.com/tlrobinson/narwhal/tree/integration http://github.com/tlrobinson/narwhal/tree/integration). One neat thing about that is you can create new versions just by tagging.

by defunkt 17y ago

jrockway 17y ago

Excellent. Please send me an email (jon AT jrock.us) and I can give you more detail (and help you implement). Basically, CPAN modules are just archives (usually tar.gz). There is a program called the "PAUSE indexer" that examines the archives, and generates a mapping of module name => archive file. The CPAN client fetches this file, and then knows what archive to download to install the named module. All github would need to do is maintain this index file; the rest is handled client-side. (The indexing algorithm isn't much more than this: http://github.com/jrockway/metacpan/blob/be78711a6f12afd3b71fa10ae137d70af6a785c9/lib/MetaCPAN/Distribution.pm#L211 http://github.com/jrockway/metacpan/blob/be78711a6f12afd3b71...)

ubernostrum 17y ago

Python may be a bit more difficult than you'd expect. The tools -- easy_install, pip, etc. -- are built around a core which, though it can use alternate sources to look for packages, happens to have things like heuristics for screen-scraping pypi.python.org hard-coded in.

defunkt 17y ago
Luckily the documentation explains clearly those heuristics and provides examples so you can appease them yourself :)
simonw 17y ago
Does anyone else find the idea of a package installer that /screen scrapes a website for download links/ deeply and fundamentally disturbing, or is it just me?
- jrockway 17y ago
  It's obviously impossible to write bad code in Python, so clearly everyone else is wrong. Guido said so, and he works for Google!

tlrobinson 17y ago

I wish there was a standard package management protocol. Why does every language have to reinvent package management? They could still write the package manager clients (and servers) in their language, as they all seem to love to do. We (the ServerJS pseudo working group: https://wiki.mozilla.org/ServerJS https://wiki.mozilla.org/ServerJS) are currently looking at package management solutions for server-side JavaScript. Using Rubygems with other languages like JS is one possibility, then we would get the Github integration for free. And there's JSAN (http://openjsan.org/ http://openjsan.org/). Kris Kowal hacked together one that pulls zipped repos directly from GitHub (or potentially any HTTP server) (tentatively named "tusk", implemented on "narwhal": http://github.com/tlrobinson/narwhal/tree/integration http://github.com/tlrobinson/narwhal/tree/integration). One neat thing about that is you can create new versions just by tagging.