6 ms·

Libraries.io – Open-Source Discovery Service

136 points by dhendo 12y ago

andrewnez 12y ago
More details on how it works in this blog post: https://medium.com/@teabass/solving-open-source-discovery-db43a04cd9e7 https://medium.com/@teabass/solving-open-source-discovery-db...
dannypgh 12y ago
Browsing C++ and coming up mostly with nodejs libraries. http://libraries.io/search?languages=C%2B%2B&order=desc&sort=rank http://libraries.io/search?languages=C%2B%2B&order=desc&sort...
- MordodeMaru 12y ago
  You should check biicode (C++ deps manager) which will be compatible with VersionEye soon allowing you to know about updates in the libs you depend. https://www.biicode.com https://www.biicode.com
- andrewnez 12y ago
  Currently I'm only pulling data from package managers like rubygems and npm, I couldn't find a C++ one yet, although https://www.biicode.com/ https://www.biicode.com/ looks handy for that, I'll investigate.
  - humanrebar 12y ago
    C and C++ is usually packaged with the OS's package manager, like dpkg, rpm, pacman, etc. The closest thing to rubygems might be Portage (1) or AUR (2). You might be able to search each of those for "c++". Or maybe peek into the constituent files for '.c' or '.cpp', '.cxx', etc. 1) http://en.wikipedia.org/wiki/Portage_%28software%29 http://en.wikipedia.org/wiki/Portage_%28software%29 2) https://wiki.archlinux.org/index.php/Arch_User_Repository https://wiki.archlinux.org/index.php/Arch_User_Repository
  - emilsedgh 12y ago
    include [0] lists reusable Qt libraries. [0] http://inqlude.org http://inqlude.org
dom96 12y ago
Nice to see that even my package manager for the Nim programming language is there. Pity that the language is still called "Nimrod" there though.
- andrewnez 12y ago
  I'm using https://github.com/github/linguist https://github.com/github/linguist for language information, still waiting for this pr to be merged for the Nimrod => Nim change: https://github.com/github/linguist/pull/1936 https://github.com/github/linguist/pull/1936
  - dom96 12y ago
    I see. I wonder what's taking them so long. In any case I am impressed that you implemented support for Nimble :)
guardian5x 12y ago
Looks very interesting. Can't seem to find Qt though, but lots of related things, like qt bindings for node.js
WimLeers 12y ago
I don't understand why Wordpress is considered a platform next to npm, bower, pypi etc.
stared 12y ago
It would be great if: - it sorted searches by GitHub starts (otherwise for any popular thing, you get first tons of stuff, in a non-relevant order), - for PyPI, there is also anther figure of merit, i.e. no of downloads (most are open source, but some are on other repositories), - in general, for Python it misses some very popular packages... for example, http://libraries.io/search?q=networkx http://libraries.io/search?q=networkx is from PyPI but not from Python (why?). Otherwise, a very needed idea! I wanted to do my own some time ago (http://pypi.meteor.com/ http://pypi.meteor.com/), but didn't have time to bring it to a useful (or nice) version.
- stared 12y ago
  And, for some packages clearly being on GitHub, it does not find their place, e.g. http://libraries.io/pypi/pandas http://libraries.io/pypi/pandas vs https://github.com/pydata/pandas https://github.com/pydata/pandas (4k+ stars).
- andrewnez 12y ago
  You can sort by stars, there's just not a UI yet: http://libraries.io/search?order=desc&sort=stars http://libraries.io/search?order=desc&sort=stars Pypi is lacking in information but I'm working on pulling it from other sources to improve the quality for Python libraries.
  - stared 12y ago
    For PyPI it is rather straightforward to get downloads information: http://stackoverflow.com/questions/21419009/json-api-for-pypi-how-to-list-packages http://stackoverflow.com/questions/21419009/json-api-for-pyp... (along with some other data, e.g. categorization). See example: https://pypi.python.org/pypi/seaborn/json https://pypi.python.org/pypi/seaborn/json
mlhamel 12y ago
weird there's no not many python entries... where it seems there's way more than 16k package in it (ie: https://alexgaynor.net/2014/jan/03/pypi-download-statistics/ https://alexgaynor.net/2014/jan/03/pypi-download-statistics/)
deepakprakash 12y ago
Quick feedback: Browsing the Go section, each sub directory(at all depths) in the Docker project seems to be listed as a separate project. Leads to showing a few pages of results linking to the same project(ie, Docker). Also may explain why Go seems to have far more number of projects listed than anything else. :) Update: Better wording
amirmc 12y ago
A few of the OCaml projects are showing up under Standard ML [1]. A little disappointing since anyone using OCaml would really benefit from finding out about Merlin (see the link). This is due to a recent issue (or regression) with how GitHub's Linguist disambiguates code, which I hope will be fixed soon [2]. [1] http://libraries.io/search?languages=Standard+ML http://libraries.io/search?languages=Standard+ML [2] https://github.com/github/linguist/issues/2208 https://github.com/github/linguist/issues/2208
- andrewnez 12y ago
  At the moment we lean completely on linguist for language detection, which isn't perfect but gets us a good amount of the way there. Once we've got OPAM support more OCaml libraries will start to show up.
rane 12y ago
Why can't this library http://libraries.io/npm/debug http://libraries.io/npm/debug be found in this list? http://libraries.io/search?keywords=debug&platforms=NPM http://libraries.io/search?keywords=debug&platforms=NPM
- andrewnez 12y ago
  The keyword filtering is slightly borked at the moment, this does work though: http://libraries.io/search?platforms=NPM&q=debug http://libraries.io/search?platforms=NPM&q=debug
RossDM 12y ago
Good idea but doesn't seem entirely accurate. This project[http://libraries.io/maven/org.bitbucket.b_c:jose4j http://libraries.io/maven/org.bitbucket.b_c:jose4j] was updated last week but Libraries.io says it hasn't been updated since last year.
- cheald 12y ago
  OTOH, I pushed a new version of http://libraries.io/rubygems/manticore http://libraries.io/rubygems/manticore on Sunday and it's found it. Maybe different repos have different update feeds or something?
rmetzler 12y ago
http://versioneye.com/ http://versioneye.com/ is similar to this and allows to monitor dependency files like pom.xml, Gemfile.lock, package.json, etc.
xixixao 12y ago
Are Go libraries split up more granular than npm packages? Surprises me Go has such a large number of projects listed.
- andrewnez 12y ago
  They are at the moment, Go doesn't have a central repository like npmjs.org, each project is just a url that you can "go get", I'm trying to find a good way to group them that keeps the full dependency graph intact.
lfj 12y ago
Searching "dancer" brought up related perl modules but not the primary one.
yellowbkpk 12y ago
One feature that would be helpful is to parse various formats of dependency specification (Node's package.json, pip's requirements.txt, Maven, etc.) and use that to build dependency lists so that I can (a) get notified of updates and (b) get a list of licenses for internal license audits.
- andrewnez 12y ago
  Yep, I'm working on that at the moment.
emmanueloga_ 12y ago
Open source discovery... makes me think of freshmeat, later renamed to freecode.com, no longer maitained since a while ago. And before freshmeat, sourceforge... it was always more interesting to me as a catalog than as a hosting provider.
kgc 12y ago
How is it that Go seems to be the most popular?

136 points by dhendo 12y ago

More details on how it works in this blog post: https://medium.com/@teabass/solving-open-source-discovery-db43a04cd9e7 https://medium.com/@teabass/solving-open-source-discovery-db...

dannypgh 12y ago

Browsing C++ and coming up mostly with nodejs libraries. http://libraries.io/search?languages=C%2B%2B&order=desc&sort=rank http://libraries.io/search?languages=C%2B%2B&order=desc&sort...

MordodeMaru 12y ago
You should check biicode (C++ deps manager) which will be compatible with VersionEye soon allowing you to know about updates in the libs you depend. https://www.biicode.com https://www.biicode.com
andrewnez 12y ago
Currently I'm only pulling data from package managers like rubygems and npm, I couldn't find a C++ one yet, although https://www.biicode.com/ https://www.biicode.com/ looks handy for that, I'll investigate.
- humanrebar 12y ago
  C and C++ is usually packaged with the OS's package manager, like dpkg, rpm, pacman, etc. The closest thing to rubygems might be Portage (1) or AUR (2). You might be able to search each of those for "c++". Or maybe peek into the constituent files for '.c' or '.cpp', '.cxx', etc. 1) http://en.wikipedia.org/wiki/Portage_%28software%29 http://en.wikipedia.org/wiki/Portage_%28software%29 2) https://wiki.archlinux.org/index.php/Arch_User_Repository https://wiki.archlinux.org/index.php/Arch_User_Repository
- emilsedgh 12y ago
  include [0] lists reusable Qt libraries. [0] http://inqlude.org http://inqlude.org

dom96 12y ago

Nice to see that even my package manager for the Nim programming language is there. Pity that the language is still called "Nimrod" there though.

andrewnez 12y ago
I'm using https://github.com/github/linguist https://github.com/github/linguist for language information, still waiting for this pr to be merged for the Nimrod => Nim change: https://github.com/github/linguist/pull/1936 https://github.com/github/linguist/pull/1936
- dom96 12y ago
  I see. I wonder what's taking them so long. In any case I am impressed that you implemented support for Nimble :)

guardian5x 12y ago

Looks very interesting. Can't seem to find Qt though, but lots of related things, like qt bindings for node.js

WimLeers 12y ago

I don't understand why Wordpress is considered a platform next to npm, bower, pypi etc.

stared 12y ago

It would be great if: - it sorted searches by GitHub starts (otherwise for any popular thing, you get first tons of stuff, in a non-relevant order), - for PyPI, there is also anther figure of merit, i.e. no of downloads (most are open source, but some are on other repositories), - in general, for Python it misses some very popular packages... for example, http://libraries.io/search?q=networkx http://libraries.io/search?q=networkx is from PyPI but not from Python (why?). Otherwise, a very needed idea! I wanted to do my own some time ago (http://pypi.meteor.com/ http://pypi.meteor.com/), but didn't have time to bring it to a useful (or nice) version.

stared 12y ago
And, for some packages clearly being on GitHub, it does not find their place, e.g. http://libraries.io/pypi/pandas http://libraries.io/pypi/pandas vs https://github.com/pydata/pandas https://github.com/pydata/pandas (4k+ stars).
andrewnez 12y ago
You can sort by stars, there's just not a UI yet: http://libraries.io/search?order=desc&sort=stars http://libraries.io/search?order=desc&sort=stars Pypi is lacking in information but I'm working on pulling it from other sources to improve the quality for Python libraries.
- stared 12y ago
  For PyPI it is rather straightforward to get downloads information: http://stackoverflow.com/questions/21419009/json-api-for-pypi-how-to-list-packages http://stackoverflow.com/questions/21419009/json-api-for-pyp... (along with some other data, e.g. categorization). See example: https://pypi.python.org/pypi/seaborn/json https://pypi.python.org/pypi/seaborn/json

mlhamel 12y ago

weird there's no not many python entries... where it seems there's way more than 16k package in it (ie: https://alexgaynor.net/2014/jan/03/pypi-download-statistics/ https://alexgaynor.net/2014/jan/03/pypi-download-statistics/)

deepakprakash 12y ago

Quick feedback: Browsing the Go section, each sub directory(at all depths) in the Docker project seems to be listed as a separate project. Leads to showing a few pages of results linking to the same project(ie, Docker). Also may explain why Go seems to have far more number of projects listed than anything else. :) Update: Better wording

amirmc 12y ago

A few of the OCaml projects are showing up under Standard ML [1]. A little disappointing since anyone using OCaml would really benefit from finding out about Merlin (see the link). This is due to a recent issue (or regression) with how GitHub's Linguist disambiguates code, which I hope will be fixed soon [2]. [1] http://libraries.io/search?languages=Standard+ML http://libraries.io/search?languages=Standard+ML [2] https://github.com/github/linguist/issues/2208 https://github.com/github/linguist/issues/2208

andrewnez 12y ago
At the moment we lean completely on linguist for language detection, which isn't perfect but gets us a good amount of the way there. Once we've got OPAM support more OCaml libraries will start to show up.

rane 12y ago

Why can't this library http://libraries.io/npm/debug http://libraries.io/npm/debug be found in this list? http://libraries.io/search?keywords=debug&platforms=NPM http://libraries.io/search?keywords=debug&platforms=NPM

andrewnez 12y ago
The keyword filtering is slightly borked at the moment, this does work though: http://libraries.io/search?platforms=NPM&q=debug http://libraries.io/search?platforms=NPM&q=debug

RossDM 12y ago

Good idea but doesn't seem entirely accurate. This project[http://libraries.io/maven/org.bitbucket.b_c:jose4j http://libraries.io/maven/org.bitbucket.b_c:jose4j] was updated last week but Libraries.io says it hasn't been updated since last year.

cheald 12y ago
OTOH, I pushed a new version of http://libraries.io/rubygems/manticore http://libraries.io/rubygems/manticore on Sunday and it's found it. Maybe different repos have different update feeds or something?

rmetzler 12y ago

http://versioneye.com/ http://versioneye.com/ is similar to this and allows to monitor dependency files like pom.xml, Gemfile.lock, package.json, etc.

xixixao 12y ago

Are Go libraries split up more granular than npm packages? Surprises me Go has such a large number of projects listed.

andrewnez 12y ago
They are at the moment, Go doesn't have a central repository like npmjs.org, each project is just a url that you can "go get", I'm trying to find a good way to group them that keeps the full dependency graph intact.

lfj 12y ago

Searching "dancer" brought up related perl modules but not the primary one.

yellowbkpk 12y ago

One feature that would be helpful is to parse various formats of dependency specification (Node's package.json, pip's requirements.txt, Maven, etc.) and use that to build dependency lists so that I can (a) get notified of updates and (b) get a list of licenses for internal license audits.

andrewnez 12y ago
Yep, I'm working on that at the moment.

emmanueloga_ 12y ago

Open source discovery... makes me think of freshmeat, later renamed to freecode.com, no longer maitained since a while ago. And before freshmeat, sourceforge... it was always more interesting to me as a catalog than as a hosting provider.

kgc 12y ago

How is it that Go seems to be the most popular?