5 ms·

Full source of 130k+ extensions: https://crx.dam.io/ https://crx.dam.io/ Github repo: https://github.com/mdamien/chrome-extensions-archive https://github.com/m

by dorianm 10y ago

Full source of 130k+ extensions: https://crx.dam.io/ https://crx.dam.io/ Github repo: https://github.com/mdamien/chrome-extensions-archive https://github.com/mdamien/chrome-extensions-archive

jonathansampson 10y ago
Holy smokes! That's exhaustive
carussell 10y ago
Neat! Fun fact: The main way to peruse the source of the various Mozilla projects[1] was to use MXR[2], and we[3] tried to get Mozilla Corp to index the extensions that are published through addons.mozilla.org, but they weren't willing to provide any support. This was a big problem because if you recall, every time a new Firefox release dropped, it would cause massive breakage because APIs would be obsoleted and removed within the same release cycle. As a Mozilla contributor, even if you wanted to bend over backwards and check which extensions you would be breaking with any given patch to mozilla-central (and maybe send fixes to the extension maintainers if you were really nice), there was no way to do it short of setting up a crawler yourself (to spider AMO) and either running an instance of some shiny code indexing tool yourself on localhost, or more likely just dumping all extensions' source off a subdirectory of $HOME and running a slow recursive grep every time you wanted to check on some symbol use. It would be hard to overstate how many patches never got written and upstreamed because of this. As a Mozilla contributor, you had the choice of either throwing in and piling on to the bonfire of wanton breakage, or quietly opting to take no action when running into instances where you could improve and even stabilize the underlying "platform". The former was actually an uneasy thing to think about, because it was clear that TPTB didn't give a damn about one of the single biggest driving forces behind Firefox user retention, which was the extension ecosystem[4]. Opting for the latter was "safer" because although it kept the codebase in a poorer state than it could have been in, it might stem the flow just enough so that the number of users bleeding out wasn't so bad as to cause Mozilla to lose its negotiating power. And when the future is still not guaranteed, you can hold out hope that maybe people will come around and start doing the right thing. Eventually, an index of add-on sources did get built and grafted onto MXR. Only problem is Mozilla allowed proprietary add-ons to be published on AMO, and whoever set it up chose to put those in the index, too. So they threw it behind an HTTP auth wall that you could only get through by logging in with your LDAP password. Just another example of the uphill battle it was to be working on Mozilla but not from behind an @mozilla.com email address. 1. For example, Firefox, SpiderMonkey, the web properties, and less popular client software like Thunderbird and SeaMonkey 2. MXR was a heavily modified version of the LXR tool, which itself was meant to make it less painful to browse and search the Linux kernel sources 3. "We" here just means the set of people who pushed for this over the years, not like a consolidated effort coming from a single team that I happened to be part of or anything 4. To some, this will probably read as run-of-the-mill lamentations of someone's pet use case not being prioritized and coddled, but it's not so. I'm not and never have been a heavy extension user, but a) it was stupid not to recognize that extensions were highly valued by both the vocal minority and silent majority that were responsible for the number of Firefox holdouts, and b) making an end-user-hackable browser was, like, the natural consequence to be working on if you were serious about all rhetoric the people from your camp were spewing about making something that really was aiming to be a "user agent" meant to "put end users in control" so they could "experience the web on their own terms". ---- Anyway, this got really long. That link to the repo of Chrome extensions is a nice thing. I'm on record as not being too much of a fan of GitHub, but it's neat how it enables something like this to exist, even if it still takes a lot of manual work to keep things reasonably up to date. If it had been clearer that git was going to win the DVCS battle and the timing were better so that cheap symbol lookup (in the form of GitHub's code search) were available in the timeframe when all this was taking place, then it would have been nice to throw something like this together for extensions in the Mozilla add-on ecosystem, back when it could have made a difference.
- gcp 10y ago
  IMHO your rant does boil down to the fact that making the browser "end-user-hackable" through extensions was an untenable situation. I mean, you're both saying "This was a big problem because if you recall, every time a new Firefox release dropped, it would cause massive breakage because APIs would be obsoleted and removed within the same release cycle." and "it was stupid not to recognize that extensions were highly valued by both the vocal minority and silent majority that were responsible for the number of Firefox holdouts, and b) making an end-user-hackable browser was, like, the natural consequence to be working on if you were serious about all rhetoric the people from your camp were spewing about making something that really was aiming to be a "user agent" meant to "put end users in control" so they could "experience the web on their own terms"." I don't buy the argument that a searchable source index is what made or broke this. For starters, the approach you describe obviously doesn't scale. It's fixing a leaking roof with buckets. Resources were in fact better spent elsewhere, so yes, it's exactly "run-of-the-mill lamentations of someone's pet use case not being prioritized and coddled". The idea that Firefox's add-on ecosystem was an asset rather than a liability did much to set the browser and the users' experience with it back for years, and it'll continue to do so as users lash out when their unrealistic expectations of XUL addons continuing to work are broken.
  - carussell 10y ago
    > making the browser "end-user-hackable" through extensions was an untenable situation This is in response to a blog post where extension support is being added. And those extensions exist in great numbers. Targeting the most popular browser in the world. And there's no sign that they are being deprecated. So how untenable are extensions? Is Chrome untenable? > I don't buy the argument that a searchable source index is what made or broke this. I'm very glad, then, that I didn't say that. > it's exactly "run-of-the-mill lamentations of someone's pet use case not being prioritized and coddled" I'm flummoxed at this comment. There's no way this is an instance of that, nor would it even be possible for it to be, because it's not even my use case. I think I made that clear enough in my original comment.
    - 10y ago

by dorianm 10y ago

Full source of 130k+ extensions: https://crx.dam.io/ https://crx.dam.io/ Github repo: https://github.com/mdamien/chrome-extensions-archive https://github.com/mdamien/chrome-extensions-archive

jonathansampson 10y ago

Holy smokes! That's exhaustive

carussell 10y ago

Neat! Fun fact: The main way to peruse the source of the various Mozilla projects[1] was to use MXR[2], and we[3] tried to get Mozilla Corp to index the extensions that are published through addons.mozilla.org, but they weren't willing to provide any support. This was a big problem because if you recall, every time a new Firefox release dropped, it would cause massive breakage because APIs would be obsoleted and removed within the same release cycle. As a Mozilla contributor, even if you wanted to bend over backwards and check which extensions you would be breaking with any given patch to mozilla-central (and maybe send fixes to the extension maintainers if you were really nice), there was no way to do it short of setting up a crawler yourself (to spider AMO) and either running an instance of some shiny code indexing tool yourself on localhost, or more likely just dumping all extensions' source off a subdirectory of $HOME and running a slow recursive grep every time you wanted to check on some symbol use. It would be hard to overstate how many patches never got written and upstreamed because of this. As a Mozilla contributor, you had the choice of either throwing in and piling on to the bonfire of wanton breakage, or quietly opting to take no action when running into instances where you could improve and even stabilize the underlying "platform". The former was actually an uneasy thing to think about, because it was clear that TPTB didn't give a damn about one of the single biggest driving forces behind Firefox user retention, which was the extension ecosystem[4]. Opting for the latter was "safer" because although it kept the codebase in a poorer state than it could have been in, it might stem the flow just enough so that the number of users bleeding out wasn't so bad as to cause Mozilla to lose its negotiating power. And when the future is still not guaranteed, you can hold out hope that maybe people will come around and start doing the right thing. Eventually, an index of add-on sources did get built and grafted onto MXR. Only problem is Mozilla allowed proprietary add-ons to be published on AMO, and whoever set it up chose to put those in the index, too. So they threw it behind an HTTP auth wall that you could only get through by logging in with your LDAP password. Just another example of the uphill battle it was to be working on Mozilla but not from behind an @mozilla.com email address. 1. For example, Firefox, SpiderMonkey, the web properties, and less popular client software like Thunderbird and SeaMonkey 2. MXR was a heavily modified version of the LXR tool, which itself was meant to make it less painful to browse and search the Linux kernel sources 3. "We" here just means the set of people who pushed for this over the years, not like a consolidated effort coming from a single team that I happened to be part of or anything 4. To some, this will probably read as run-of-the-mill lamentations of someone's pet use case not being prioritized and coddled, but it's not so. I'm not and never have been a heavy extension user, but a) it was stupid not to recognize that extensions were highly valued by both the vocal minority and silent majority that were responsible for the number of Firefox holdouts, and b) making an end-user-hackable browser was, like, the natural consequence to be working on if you were serious about all rhetoric the people from your camp were spewing about making something that really was aiming to be a "user agent" meant to "put end users in control" so they could "experience the web on their own terms". ---- Anyway, this got really long. That link to the repo of Chrome extensions is a nice thing. I'm on record as not being too much of a fan of GitHub, but it's neat how it enables something like this to exist, even if it still takes a lot of manual work to keep things reasonably up to date. If it had been clearer that git was going to win the DVCS battle and the timing were better so that cheap symbol lookup (in the form of GitHub's code search) were available in the timeframe when all this was taking place, then it would have been nice to throw something like this together for extensions in the Mozilla add-on ecosystem, back when it could have made a difference.

gcp 10y ago
IMHO your rant does boil down to the fact that making the browser "end-user-hackable" through extensions was an untenable situation. I mean, you're both saying "This was a big problem because if you recall, every time a new Firefox release dropped, it would cause massive breakage because APIs would be obsoleted and removed within the same release cycle." and "it was stupid not to recognize that extensions were highly valued by both the vocal minority and silent majority that were responsible for the number of Firefox holdouts, and b) making an end-user-hackable browser was, like, the natural consequence to be working on if you were serious about all rhetoric the people from your camp were spewing about making something that really was aiming to be a "user agent" meant to "put end users in control" so they could "experience the web on their own terms"." I don't buy the argument that a searchable source index is what made or broke this. For starters, the approach you describe obviously doesn't scale. It's fixing a leaking roof with buckets. Resources were in fact better spent elsewhere, so yes, it's exactly "run-of-the-mill lamentations of someone's pet use case not being prioritized and coddled". The idea that Firefox's add-on ecosystem was an asset rather than a liability did much to set the browser and the users' experience with it back for years, and it'll continue to do so as users lash out when their unrealistic expectations of XUL addons continuing to work are broken.
- carussell 10y ago
  > making the browser "end-user-hackable" through extensions was an untenable situation This is in response to a blog post where extension support is being added. And those extensions exist in great numbers. Targeting the most popular browser in the world. And there's no sign that they are being deprecated. So how untenable are extensions? Is Chrome untenable? > I don't buy the argument that a searchable source index is what made or broke this. I'm very glad, then, that I didn't say that. > it's exactly "run-of-the-mill lamentations of someone's pet use case not being prioritized and coddled" I'm flummoxed at this comment. There's no way this is an instance of that, nor would it even be possible for it to be, because it's not even my use case. I think I made that clear enough in my original comment.
  - 10y ago