17 ms·

Mozilla/web-ext: A command line tool to help build, run, and test web extensions

177 points by nkjoep 6y ago

politelemon 6y ago
> npm install --global web-ext Please don't install npm packages globally. Use npx. You can `npm install web-ext`, then run it with `npx web-ext arg1 arg2`. Or just run `npx web-ext arg1 arg2` without installing, npx will install it first (if the package name matches the command). So in the 'for your project' section, if you want to run without setting stuff up, npx web-ext run --source-dir ./extension-dist/
- tomashubelbauer 6y ago
  You never say why. I install tools I use often globally, use NPX for ad-hoc tools. NPX invocation is slightly slower and its more to type. Never had any problems with either.
  - Vinnl 6y ago
    The why is that an upgrade of the globally-installed tool will apply to all projects using that tool. So if a breaking change in web-ext would require changes to your project, then you'll have to apply those changes to all your projects at once - unless they each have their own instance of it.
    - tomashubelbauer 6y ago
      I see, I don't use any dependencies globally, but I do use global install for CLI tools I either invoke manually or use from scripts which I want to break if the tool CLI API changes so I can fix them and continue working on latest.
  - mrozbarry 6y ago
    Global packages are harder to maintain. Let's say you installed foobar version 5 globally last year. This year, you're using a newer version of the foobar package that isn't backwards compatible. You install it globally, which replaces the old version 5. Now if you go back to the old project, your use of foobar is broken. Depending on the package, it might be fairly invasive/difficult to simply update. For myself, I always install global-recommended packages as dev dependencies. It can be a little painful, but it also means there are no special install instructions, and it _just works_ (tm).
- bamboleo 6y ago
  Yes and no. I love npx but if you’re launching a package more often than once a week you should probably install it because it’s just faster to do so. Just… update your global packages occasionally too. I use `npx npm-check -u -g`
bamboleo 6y ago
Little known fact: web-ext not only can launch both Firefox and Chrome in a new profile with the extension preloaded, but it will also refresh them when any of the extensions’ files changes. It’s game changing.
- diggan 6y ago
  Heh, once you want even tighter feedback than just reloading the extension itself, try out developing a extension with ClojureScript ;) https://github.com/binaryage/chromex https://github.com/binaryage/chromex is a good starting point if you're curious.
- felixfbecker 6y ago
  Not really. I would prefer to keep my profile which has all my history, sessions etc instead of launching a new Firefox every time. Reloading extensions is really not that hard, especially with something like https://chrome.google.com/webstore/detail/extensions-reloader/fimgfedafeadlieiabdeeaodndnlbhid https://chrome.google.com/webstore/detail/extensions-reloade...
  - diggan 6y ago
    I'm sorry, but I seem to not understand how that extension is better than web-ext. web-ext reloads the extension automatically when it changes on disk, "Extensions Reloader" seems to require you to either click a button to reload, or setup your build system to redirect current browser session to "http://reload.extensions http://reload.extensions", both inferior solutions to just using web-ext.
  - bamboleo 6y ago
    Me too, in fact it has a —preserve-changes flag (or something like that) that combined with the -p flag let’s you have a profile just for development.
- jerrygoyal 6y ago
  > it will also refresh them when any of the extensions’ files changes. I don't see that in the docs. would it support webpack? for now, I'm using webpack-extension-reloader plugin for an extension I built for Notion.so and it works (reload) for chrome. https://github.com/GorvGoyl/Notion-Boost-browser-extension https://github.com/GorvGoyl/Notion-Boost-browser-extension
pimterry 6y ago
I note this 'aims' to support browser extensions in a cross-platform way, but it's primarily focused on Firefox. Does anybody know how far it is from that aim? Is it useful at all if you're trying to build Chrome or Safari or all-3 webextensions?
- karlicoss 6y ago
  Absolutely, I'm using it to build an extension both for Chrome and Firefox [0]. Perhaps the only Firefox-specific thing is 'sign' command which signs for addonz.mozilla.org [0] https://github.com/karlicoss/promnesia/blob/master/extension/build https://github.com/karlicoss/promnesia/blob/master/extension...
oefrha 6y ago
Last time I tried to test an extension on Firefox, with or without web-ext, two things of note: 1. yarn add web-ext fails with a bunch of errors on macOS 10.15, but npm install works. Not sure why. 2. The chrome.storage (or is it only storage.sync? can’t recall) API doesn’t work when the extension is loaded temporarily (since there’s no stable extension ID assigned or something), which is how `web-ext run` loads the extension, and the only way I could find to test the extension without submitting it to Mozilla for signing. Yeah I tweaked all the related settings in Firefox Developer Edition but couldn’t for the life of me figure out how to non-temporarily install my unsigned extension. So in the end I just gave up on testing the part of the extension that used chrome.storage, which was like 75% of the extension. A really hostile experience that will make sure I don’t port my extensions to Firefox in the future.
- karlicoss 6y ago
  I think it's storage.sync. I get a warning about it too, but just ignored it so far. I think it's just the actual sync that doesn't work (settings still get set and loaded locally).
  - oefrha 6y ago
    Okay, I pulled out the code and ran it on Firefox again to jog my memory. This is the error when doing anything with chrome.storage.sync: Error: The storage API will not work with a temporary addon ID. Please add an explicit addon ID to your manifest. For more information see https://bugzil.la/1323228. It's an error, not a warning, and set/get definitely don't work. If I replace all storage.sync invocations with storage.local then it's okay, not error/warning, and setting/getting seems to work. So the problem is limited to storage.sync, but it's not just a warning. Edit: now that I googled the problem again, it seems adding an explicit addon ID according to https://extensionworkshop.com/documentation/develop/extensions-and-the-add-on-id/ https://extensionworkshop.com/documentation/develop/extensio... should work. I don't recall what I did back then, but I definitely did quite a few things to no avail.
    - karlicoss 6y ago
pyepye 6y ago
I've been using webextension-toolbox[1] for a few years now to develop cross-platform extensions and it's worked really well. I always got the feeling other browsers other than Firefox were Second-class citizens with web-ext (if supported at all) whereas with webextension-toolbox they all appeared to work out of the box. The first time I tested my extension on Firefox after developing it one Chrome for months it worked the first time. I was so shocked how seamless it was I actually thought something must be really wrong, it wasn't. [1] https://github.com/webextension-toolbox/webextension-toolbox https://github.com/webextension-toolbox/webextension-toolbox
- AlexITC 6y ago
  While I have the same feeling about Firefox, it still lacks supporting some APIs, the Web Push API being the latest I missed that works properly on Chrome, enabling real push notifications.
idoubtit 6y ago
In my opinion, Firefox imposes complex constraints on casual developers, and web-ext is part of it. I have a few web extensions I wrote in order to customize a few web sites (automate tasks, alter display, etc). After a Firefox update, they were not allowed anymore, because they're not signed. The extensions page cannot load local extensions. Then I discovered the secret debug-extensions page of Firefox where I could load them. Unfortunately, Firefox removes local extension when it closes. I had the choice between loading each local extension after launching a new Firefox, which mean a dozen of clicks, or installing web-ext in order to sign my extensions, and sign them again after each change (no idea if there were extra actions needed in the process, like manual reload). I opted for a much simple option: with Chromium-based browsers, you can enable the extension-developer mode, and then you're allowed to load local extensions from the normal extensions page. Extensions are kept across restarts.
- badsectoracula 6y ago
  I had the same issue but i solved it by using Firefox Developer edition instead which allows you to install unsigned extensions. This allowed me to use my Firefox profile (by manually copying the files over the existing profile files) which i have going back several years now. If that ever changes i'll look into making my own Firefox binaries or whatever that have this check removed.
  - diegocg 6y ago
    Unbranded stable (and beta) builds also allows disabling the signature check. There are plenty of options, just not in the official stable builds.
    - iggldiggl 6y ago
      And the long-term support (ESR) version as well, if I remember correctly.
    - badsectoracula 6y ago
      That makes things even easier then :-P
somehnrdr14726 6y ago
This is a lovely tool, you can also implement automated builds and deploys using Github Actions to drive web-ext. Chrome is pushing for manifest v3 and not seeking buy-in from the other web extension standard bearers. Also the new Firefox Mobile has incredibly spotty support for the API. Given these recent events, I worry that the cross-browser nature of this tool will be lost in time. And web extension development in general may be entering a dark age.
- danuker 6y ago
  There will always be Greasemonkey & co.
- toyg 6y ago
  > And web extension development in general may be entering a dark age. Was it ever not in a dark age? The overlap between FF and Chrome extension tech has actually existed for what, two years or so...?
- dblohm7 6y ago
  To be clear: Chrome was never part of any WebExtension standard. They do their own thing and WebExtension browsers decide which (if any) changes they want to support.
ropeladder 6y ago
Thanks for posting this and for everyone else adding similar suggestions. I dove into web extension development earlier this year and couldn't find anything at all on automating testing. It was hard to Google testing for web extensions instead of web extensions used for testing. Couldn't find anything in mozilla's otherwise excellent docs either.
- visarga 6y ago
  > It was hard to Google testing for web extensions instead of web extensions used for testing. Off-topic: didn't Google recently announce they were using transformers for almost all English language queries? I tried "testing for web extensions" in Google and got: FAIL - 8 Best Chrome Extensions for QA Testing (featured result) OK - Testing your web extension with integration tests OK - How to test web extensions? So the featured result was bad but results one and two were good.
toyg 6y ago
I hit this yesterday and I was very annoyed by it. - The npm install fails on Windows if you don't have Visual Studio 2017+ installed. How is this acceptable for a utility that will mostly do some simple text-mangling? - Mozilla is the birthplace of Rust. Why do I have to install Node to get a pretty simple utility like this? This should be a basic Rust executable you can just drop anywhere, no installation required. - the very first line of `npm install web-ext` is a warning that some library is obsolete and should not be used. It's promptly followed by tons of similar ones. Again, how is this acceptable in 2020 for something so basic? In the end I just dropped it. Web-ext is not doing machine-learning billion-pageview-crunching rocket-launching stuff. It's a utility to roll up a few text files. The fact that it was built with Node seems yet another sign that Mozilla is dominated by SFBA cargo-cultism.
- diggan 6y ago
  > mostly do some simple text-mangling If you call communication with two different browsers (at least), validating source code, signing code and packing it up "simple text-mangling", I understand you're a bit frustrated. But I'm quite sure many see those features as a bit more than that, hence it is quite a popular tool to use in conjunction with browser extensions. > Why do I have to install Node Because web developers who do JavaScript for a living find it easier to write JavaScript to support their developer tools. Not a huge surprise here. I'm sure if you find the time to write the "simple text-mangling" in Rust, support it and maintain it, people would be more than happy to use it. But why rewrite something that works well enough for the intended users? Speaking of cargo-cultism, you seem to have a bit of that stench on yourself as well, where suddenly every utility "should of course" be written in Rust, or otherwise be rewritten in Rust if they are not doing it yet. Programmers will scratch their own itch with the languages that are familiar to them. Of course web extension dev tools will be written in languages and platforms that are the closest to the web ecosystem, which today is JavaScript.
  - toyg 6y ago
    > communication with two different browsers (at least), validating source code, signing code and packing it up ... is not something that requires a massive engine like Node. It just isn't. > Because web developers who do JavaScript for a living find it easier to write JavaScript This is an argument I can accept from a random free-time-opensource developer, not from a well-funded behemoth like Mozilla. An org that big should pay attention to what their objectives are, not what they feel like hacking together on any given day. How can they not see that making this sort of tool significantly burdensome to install on 90% of desktops is bound to limit its impact? > suddenly every utility "should of course" be written in Rust No. Write it in Go if you prefer. Write it in brainfuck, I don't care. What I care about, as a user, is that I can just get a single executable I can run on my machine. I care about usability, that's all I care about. How can it be acceptable for a tool like web-ext to have a setup story that is "figure out how to install npm; npm install web-ext and see it fail; figure out how to install Visual Studio (will it even work with the free version? I hope so); re-do npm install web-ext, ignoring a bunch of warnings about everything being horribly insecure and unmaintained"...? And then they have the gall to say in their copy that "you only need a text editor", just before pushing this crap. I mentioned Rust only because it's terrible optics that its very home doesn't dogfood it. Again, not rocket science to notice, at organizational level. As a fervent Firefox advocate, this is the sort of thing that really riles me up.
gabrielsroka 6y ago
For more information and tools, see https://youtu.be/6jgBM8twUIQ https://youtu.be/6jgBM8twUIQ https://github.com/mozilla/webextension-polyfill https://github.com/mozilla/webextension-polyfill
dcgudeman 6y ago
I wonder why they chose to go with flow over typescript.

177 points by nkjoep 6y ago

> npm install --global web-ext Please don't install npm packages globally. Use npx. You can `npm install web-ext`, then run it with `npx web-ext arg1 arg2`. Or just run `npx web-ext arg1 arg2` without installing, npx will install it first (if the package name matches the command). So in the 'for your project' section, if you want to run without setting stuff up, npx web-ext run --source-dir ./extension-dist/

tomashubelbauer 6y ago
You never say why. I install tools I use often globally, use NPX for ad-hoc tools. NPX invocation is slightly slower and its more to type. Never had any problems with either.
- Vinnl 6y ago
  The why is that an upgrade of the globally-installed tool will apply to all projects using that tool. So if a breaking change in web-ext would require changes to your project, then you'll have to apply those changes to all your projects at once - unless they each have their own instance of it.
  - tomashubelbauer 6y ago
    I see, I don't use any dependencies globally, but I do use global install for CLI tools I either invoke manually or use from scripts which I want to break if the tool CLI API changes so I can fix them and continue working on latest.
- mrozbarry 6y ago
  Global packages are harder to maintain. Let's say you installed foobar version 5 globally last year. This year, you're using a newer version of the foobar package that isn't backwards compatible. You install it globally, which replaces the old version 5. Now if you go back to the old project, your use of foobar is broken. Depending on the package, it might be fairly invasive/difficult to simply update. For myself, I always install global-recommended packages as dev dependencies. It can be a little painful, but it also means there are no special install instructions, and it _just works_ (tm).
bamboleo 6y ago
Yes and no. I love npx but if you’re launching a package more often than once a week you should probably install it because it’s just faster to do so. Just… update your global packages occasionally too. I use `npx npm-check -u -g`

bamboleo 6y ago

Little known fact: web-ext not only can launch both Firefox and Chrome in a new profile with the extension preloaded, but it will also refresh them when any of the extensions’ files changes. It’s game changing.

diggan 6y ago
Heh, once you want even tighter feedback than just reloading the extension itself, try out developing a extension with ClojureScript ;) https://github.com/binaryage/chromex https://github.com/binaryage/chromex is a good starting point if you're curious.
felixfbecker 6y ago
Not really. I would prefer to keep my profile which has all my history, sessions etc instead of launching a new Firefox every time. Reloading extensions is really not that hard, especially with something like https://chrome.google.com/webstore/detail/extensions-reloader/fimgfedafeadlieiabdeeaodndnlbhid https://chrome.google.com/webstore/detail/extensions-reloade...
- diggan 6y ago
  I'm sorry, but I seem to not understand how that extension is better than web-ext. web-ext reloads the extension automatically when it changes on disk, "Extensions Reloader" seems to require you to either click a button to reload, or setup your build system to redirect current browser session to "http://reload.extensions http://reload.extensions", both inferior solutions to just using web-ext.
- bamboleo 6y ago
  Me too, in fact it has a —preserve-changes flag (or something like that) that combined with the -p flag let’s you have a profile just for development.
jerrygoyal 6y ago
> it will also refresh them when any of the extensions’ files changes. I don't see that in the docs. would it support webpack? for now, I'm using webpack-extension-reloader plugin for an extension I built for Notion.so and it works (reload) for chrome. https://github.com/GorvGoyl/Notion-Boost-browser-extension https://github.com/GorvGoyl/Notion-Boost-browser-extension

pimterry 6y ago

I note this 'aims' to support browser extensions in a cross-platform way, but it's primarily focused on Firefox. Does anybody know how far it is from that aim? Is it useful at all if you're trying to build Chrome or Safari or all-3 webextensions?

karlicoss 6y ago
Absolutely, I'm using it to build an extension both for Chrome and Firefox [0]. Perhaps the only Firefox-specific thing is 'sign' command which signs for addonz.mozilla.org [0] https://github.com/karlicoss/promnesia/blob/master/extension/build https://github.com/karlicoss/promnesia/blob/master/extension...

oefrha 6y ago

Last time I tried to test an extension on Firefox, with or without web-ext, two things of note: 1. yarn add web-ext fails with a bunch of errors on macOS 10.15, but npm install works. Not sure why. 2. The chrome.storage (or is it only storage.sync? can’t recall) API doesn’t work when the extension is loaded temporarily (since there’s no stable extension ID assigned or something), which is how `web-ext run` loads the extension, and the only way I could find to test the extension without submitting it to Mozilla for signing. Yeah I tweaked all the related settings in Firefox Developer Edition but couldn’t for the life of me figure out how to non-temporarily install my unsigned extension. So in the end I just gave up on testing the part of the extension that used chrome.storage, which was like 75% of the extension. A really hostile experience that will make sure I don’t port my extensions to Firefox in the future.

karlicoss 6y ago
I think it's storage.sync. I get a warning about it too, but just ignored it so far. I think it's just the actual sync that doesn't work (settings still get set and loaded locally).
- oefrha 6y ago
  Okay, I pulled out the code and ran it on Firefox again to jog my memory. This is the error when doing anything with chrome.storage.sync: Error: The storage API will not work with a temporary addon ID. Please add an explicit addon ID to your manifest. For more information see https://bugzil.la/1323228. It's an error, not a warning, and set/get definitely don't work. If I replace all storage.sync invocations with storage.local then it's okay, not error/warning, and setting/getting seems to work. So the problem is limited to storage.sync, but it's not just a warning. Edit: now that I googled the problem again, it seems adding an explicit addon ID according to https://extensionworkshop.com/documentation/develop/extensions-and-the-add-on-id/ https://extensionworkshop.com/documentation/develop/extensio... should work. I don't recall what I did back then, but I definitely did quite a few things to no avail.
  - karlicoss 6y ago

pyepye 6y ago

I've been using webextension-toolbox[1] for a few years now to develop cross-platform extensions and it's worked really well. I always got the feeling other browsers other than Firefox were Second-class citizens with web-ext (if supported at all) whereas with webextension-toolbox they all appeared to work out of the box. The first time I tested my extension on Firefox after developing it one Chrome for months it worked the first time. I was so shocked how seamless it was I actually thought something must be really wrong, it wasn't. [1] https://github.com/webextension-toolbox/webextension-toolbox https://github.com/webextension-toolbox/webextension-toolbox

AlexITC 6y ago
While I have the same feeling about Firefox, it still lacks supporting some APIs, the Web Push API being the latest I missed that works properly on Chrome, enabling real push notifications.

idoubtit 6y ago

In my opinion, Firefox imposes complex constraints on casual developers, and web-ext is part of it. I have a few web extensions I wrote in order to customize a few web sites (automate tasks, alter display, etc). After a Firefox update, they were not allowed anymore, because they're not signed. The extensions page cannot load local extensions. Then I discovered the secret debug-extensions page of Firefox where I could load them. Unfortunately, Firefox removes local extension when it closes. I had the choice between loading each local extension after launching a new Firefox, which mean a dozen of clicks, or installing web-ext in order to sign my extensions, and sign them again after each change (no idea if there were extra actions needed in the process, like manual reload). I opted for a much simple option: with Chromium-based browsers, you can enable the extension-developer mode, and then you're allowed to load local extensions from the normal extensions page. Extensions are kept across restarts.

badsectoracula 6y ago
I had the same issue but i solved it by using Firefox Developer edition instead which allows you to install unsigned extensions. This allowed me to use my Firefox profile (by manually copying the files over the existing profile files) which i have going back several years now. If that ever changes i'll look into making my own Firefox binaries or whatever that have this check removed.
- diegocg 6y ago
  Unbranded stable (and beta) builds also allows disabling the signature check. There are plenty of options, just not in the official stable builds.
  - iggldiggl 6y ago
    And the long-term support (ESR) version as well, if I remember correctly.
  - badsectoracula 6y ago
    That makes things even easier then :-P

somehnrdr14726 6y ago

This is a lovely tool, you can also implement automated builds and deploys using Github Actions to drive web-ext. Chrome is pushing for manifest v3 and not seeking buy-in from the other web extension standard bearers. Also the new Firefox Mobile has incredibly spotty support for the API. Given these recent events, I worry that the cross-browser nature of this tool will be lost in time. And web extension development in general may be entering a dark age.

danuker 6y ago
There will always be Greasemonkey & co.
toyg 6y ago
> And web extension development in general may be entering a dark age. Was it ever not in a dark age? The overlap between FF and Chrome extension tech has actually existed for what, two years or so...?
dblohm7 6y ago
To be clear: Chrome was never part of any WebExtension standard. They do their own thing and WebExtension browsers decide which (if any) changes they want to support.

ropeladder 6y ago

Thanks for posting this and for everyone else adding similar suggestions. I dove into web extension development earlier this year and couldn't find anything at all on automating testing. It was hard to Google testing for web extensions instead of web extensions used for testing. Couldn't find anything in mozilla's otherwise excellent docs either.

visarga 6y ago
> It was hard to Google testing for web extensions instead of web extensions used for testing. Off-topic: didn't Google recently announce they were using transformers for almost all English language queries? I tried "testing for web extensions" in Google and got: FAIL - 8 Best Chrome Extensions for QA Testing (featured result) OK - Testing your web extension with integration tests OK - How to test web extensions? So the featured result was bad but results one and two were good.

toyg 6y ago

I hit this yesterday and I was very annoyed by it. - The npm install fails on Windows if you don't have Visual Studio 2017+ installed. How is this acceptable for a utility that will mostly do some simple text-mangling? - Mozilla is the birthplace of Rust. Why do I have to install Node to get a pretty simple utility like this? This should be a basic Rust executable you can just drop anywhere, no installation required. - the very first line of `npm install web-ext` is a warning that some library is obsolete and should not be used. It's promptly followed by tons of similar ones. Again, how is this acceptable in 2020 for something so basic? In the end I just dropped it. Web-ext is not doing machine-learning billion-pageview-crunching rocket-launching stuff. It's a utility to roll up a few text files. The fact that it was built with Node seems yet another sign that Mozilla is dominated by SFBA cargo-cultism.

diggan 6y ago
> mostly do some simple text-mangling If you call communication with two different browsers (at least), validating source code, signing code and packing it up "simple text-mangling", I understand you're a bit frustrated. But I'm quite sure many see those features as a bit more than that, hence it is quite a popular tool to use in conjunction with browser extensions. > Why do I have to install Node Because web developers who do JavaScript for a living find it easier to write JavaScript to support their developer tools. Not a huge surprise here. I'm sure if you find the time to write the "simple text-mangling" in Rust, support it and maintain it, people would be more than happy to use it. But why rewrite something that works well enough for the intended users? Speaking of cargo-cultism, you seem to have a bit of that stench on yourself as well, where suddenly every utility "should of course" be written in Rust, or otherwise be rewritten in Rust if they are not doing it yet. Programmers will scratch their own itch with the languages that are familiar to them. Of course web extension dev tools will be written in languages and platforms that are the closest to the web ecosystem, which today is JavaScript.
- toyg 6y ago
  > communication with two different browsers (at least), validating source code, signing code and packing it up ... is not something that requires a massive engine like Node. It just isn't. > Because web developers who do JavaScript for a living find it easier to write JavaScript This is an argument I can accept from a random free-time-opensource developer, not from a well-funded behemoth like Mozilla. An org that big should pay attention to what their objectives are, not what they feel like hacking together on any given day. How can they not see that making this sort of tool significantly burdensome to install on 90% of desktops is bound to limit its impact? > suddenly every utility "should of course" be written in Rust No. Write it in Go if you prefer. Write it in brainfuck, I don't care. What I care about, as a user, is that I can just get a single executable I can run on my machine. I care about usability, that's all I care about. How can it be acceptable for a tool like web-ext to have a setup story that is "figure out how to install npm; npm install web-ext and see it fail; figure out how to install Visual Studio (will it even work with the free version? I hope so); re-do npm install web-ext, ignoring a bunch of warnings about everything being horribly insecure and unmaintained"...? And then they have the gall to say in their copy that "you only need a text editor", just before pushing this crap. I mentioned Rust only because it's terrible optics that its very home doesn't dogfood it. Again, not rocket science to notice, at organizational level. As a fervent Firefox advocate, this is the sort of thing that really riles me up.

gabrielsroka 6y ago

For more information and tools, see https://youtu.be/6jgBM8twUIQ https://youtu.be/6jgBM8twUIQ https://github.com/mozilla/webextension-polyfill https://github.com/mozilla/webextension-polyfill

dcgudeman 6y ago

I wonder why they chose to go with flow over typescript.