4 ms·

ES modules: A cartoon deep-dive (2018)

102 points by luu 3y ago

tuukkah 3y ago
(2018) Search for "live bindings" if you don't know what difference those make between require and import. Also explained here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
waveywaves 3y ago
Nice pictures
dickfickling 3y ago
If anybody has tried ES modules in the past five years and had to turn back due to incompatibility / lack of library support / javascript is a minefield, I'd urge you to try again. I managed to migrate our Typescript/Node codebase over to ESM about a month ago, with only a few hiccups. It's nice to be able to use the latest versions of all of sindresorhus's packages again.
- herpdyderp 3y ago
  I'd be curious to learn more, got any tips? I still export all my Node.js packages in CommonJS so they're usable and I'd love to switch over as well. (I'm already pure ESM in the frontend at least.)
  - WorldMaker 3y ago
    I suggest just adding "type": "module" to your package.json, then you just need to learn the ESM parts of the new "exports" syntax in package.json, ripping the band-aid off, and only shipping ESM today. That limits the low-end version of Node you can support with your package, but that water line is already now below LTS.
    - herpdyderp 3y ago
      > "type": "module" I've tried using this in the past and it didn't go well, so I suppose support is better these days. I'll give it another shot! Thanks for the tips!
      - WorldMaker 3y ago
        One trick that works better now than at first is that if you need to fallback to CJS for a config file for a CLI tool because it doesn't yet understand the ESM loaders in Node you can just give individual CJS files the .cjs file extension. I like ESM being the default for .js (which is what "type": "module" mostly does) in a project and then using .cjs files sparingly when necessary (which seems to be mostly just config files for build-time/dev-time tooling with older CLIs today using deprecated loading APIs). That better reflects which is the "present" of JS rather than its past and no need to worry ever about the .mjs bandaid file extension.
  - dickfickling 3y ago
    Basically what WorldMaker said. We updated our tsconfig and package.json, updated all our imports to include ".js", and fixed/upgraded things as necessary. The only real hiccup was that `require.main === module` no longer works, so we had to write some workaround code to determine whether a script was being run directly.
- mattwad 3y ago
  If you have one build pipeline, I would recommend it. But it's still been a nightmare for me. Key issues are some packages are 5+ yrs old and not maintained, and we have a very complicated setup, one common repo shared by a Next.js app and custom Koa app. Every tool you work with, like Jest, needs its own workarounds. I have noticed more and more libs are exporting ESM, but lord knows when we can stop adding special compiler rules.
  - herpdyderp 3y ago
    dependencies that we need which are still CJS-only are what kill our ability to switch completely to ESM :(
    - WorldMaker 3y ago
      You can use Parcel or esbuild just on that CJS dependency (rather than as a top-level bundler for your whole project), and then cache the ESM result somewhere. If the package is that old and that unmaintained you can just about cache those ESM builds indefinitely. (That's what Vite kind of does under the hood. That's what snowpack used to do.) I think npm should probably support doing that at install time. ESM importing CJS works in Node, mostly, now, but it does have quite a bit of runtime overhead and prebaking it would be good. Especially because it is unlikely to ever see a CJS loader in the browser (and that would be awful if it did exist).
    - no_wizard 3y ago
      if you use a package manager like `pnpm` you can use the `.pnpmfile.cjs`[0] to intercept packages and add `"type": "commonjs"` to the package.json. This tells node that it needs to load it as a CommonJS package and should work fine with ESM. There's also creating a require function from `node:module` package[1] [0]: https://pnpm.io/pnpmfile https://pnpm.io/pnpmfile [1]: https://nodejs.org/dist/latest-v18.x/docs/api/module.html#modulecreaterequirefilename https://nodejs.org/dist/latest-v18.x/docs/api/module.html#mo...
  - paulddraper 3y ago
    Kinda niche, I wrote Bazel rules (https://github.com/rivethealth/rules_javascript https://github.com/rivethealth/rules_javascript), and with a build setting `--//javascript:module`. Jest tests automatically use the CJS version. Webpack builds use the ESM version. (And all my stuff is in TypeScript, so it needs a build step anyway.)
- Dylan16807 3y ago
  We're still getting there. Firefox only got the code to allow workers to use modules a couple months ago, and it's currently disabled by default. I currently have some very exciting code that dumps constructors to text form and sends them to a worker to run...
  - lioeters 3y ago
    > dumps constructors to text form and sends them to a worker Oh do tell more, that sounds devious.
    - Dylan16807 3y ago
      It's mostly a series of `foo=${JSON.stringify(foo)};` and Bar.prototype.constructor.toString(). There's one object that didn't cooperate with either of those, but it's only used in a very limited way, so I did a quick hack to pack its prototype up. At the end is a function that sets up self.onmessage, and a line that executes that function. Then I do .join('\r\n') on the mess, turn it into a Blob, and pass the URL of the blob into new Worker(). The code running in the worker is only a few classes so it's not worth the hassle of setting up code-copying workarounds or avoiding modules for that portion of the code. It'll be nice when I can cut that and have nothing more than a few imports and self.onmessage.
      - lioeters 3y ago
        I see, a wonderous hack. Sounds like something a build tool/plugin or code transformation step could make it work magically behind the scenes. It's cool and also a bit horrifying (haha) that this is possible dynamically, to stringify a function and send it over to a worker thread to run. After all, code is data..
- akira2501 3y ago
  They may be fine.. but nodejs still makes working with them at the cli particularly inconvenient for me. I'm sure there's rational reasons they've made the choices they did, like import being async and which lexical context you're in being significant, but they create an inconvenient mess when I'm trying to test and refactor code around. Meanwhile, import cycles are easy to design around and to fix with a single interstitial module if you need it, and "live value" exporting has never been an impediment to me. You can export objects, and their properties are "live" enough for me. Outside of one specific problem in browsers, I'm not sure what ES6 is actually supposed to buy me. I'm still trying to figure out how to turn of ts-ls "File is a CommonJS module; it may be converted to ES6" disagnostic.
- Karrot_Kream 3y ago
  I'm new to the Node ecosystem but still find it a big pain to get ES Modules working properly with Typescript and `ts-node`, and only a minor pair to get it to work with a Typescript build/run workflow. I have no problems with this feature being experimental but I wish the ecosystem acknowledged it more in documentation.
  - herpdyderp 3y ago
    try giving tsx a try instead of ts-node: http://npmjs.com/tsx http://npmjs.com/tsx
    - loh 3y ago
      Thanks for this. Just a few minutes ago I pulled up an old project using `ts-node` with ESM and tried to run it on a new machine, had some issues and remembered reading your comment here earlier. I switched to `tsx` and in less than a minute everything was working beautifully. I'll probably migrate all my TS projects to `tsx`.
imbnwa 3y ago
Node should've offered an extended LTS support for a final CJS supported release and just ripped the band-aid off, making ESM first-class for all subsequent releases. As Mike Ehrmentraut once said, "no more half-measures".
- rgoulter 3y ago
  I think the pain that the python 2 to python 3 upgrade caused would discourage this. I also see many people complained about the removal of the long-deprecated egrep and fgrep commands.
- Chyzwar 3y ago
  The problem is that for many use cases, ESM is not feature complete. There are important features that are under experimental flags or broken in node.js. node.js released broken, backwards incompatible version of ESM with new resolution algorithm and is waiting for everyone to figure out workarounds for theirs shitty implementation. [jest ESM support] https://github.com/jestjs/jest/issues/9430 [typescript extensions] https://github.com/microsoft/TypeScript/issues/16577 [typescript exports] https://github.com/microsoft/TypeScript/issues/33079 [node.js issues] https://github.com/nodejs/node/issues?q=is%3Aissue+is%3Aopen+sort%3Acomments-desc+label%3Aesm Only in the from few months ago, tooling support is close enough to make it usable.
rektide 3y ago
Notably no browser has we workers or service workers of any sort which are ESM capable. Fuck a duck.
ElfinTrousers 3y ago
Modules: one of the great programming innovations of the 1960s, first available in JS just fifty years later.
- ixtli 3y ago
  its my experience that you can say this for most things the non-academic community does. this isn't a criticism of them or you for saying it, but its p constant. the worst offenders in my experience are AAA game devs. ive had so many conversations where they imply they've solved some deep computing problem that was formally solved in 1954 by a set theorist or linguist heh
- hombre_fatal 3y ago
  This doesn't really make sense outside of "JS suxx amirite fellas?" JS had modules before ESM. ES modules try to solve different problems and offer new solutions with browsers fetching modules themselves, something other module systems don't even do. Whether a language has modules or not is just a design / ecosystem decision, not a technological one. Swift is a modern language (2014) that doesn't even have modules comparable to what Node.js introduced a 14 years ago. Yet it does little to hamper the quality of apps people are building for the Apple ecosystem.
  - deleted 3y ago
    [deleted]
- paulryanrogers 3y ago
  Considering JS didn't exist until the mid 90's, and it's usage was quite limited until the mid oughts, that's not surprising.
  - ElfinTrousers 3y ago
    I'd say not so much because of the timing but because JS was only meant to be a crude stopgap: the scripting language they'd use for a year or two and then replace. The rest is history, the history of a huge sector of our industry built on a hilariously inadequate foundation.

102 points by luu 3y ago

(2018) Search for "live bindings" if you don't know what difference those make between require and import. Also explained here: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

waveywaves 3y ago

Nice pictures

dickfickling 3y ago

If anybody has tried ES modules in the past five years and had to turn back due to incompatibility / lack of library support / javascript is a minefield, I'd urge you to try again. I managed to migrate our Typescript/Node codebase over to ESM about a month ago, with only a few hiccups. It's nice to be able to use the latest versions of all of sindresorhus's packages again.

herpdyderp 3y ago
I'd be curious to learn more, got any tips? I still export all my Node.js packages in CommonJS so they're usable and I'd love to switch over as well. (I'm already pure ESM in the frontend at least.)
- WorldMaker 3y ago
  I suggest just adding "type": "module" to your package.json, then you just need to learn the ESM parts of the new "exports" syntax in package.json, ripping the band-aid off, and only shipping ESM today. That limits the low-end version of Node you can support with your package, but that water line is already now below LTS.
  - herpdyderp 3y ago
    > "type": "module" I've tried using this in the past and it didn't go well, so I suppose support is better these days. I'll give it another shot! Thanks for the tips!
    - WorldMaker 3y ago
      One trick that works better now than at first is that if you need to fallback to CJS for a config file for a CLI tool because it doesn't yet understand the ESM loaders in Node you can just give individual CJS files the .cjs file extension. I like ESM being the default for .js (which is what "type": "module" mostly does) in a project and then using .cjs files sparingly when necessary (which seems to be mostly just config files for build-time/dev-time tooling with older CLIs today using deprecated loading APIs). That better reflects which is the "present" of JS rather than its past and no need to worry ever about the .mjs bandaid file extension.
- dickfickling 3y ago
  Basically what WorldMaker said. We updated our tsconfig and package.json, updated all our imports to include ".js", and fixed/upgraded things as necessary. The only real hiccup was that `require.main === module` no longer works, so we had to write some workaround code to determine whether a script was being run directly.
mattwad 3y ago
If you have one build pipeline, I would recommend it. But it's still been a nightmare for me. Key issues are some packages are 5+ yrs old and not maintained, and we have a very complicated setup, one common repo shared by a Next.js app and custom Koa app. Every tool you work with, like Jest, needs its own workarounds. I have noticed more and more libs are exporting ESM, but lord knows when we can stop adding special compiler rules.
- herpdyderp 3y ago
  dependencies that we need which are still CJS-only are what kill our ability to switch completely to ESM :(
  - WorldMaker 3y ago
    You can use Parcel or esbuild just on that CJS dependency (rather than as a top-level bundler for your whole project), and then cache the ESM result somewhere. If the package is that old and that unmaintained you can just about cache those ESM builds indefinitely. (That's what Vite kind of does under the hood. That's what snowpack used to do.) I think npm should probably support doing that at install time. ESM importing CJS works in Node, mostly, now, but it does have quite a bit of runtime overhead and prebaking it would be good. Especially because it is unlikely to ever see a CJS loader in the browser (and that would be awful if it did exist).
  - no_wizard 3y ago
    if you use a package manager like `pnpm` you can use the `.pnpmfile.cjs`[0] to intercept packages and add `"type": "commonjs"` to the package.json. This tells node that it needs to load it as a CommonJS package and should work fine with ESM. There's also creating a require function from `node:module` package[1] [0]: https://pnpm.io/pnpmfile https://pnpm.io/pnpmfile [1]: https://nodejs.org/dist/latest-v18.x/docs/api/module.html#modulecreaterequirefilename https://nodejs.org/dist/latest-v18.x/docs/api/module.html#mo...
- paulddraper 3y ago
  Kinda niche, I wrote Bazel rules (https://github.com/rivethealth/rules_javascript https://github.com/rivethealth/rules_javascript), and with a build setting `--//javascript:module`. Jest tests automatically use the CJS version. Webpack builds use the ESM version. (And all my stuff is in TypeScript, so it needs a build step anyway.)
Dylan16807 3y ago
We're still getting there. Firefox only got the code to allow workers to use modules a couple months ago, and it's currently disabled by default. I currently have some very exciting code that dumps constructors to text form and sends them to a worker to run...
- lioeters 3y ago
  > dumps constructors to text form and sends them to a worker Oh do tell more, that sounds devious.
  - Dylan16807 3y ago
    It's mostly a series of `foo=${JSON.stringify(foo)};` and Bar.prototype.constructor.toString(). There's one object that didn't cooperate with either of those, but it's only used in a very limited way, so I did a quick hack to pack its prototype up. At the end is a function that sets up self.onmessage, and a line that executes that function. Then I do .join('\r\n') on the mess, turn it into a Blob, and pass the URL of the blob into new Worker(). The code running in the worker is only a few classes so it's not worth the hassle of setting up code-copying workarounds or avoiding modules for that portion of the code. It'll be nice when I can cut that and have nothing more than a few imports and self.onmessage.
    - lioeters 3y ago
      I see, a wonderous hack. Sounds like something a build tool/plugin or code transformation step could make it work magically behind the scenes. It's cool and also a bit horrifying (haha) that this is possible dynamically, to stringify a function and send it over to a worker thread to run. After all, code is data..
akira2501 3y ago
They may be fine.. but nodejs still makes working with them at the cli particularly inconvenient for me. I'm sure there's rational reasons they've made the choices they did, like import being async and which lexical context you're in being significant, but they create an inconvenient mess when I'm trying to test and refactor code around. Meanwhile, import cycles are easy to design around and to fix with a single interstitial module if you need it, and "live value" exporting has never been an impediment to me. You can export objects, and their properties are "live" enough for me. Outside of one specific problem in browsers, I'm not sure what ES6 is actually supposed to buy me. I'm still trying to figure out how to turn of ts-ls "File is a CommonJS module; it may be converted to ES6" disagnostic.
Karrot_Kream 3y ago
I'm new to the Node ecosystem but still find it a big pain to get ES Modules working properly with Typescript and `ts-node`, and only a minor pair to get it to work with a Typescript build/run workflow. I have no problems with this feature being experimental but I wish the ecosystem acknowledged it more in documentation.
- herpdyderp 3y ago
  try giving tsx a try instead of ts-node: http://npmjs.com/tsx http://npmjs.com/tsx
  - loh 3y ago
    Thanks for this. Just a few minutes ago I pulled up an old project using `ts-node` with ESM and tried to run it on a new machine, had some issues and remembered reading your comment here earlier. I switched to `tsx` and in less than a minute everything was working beautifully. I'll probably migrate all my TS projects to `tsx`.

imbnwa 3y ago

Node should've offered an extended LTS support for a final CJS supported release and just ripped the band-aid off, making ESM first-class for all subsequent releases. As Mike Ehrmentraut once said, "no more half-measures".

rgoulter 3y ago
I think the pain that the python 2 to python 3 upgrade caused would discourage this. I also see many people complained about the removal of the long-deprecated egrep and fgrep commands.
Chyzwar 3y ago
The problem is that for many use cases, ESM is not feature complete. There are important features that are under experimental flags or broken in node.js. node.js released broken, backwards incompatible version of ESM with new resolution algorithm and is waiting for everyone to figure out workarounds for theirs shitty implementation. [jest ESM support] https://github.com/jestjs/jest/issues/9430 [typescript extensions] https://github.com/microsoft/TypeScript/issues/16577 [typescript exports] https://github.com/microsoft/TypeScript/issues/33079 [node.js issues] https://github.com/nodejs/node/issues?q=is%3Aissue+is%3Aopen+sort%3Acomments-desc+label%3Aesm Only in the from few months ago, tooling support is close enough to make it usable.

rektide 3y ago

Notably no browser has we workers or service workers of any sort which are ESM capable. Fuck a duck.

ElfinTrousers 3y ago

Modules: one of the great programming innovations of the 1960s, first available in JS just fifty years later.

ixtli 3y ago
its my experience that you can say this for most things the non-academic community does. this isn't a criticism of them or you for saying it, but its p constant. the worst offenders in my experience are AAA game devs. ive had so many conversations where they imply they've solved some deep computing problem that was formally solved in 1954 by a set theorist or linguist heh
hombre_fatal 3y ago
This doesn't really make sense outside of "JS suxx amirite fellas?" JS had modules before ESM. ES modules try to solve different problems and offer new solutions with browsers fetching modules themselves, something other module systems don't even do. Whether a language has modules or not is just a design / ecosystem decision, not a technological one. Swift is a modern language (2014) that doesn't even have modules comparable to what Node.js introduced a 14 years ago. Yet it does little to hamper the quality of apps people are building for the Apple ecosystem.
- deleted 3y ago
  [deleted]
paulryanrogers 3y ago
Considering JS didn't exist until the mid 90's, and it's usage was quite limited until the mid oughts, that's not surprising.
- ElfinTrousers 3y ago
  I'd say not so much because of the timing but because JS was only meant to be a crude stopgap: the scripting language they'd use for a year or two and then replace. The rest is history, the history of a huge sector of our industry built on a hilariously inadequate foundation.