8 ms·

Using ES Modules in Node.js: A Practical Guide (Part 1)

timdaub 6y ago
I've started using ESM in a static website builder too [1]. It works, kinda. Problems that I encountered: To get `__dirname`, you have to do an awkward workaround using path and url [3]. For npm modules that implement a path e.g. `@lib/method`, the import of this currently isn't allowed in node without a flag. For ESM incompatible modules, a createRequire [2] function is available to get the old `require` syntax. For import statements, the cache cannot be deleted yet [4]. Overall, I'm please to see that nodejs will soon be ESM compatible. All changes that have been made look reasonable and useful to me. In fact, I now often get the pleasurable "wow that makes sense" feeling when using node. It has has great docs, nice new features like worker_threads and it's productive for me. I'm a fan! 1: https://github.com/TimDaub/kloi https://github.com/TimDaub/kloi 2: https://nodejs.org/api/module.html#module_module_createrequire_filename https://nodejs.org/api/module.html#module_module_createrequi... 3: https://stackoverflow.com/questions/46745014/alternative-for-dirname-in-node-when-using-the-experimental-modules-flag https://stackoverflow.com/questions/46745014/alternative-for... 4: https://github.com/nodejs/modules/issues/307 https://github.com/nodejs/modules/issues/307
- madeofpalk 6y ago
  > For npm modules that implement a path e.g. `@lib/method`, the import of this currently isn't allowed in node without a flag. FWIW, (and im sure you know this, just pointing it out for others) @lib/method is not a path - the @ at the beginning indicates the package is namespaced under an org. @lib/method is a full and proper regular package, whereas `lib/method` would be pointing to a path within the lib package.
- cookiengineer 6y ago
  If you decide to work with ESM there's only one issue I have with them in practice: There's no way to remap a "global package identifier" to the entry point (e.g. remap import from "package" to "../path/to/mymodules/package/index.mjs") on the Browser-side. Node.js has the workaround with the "exports{}" map in the package.json file, but there is no equivalent on the Browser-side, so you constantly have to work with dozens of "../../to/index.mjs" files as wrappers that do nothing more than a lot of "export * from ./some/other/file.mjs" inside them, which is a bit annoying. As a side note (most people aren't aware of it): All Browsers have support for ESM modules, including Safari; and even WebKit2GTK. [1] [1] https://caniuse.com/es6-module https://caniuse.com/es6-module
  - Zardoz84 6y ago
    IE11 not
    - deleted 6y ago
      [deleted]
    - imbnwa 6y ago
      Dunno why this is downvoted, IE11 isn't going anywhere in some rather crucial places, like all healthcare
      - deleted 6y ago
        [deleted]
  - timdaub 6y ago
    > There's no way to remap a "global package identifier" to the entry point (e.g. remap import from "package" to "../path/to/mymodules/package/index.mjs") on the Browser-side. I think what you're looking for is `require.resolve` [1]. ```mjs import { createRequire } from "module"; const require = createRequire(import.meta.url); const path = require.resolve("package"); console.log(path); >> ../path/to/mymodules/package/index.mjs ``` 1: https://nodejs.org/api/esm.html#esm_no_require_resolve https://nodejs.org/api/esm.html#esm_no_require_resolve
    - cookiengineer 6y ago
      I was talking about Browser-side, the cleaner node.js solution is to use the "exports{}" map in the package.json [1] that I mentioned before. Personally, I wouldn't recommend to override the require.resolve or import.meta.resolve algorithm, as this is heavily experimental. [1] https://nodejs.org/api/packages.html#packages_exports https://nodejs.org/api/packages.html#packages_exports
  - reaktivo 6y ago
    I'm hoping for the import-map[1] spec to supported soon. 1 https://blog.logrocket.com/es-modules-in-browsers-with-import-maps/ https://blog.logrocket.com/es-modules-in-browsers-with-impor...
- deleted 6y ago
  [deleted]
hrishi 6y ago
This is really helpful! I haven't started using EJS because the internal gotchas and problems always worried me.
anaisbetts 6y ago
ESM modules are literally only downsides and workarounds, there is absolutely no user benefit at the moment, why would anyone adopt this technology other than a standards board saying "You Should Do It"
- ratww 6y ago
  Quite the contrary. In the browser, workarounds have always been the other stopgap solutions, like script concatenation, require.js, and bundlers like Webpack. With ES6 modules, bundlers and transpilers are not mandatory anymore for a large part of modern JS development. This is a good thing, as they were a giant source of complexity.
  - londons_explore 6y ago
    Setting up a project that could either be bundled with webpack et-al, or simply hosted direct on GitHub proved tricky!
- bwindels 6y ago
  One reason is that it allows to run the same code in the browser and in node without any transpilation.
- mschuetz 6y ago
  I absolutely love ESM modules, pretty much the best thing since sliced bread. Node.js is really botching it, though, with the requirement to use the mjs extension. It makes it very awkward and cumbersome to use the same code in the browser and node.
  - Zardoz84 6y ago
    you not need to use .mjs extension if you setup correctly package.json
    - mschuetz 6y ago
      I don't want to setup a package.json for projects that don't need it.
danez 6y ago
The only thing missing for me now is that TypeScript supports it as output type. Right now you cannot use Typescript and compile it to ESM modules.
- maga 6y ago
  Check out esbuild.
- no_wizard 6y ago
  Set `ESNext` as the value for `module` under `compilerOptions` in your TSConfig will output an ES module every time. What it doesn't support currently is changing the output file extension, if thats what you meant https://www.typescriptlang.org/tsconfig#module https://www.typescriptlang.org/tsconfig#module
  - danez 6y ago
    Right, I remember now. The file extensions weren't correct, but even after renaming the files it does not work because the imports were also missing the correct file extension (which is mandatory in ESM mode in Node.JS). At that point I gave up.
- spankalee 6y ago
  TypeScript has supported JS module output since the very beginning: https://www.typescriptlang.org/tsconfig#module https://www.typescriptlang.org/tsconfig#module
desmap 6y ago
I went down the ESM rabbit hole, after 2 months I decided to stay away. Main reason is/was I want true monorepos in TypeScript with single DRY type definitions (so you need to import them to the FE). This alone is with CommonJS already a bigger endeavor but impossible when mixing in ESM for the frontend parts. Nobody should be blamed here because it's so much legacy, so many systems involved, so yeah.
- horsawlarway 6y ago
  I've been doing this in a personal project. I ended up settling on ts-node as the fix -https://www.npmjs.com/package/ts-node https://www.npmjs.com/package/ts-node Frankly - it does pretty much everything I want it to. I can write code in Typescript and have it feel like there's no build step at all. I can use ESM everywhere - frontend, backend, buildscripts, random tasks, etc - and have no problems. That said - It's not a legacy codebase, and it's only getting hobby level usage (currently serving a few requests a second), so there might be hurdles I'm just not hitting.
  - desmap 6y ago
    > I can use ESM everywhere - frontend, backend, buildscripts, Once you mix commonjs and esm and try to import from one to another, it doesnt work anymore. Since major BE libs are commonjs you'll quickly face this situation. It's not related to ts-node but to ts' core. Besides, I find ts-node not that spectacular, it hides too much moving parts/it's too high level, it might be good for starters but nothing I'd use in production.
linkdd 6y ago
As far as I know, ES modules do not allow variables as module name: const moduleName = "foo" import something from moduleName // will not work const something = await import(moduleName) // will not work I'm genuinely curious about how someone would do this with ES modules: // somewhere in a function or a class method const plugins = pluginModules.map(require) EDIT: I was wrong, the import() function does work with variables.
- epidemian 6y ago
  I'm not entirely sure since i've never done this, but i think you can use the dynamic import() function[1] and await all the import promises: const plugins = await Promise.all(pluginModules.map(p => import(p))) [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#dynamic_imports https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
  - linkdd 6y ago
    Even though it is called "dynamic import", it still only accepts string literals as argument: import("module") // will work import(p) // will not work import(`${p}`) // will not work
    - realityking 6y ago
      This works just fine in Node.js 14.15: // test2.mjs console.log("works!"); // test.mjs const m = "./test2.mjs"; import(m)
    - epidemian 6y ago
      There's an `import` statement with its custom syntax, which yes, does not allow anything else than a static string literal on the `from` part. But there's also an import() function, which is, well, just a function, so you can pass to it whatever parameter you want. For example a string computed dynamically as in your example. The linked page in my comment has a couple of examples of that function.
    - 6y ago
FunnyLookinHat 6y ago
I find using ES Modules in Node.js to be full of gotchas and anti-patterns - sticking with CommonJS seems more functional in most cases. 1: With ES Modules, imports are asynchronous. Synchronous imports are incredibly powerful. You can ensure that your application doesn't block on first requests, but instead on load. Initializing connection pools, setting up caches, etc. You'll also find quite a few packages in NPM that explicitly depend on require over import just so that they can perform the bindings they have to before things get going (see https://www.npmjs.com/package/honeycomb-beeline https://www.npmjs.com/package/honeycomb-beeline as an example). 2: With ES Modules imports are commonly destructured. I know it makes your code feel lighter to have const { add } = import("whatever"); - but as your files grow larger, and those imports start to become complicated bits for middleware or other features, you're just making it more difficult for future maintainers to figure out what those functions mean. A bit of context never hurt anybody, and for my money I'll be that this: app.use(check); is not nearly as useful as app.use(Validator.check); The example may be contrived - but 9/10 times I find this makes code better than the alternative. All of these arguments go out the door for Frontend work - that's a totally different terrain. I think one of the big arguments is that developers want to use the exact same JavaScript on frontend and Node applications - but that's an argument unwilling to admit the fact that they're two very different tasks. :)
- richthegeek 6y ago
  A nice middleground on your example is renaming during destructure: `import { check as ValidatorCheck } from "validator";` This keeps the context at the expense of a slightly more verbose import. Although the primary reason I use destructuring is to enable tree-shaking, and that's only really applicable for the frontend. I'm not sure why I'd destructure on the backend.
- spankalee 6y ago
  Modules load and eval asynchronously, because there's no viable way for network connected device like a browser to do it synchronously. But modules still run in depth-first import order, so there is a perfectly fine way do work before other modules run by putting that init code in a separate module and importing it first. Your second point applies the same to imports and require.
revskill 6y ago
Why import A from B, and not from B import A, like in python instead ?
- tym0 6y ago
  Probably because it looks like an assignment. import { a } from "lib" const { b } = a Also it "sounds" like something one would say.
- have_faith 6y ago
  What's the practical difference?
  - kall 6y ago
    The python way makes autocomplete more practical
  - abdusco 6y ago
    You can get intellisense easily with python because by the time you've written a package name, IDE will start suggesting symbols to you from sys import <suggestions> compared to node, where I have to write import {} from 'path' then move the cursor in the braces and hit [Ctrl] + [Space] to trigger intellisense import {<suggestions>} from 'path' which feels backwards to me.
    - jayflux 6y ago
      This is being looked at https://github.com/microsoft/TypeScript/issues/31658 https://github.com/microsoft/TypeScript/issues/31658
DrFell 6y ago
It's sad to see a serious thread about Node.js here, complete with mention of using TypeScript on the backend. Taps
eyelidlessness 6y ago
In my opinion and recent experience working with ESM, the design and tradeoffs are worth the trouble especially if you use a bundler or an optimizing build step, and especially if you prefer named imports/exports. On the other hand, benefits of ESM on Node are currently conditional on whether you’re running the same code on the browser and want consistent behavior on the server. For server-only projects I honestly think you’re still better off using a build step that compiles to CommonJS. That may change as Deno grows in popularity and interoperability becomes more of a concern, or as other compelling ESM features are introduced (for instance native import maps).

I've started using ESM in a static website builder too [1]. It works, kinda. Problems that I encountered: To get `__dirname`, you have to do an awkward workaround using path and url [3]. For npm modules that implement a path e.g. `@lib/method`, the import of this currently isn't allowed in node without a flag. For ESM incompatible modules, a createRequire [2] function is available to get the old `require` syntax. For import statements, the cache cannot be deleted yet [4]. Overall, I'm please to see that nodejs will soon be ESM compatible. All changes that have been made look reasonable and useful to me. In fact, I now often get the pleasurable "wow that makes sense" feeling when using node. It has has great docs, nice new features like worker_threads and it's productive for me. I'm a fan! 1: https://github.com/TimDaub/kloi https://github.com/TimDaub/kloi 2: https://nodejs.org/api/module.html#module_module_createrequire_filename https://nodejs.org/api/module.html#module_module_createrequi... 3: https://stackoverflow.com/questions/46745014/alternative-for-dirname-in-node-when-using-the-experimental-modules-flag https://stackoverflow.com/questions/46745014/alternative-for... 4: https://github.com/nodejs/modules/issues/307 https://github.com/nodejs/modules/issues/307

madeofpalk 6y ago
> For npm modules that implement a path e.g. `@lib/method`, the import of this currently isn't allowed in node without a flag. FWIW, (and im sure you know this, just pointing it out for others) @lib/method is not a path - the @ at the beginning indicates the package is namespaced under an org. @lib/method is a full and proper regular package, whereas `lib/method` would be pointing to a path within the lib package.
cookiengineer 6y ago
If you decide to work with ESM there's only one issue I have with them in practice: There's no way to remap a "global package identifier" to the entry point (e.g. remap import from "package" to "../path/to/mymodules/package/index.mjs") on the Browser-side. Node.js has the workaround with the "exports{}" map in the package.json file, but there is no equivalent on the Browser-side, so you constantly have to work with dozens of "../../to/index.mjs" files as wrappers that do nothing more than a lot of "export * from ./some/other/file.mjs" inside them, which is a bit annoying. As a side note (most people aren't aware of it): All Browsers have support for ESM modules, including Safari; and even WebKit2GTK. [1] [1] https://caniuse.com/es6-module https://caniuse.com/es6-module
- Zardoz84 6y ago
  IE11 not
  - deleted 6y ago
    [deleted]
  - imbnwa 6y ago
    Dunno why this is downvoted, IE11 isn't going anywhere in some rather crucial places, like all healthcare
    - deleted 6y ago
      [deleted]
- timdaub 6y ago
  > There's no way to remap a "global package identifier" to the entry point (e.g. remap import from "package" to "../path/to/mymodules/package/index.mjs") on the Browser-side. I think what you're looking for is `require.resolve` [1]. ```mjs import { createRequire } from "module"; const require = createRequire(import.meta.url); const path = require.resolve("package"); console.log(path); >> ../path/to/mymodules/package/index.mjs ``` 1: https://nodejs.org/api/esm.html#esm_no_require_resolve https://nodejs.org/api/esm.html#esm_no_require_resolve
  - cookiengineer 6y ago
    I was talking about Browser-side, the cleaner node.js solution is to use the "exports{}" map in the package.json [1] that I mentioned before. Personally, I wouldn't recommend to override the require.resolve or import.meta.resolve algorithm, as this is heavily experimental. [1] https://nodejs.org/api/packages.html#packages_exports https://nodejs.org/api/packages.html#packages_exports
- reaktivo 6y ago
  I'm hoping for the import-map[1] spec to supported soon. 1 https://blog.logrocket.com/es-modules-in-browsers-with-import-maps/ https://blog.logrocket.com/es-modules-in-browsers-with-impor...
deleted 6y ago
[deleted]

hrishi 6y ago

This is really helpful! I haven't started using EJS because the internal gotchas and problems always worried me.

anaisbetts 6y ago

ESM modules are literally only downsides and workarounds, there is absolutely no user benefit at the moment, why would anyone adopt this technology other than a standards board saying "You Should Do It"

ratww 6y ago
Quite the contrary. In the browser, workarounds have always been the other stopgap solutions, like script concatenation, require.js, and bundlers like Webpack. With ES6 modules, bundlers and transpilers are not mandatory anymore for a large part of modern JS development. This is a good thing, as they were a giant source of complexity.
- londons_explore 6y ago
  Setting up a project that could either be bundled with webpack et-al, or simply hosted direct on GitHub proved tricky!
bwindels 6y ago
One reason is that it allows to run the same code in the browser and in node without any transpilation.
mschuetz 6y ago
I absolutely love ESM modules, pretty much the best thing since sliced bread. Node.js is really botching it, though, with the requirement to use the mjs extension. It makes it very awkward and cumbersome to use the same code in the browser and node.
- Zardoz84 6y ago
  you not need to use .mjs extension if you setup correctly package.json
  - mschuetz 6y ago
    I don't want to setup a package.json for projects that don't need it.

danez 6y ago

The only thing missing for me now is that TypeScript supports it as output type. Right now you cannot use Typescript and compile it to ESM modules.

maga 6y ago
Check out esbuild.
no_wizard 6y ago
Set `ESNext` as the value for `module` under `compilerOptions` in your TSConfig will output an ES module every time. What it doesn't support currently is changing the output file extension, if thats what you meant https://www.typescriptlang.org/tsconfig#module https://www.typescriptlang.org/tsconfig#module
- danez 6y ago
  Right, I remember now. The file extensions weren't correct, but even after renaming the files it does not work because the imports were also missing the correct file extension (which is mandatory in ESM mode in Node.JS). At that point I gave up.
spankalee 6y ago
TypeScript has supported JS module output since the very beginning: https://www.typescriptlang.org/tsconfig#module https://www.typescriptlang.org/tsconfig#module

desmap 6y ago

I went down the ESM rabbit hole, after 2 months I decided to stay away. Main reason is/was I want true monorepos in TypeScript with single DRY type definitions (so you need to import them to the FE). This alone is with CommonJS already a bigger endeavor but impossible when mixing in ESM for the frontend parts. Nobody should be blamed here because it's so much legacy, so many systems involved, so yeah.

horsawlarway 6y ago
I've been doing this in a personal project. I ended up settling on ts-node as the fix -https://www.npmjs.com/package/ts-node https://www.npmjs.com/package/ts-node Frankly - it does pretty much everything I want it to. I can write code in Typescript and have it feel like there's no build step at all. I can use ESM everywhere - frontend, backend, buildscripts, random tasks, etc - and have no problems. That said - It's not a legacy codebase, and it's only getting hobby level usage (currently serving a few requests a second), so there might be hurdles I'm just not hitting.
- desmap 6y ago
  > I can use ESM everywhere - frontend, backend, buildscripts, Once you mix commonjs and esm and try to import from one to another, it doesnt work anymore. Since major BE libs are commonjs you'll quickly face this situation. It's not related to ts-node but to ts' core. Besides, I find ts-node not that spectacular, it hides too much moving parts/it's too high level, it might be good for starters but nothing I'd use in production.

linkdd 6y ago

As far as I know, ES modules do not allow variables as module name: const moduleName = "foo" import something from moduleName // will not work const something = await import(moduleName) // will not work I'm genuinely curious about how someone would do this with ES modules: // somewhere in a function or a class method const plugins = pluginModules.map(require) EDIT: I was wrong, the import() function does work with variables.

epidemian 6y ago
I'm not entirely sure since i've never done this, but i think you can use the dynamic import() function[1] and await all the import promises: const plugins = await Promise.all(pluginModules.map(p => import(p))) [1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#dynamic_imports https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
- linkdd 6y ago
  Even though it is called "dynamic import", it still only accepts string literals as argument: import("module") // will work import(p) // will not work import(`${p}`) // will not work
  - realityking 6y ago
    This works just fine in Node.js 14.15: // test2.mjs console.log("works!"); // test.mjs const m = "./test2.mjs"; import(m)
  - epidemian 6y ago
    There's an `import` statement with its custom syntax, which yes, does not allow anything else than a static string literal on the `from` part. But there's also an import() function, which is, well, just a function, so you can pass to it whatever parameter you want. For example a string computed dynamically as in your example. The linked page in my comment has a couple of examples of that function.
  - 6y ago

FunnyLookinHat 6y ago

I find using ES Modules in Node.js to be full of gotchas and anti-patterns - sticking with CommonJS seems more functional in most cases. 1: With ES Modules, imports are asynchronous. Synchronous imports are incredibly powerful. You can ensure that your application doesn't block on first requests, but instead on load. Initializing connection pools, setting up caches, etc. You'll also find quite a few packages in NPM that explicitly depend on require over import just so that they can perform the bindings they have to before things get going (see https://www.npmjs.com/package/honeycomb-beeline https://www.npmjs.com/package/honeycomb-beeline as an example). 2: With ES Modules imports are commonly destructured. I know it makes your code feel lighter to have const { add } = import("whatever"); - but as your files grow larger, and those imports start to become complicated bits for middleware or other features, you're just making it more difficult for future maintainers to figure out what those functions mean. A bit of context never hurt anybody, and for my money I'll be that this: app.use(check); is not nearly as useful as app.use(Validator.check); The example may be contrived - but 9/10 times I find this makes code better than the alternative. All of these arguments go out the door for Frontend work - that's a totally different terrain. I think one of the big arguments is that developers want to use the exact same JavaScript on frontend and Node applications - but that's an argument unwilling to admit the fact that they're two very different tasks. :)

richthegeek 6y ago
A nice middleground on your example is renaming during destructure: `import { check as ValidatorCheck } from "validator";` This keeps the context at the expense of a slightly more verbose import. Although the primary reason I use destructuring is to enable tree-shaking, and that's only really applicable for the frontend. I'm not sure why I'd destructure on the backend.
spankalee 6y ago
Modules load and eval asynchronously, because there's no viable way for network connected device like a browser to do it synchronously. But modules still run in depth-first import order, so there is a perfectly fine way do work before other modules run by putting that init code in a separate module and importing it first. Your second point applies the same to imports and require.

revskill 6y ago

Why import A from B, and not from B import A, like in python instead ?

tym0 6y ago
Probably because it looks like an assignment. import { a } from "lib" const { b } = a Also it "sounds" like something one would say.
have_faith 6y ago
What's the practical difference?
- kall 6y ago
  The python way makes autocomplete more practical
- abdusco 6y ago
  You can get intellisense easily with python because by the time you've written a package name, IDE will start suggesting symbols to you from sys import <suggestions> compared to node, where I have to write import {} from 'path' then move the cursor in the braces and hit [Ctrl] + [Space] to trigger intellisense import {<suggestions>} from 'path' which feels backwards to me.
  - jayflux 6y ago
    This is being looked at https://github.com/microsoft/TypeScript/issues/31658 https://github.com/microsoft/TypeScript/issues/31658

DrFell 6y ago

It's sad to see a serious thread about Node.js here, complete with mention of using TypeScript on the backend. Taps

eyelidlessness 6y ago

In my opinion and recent experience working with ESM, the design and tradeoffs are worth the trouble especially if you use a bundler or an optimizing build step, and especially if you prefer named imports/exports. On the other hand, benefits of ESM on Node are currently conditional on whether you’re running the same code on the browser and want consistent behavior on the server. For server-only projects I honestly think you’re still better off using a build step that compiles to CommonJS. That may change as Deno grows in popularity and interoperability becomes more of a concern, or as other compelling ESM features are introduced (for instance native import maps).