6 ms·
ECMAScript modules in Node.js: the new plan
- deleted 8y ago[deleted]
- tokyodude 8y agoI'm sure there are valid reasons but I disagree with the extension solution. That is not in any way meeting the stated goal of > One of the goals of the Modules Team is “browser equivalence” There are already tons of browser libraries not using the .mjs extension. Not clear to me why just using 'import' makes it a module and 'require' not. I'm sure that's spelled out somewhere
- deleted 8y ago[deleted]
- mstade 8y ago> Not clear to me why just using 'import' makes it a module and 'require' not. I'm sure that's spelled out somewhere I'm probably wrong but I think it has to do with parsing rules. If you try to load a module as CJS and hit incompatible syntax (e.g. keywords such as import or export) then it has to restart parsing from the top. Likewise, if you parse the file as ESM but hit CJS syntax such as `exports.foo = bar` then again it has to restart. There are probably a bunch of similar issues but if I recall correctly it basically came down to parsing and performance, you don't want to load the same thing twice essentially. With a special extension you don't have to even load the file before deciding which parsing rules to use, supposedly making this more efficient. I've heard several proposals to resolve this issue, but my favorite two are probably the unambiguous module syntax and making import/require only compatible with their respective module formats. The first would be a breaking syntactic change, in that every ESM module would be required (no pun intended) to include the top level `export` keyword at least once, regardless of whether or not the module actually exports anything. The second would be to make node's `require` function only compatible with CJS modules and make the `import` keyword only compatible with ESM. Both of these have drawbacks of course, but I'd like to think they're mostly temporary and rather easily overcome. The unambiguous module syntax less so perhaps, but still.
- cwmma 8y agothe other issue was that a module that doesn't import, export or require anything and doesn't specify strict mode will get parsed differently depending on whether it is a CJS or an ESM and their is no in band way of telling which is which
- mstade 8y agoThat’s right, thanks for the reminder! Some syntax is ambiguous yet results differ. I think that’s why I liked the unambiguous module syntax very much, it made things explicit, without much cost.
- snek 8y ago>There are already tons of browser libraries not using the .mjs extension. Browsers don't care what extension you serve something as, so we can't really directly match that in the default behaviour. What you can do is use a resolve hook to tell node to interpret certain files as esm even when they don't have a `.mjs` extension. >Not clear to me why just using 'import' makes it a module and 'require' not. import doesn't "make something a module". if you try to require something that resolves as being esm (rn we check that with .mjs) it will throw, because esm resolution is async and require resolution is sync. You can't require esm but you can import anything.
- tokyodude 8y agoWhy does import have to work with cjs? It doesn't in the browser. Why does require have to work esm? Why not just have a package up it's major semver and those packages that want to start using it need to import it? That seems no different than the browser. AFAIK you can't import a non-module and you can't non-import a module in the browser or am I wrong?
- snek 8y ago>Why does import have to work with cjs? It doesn't in the browser. Why does require have to work esm? `import` doesn't have to work with cjs, but we want to do so because there is so much code in the ecosystem that is cjs. `require` won't and can't work with esm, because esm resolution is async and require resolution is sync.
- ricardobeat 8y agoLet this be solved in userland through compilers, like everyone already does today. The stubbornness from current node maintainers on this subject is astounding. Years of feedback, like you see in this thread, ‘but-what-if’ and ‘you-dont-get-it’ handwaved away.
- ergothus 8y ago> I disagree with the extension solution As do I. I loathe it. But I'm struggling to come up with strong rational reasons why. Part of me says that hurts interoperability, but I think that's mostly rationalization. Why do I care so much about a file extension? Because I really really do.
- tokyodude 8y agoThe reason I care is because effectively node is imposing it's standards on the browser. Effectively node is requiring all browser based modules to use .mjs even though they shouldn't have to care. Except they will care because they'll start with .js, the make an npm package, then someone will want it to run in node, then it has to be renamed .mjs, then all non node projects are effected only because of node's chosen solution. It seems bad to me for node to effectively bleed it's problems on the entire JS ecosystem
- s_tec 8y agoI wonder why they kept the ugly .mjs extension. This is a hold-over from an earlier plan, and doesn't seem necessary in this iteration. This new plan seem to be going down the path that `import` is for modules while `require` is for everything else. If that's true, then they shouldn't need a special file extension for modules - it should always be clear from the context which is which. The only exceptions I can think of are executable scripts (which could use a command-line option) and packages in node_modules (which can use a special package.json field).
- Klathmon 8y agoIf it's not needed, then I'm also questioning it. Requiring that extension breaks a lot of previously "very nice" things. Being able to create a file like `thing.js` and require it by doing `require('./thing')` made it easy to be able to switch that out with a directory later and put an `index.js` in there without having to change any import statements. That is something i've used countless times and it tends to really improve my development experience since modules can grow more naturally and it avoids lots of one-file directories, and with the `.mjs` it's entirely gone.
- treve 8y agoThe .mjs extension isn't required in browsers, but absolute paths are. So even if the .mjs extension requirement is dropped, I don't think the implication is that './foo' can successfully resolve to './foo/index.js' as there is no such thing as a directory in HTTP and you don't want browsers to make multiple guesses for which path exactly exists.
- mstade 8y agoIsn't the absolute path (not exactly true, since dots are supported) thing just a matter of punting till later? If I recall correctly, they couldn't agree on resolution rules and perhaps more specifically how node-like rules would translate to the web, so they punted and went for an "absolute or bust" kind of approach till someone comes up with a better idea rather than painting everyone into a corner. Mapping identifiers and path resolving functions have been suggested as ways to make identifiers "nicer" I think. This seems reasonable. I reckon the reason why node's behind on figuring out how to make ESM work is because of all this implicit loading magic, and a reluctance to break existing code. (A commendable goal I think.)
- bryanrasmussen 8y agoSince I've worked on International Standardization before I sort of know how the sausage is made, but you always hope for a better sausage. Anyway - browser equivalence? That, in a lot of things, especially where URLS is concerned, means HTTP equivalence. A file extension having a specific meaning is not HTTP equivalence or browser equivalence, it is at best Windows OS equivalence where not having mime types understanding meant you had to use the file extensions to identify what 'type' of thing something was. A hack solution that was maybe necessary (I don't know Windows internals enough to disagree) and that caused me a lot of problems in the years 1999-2009 approximately.
- snek 8y agoBrowsers don't care what extension you serve something as, so it doesn't break the interop we want with browsers if you put your files as `.mjs`.
- bryanrasmussen 8y agomy point exactly, browsers care what mime type you use and the extension is meaningless. I thought I was pretty clear on this matter?
- BillinghamJ 8y agoBoth old "script"-style JS and the new module kind have the same MIME anyway
- OnlyRepliesToBS 8y ago'we're gonna make this fucking bazaar tighter than any cathedral!'
- revskill 8y agoWebpack with css, less, sass assets with React as a SSR library. What's the current solution now ?
- nobleach 8y agoI think it can be a bit difficult for users of compile-to-JS setups to understand why this is better. As of right now, one can just `import foo from './bar';` and the compiler just makes it work. This has become fairly tried and true. What might be confusing is that node's internals, and perhaps their design goals do not align with this paradigm. While I'm certain that they have reasons, it's going to be awfully hard to convince developers. (It already has been, look at the backlash on `.mjs`.) The solution? Continue to use TypeScript/Webpack/ESMloader forever. I personally like the semantics of importing the way it currently works. If it never works like that natively? Fine.
- snek 8y agoThe issue is that transpilers didn't implement correct ESM (both ts and babel have opt in flags to enable more-correct-mode, but its still not 100%). they implemented the syntax but forgot about the actual semantics like dependency graph verification and such. Node is an actual implementation of js and so we must perform these tasks, even while transpilers don't.
- efdee 8y agoHonest question: Does anybody actually care that it doesn't?
- snek 8y agoI don't think its as much a matter of people caring or not as it is that node is implementing ecma262 esm, not babel esm or typescript esm, etc.
- nobleach 8y agoWhat happens if people seem to prefer the babel-esm semantics? I'm not saying that babel's semantics ARE better, but what if developers prefer them?
- 8y ago
- deleted 8y ago[deleted]
- through 8y agoThe Michael Jackson Script is an absolute joke and had arisen solely from NodeJS attempting to solve its own require legacy. It’s not the business of any other party to do such a ridiculous thing. .mjs is an aberration, a solution to a poor implementation of one web server, not a solution. NodeJS need to do the hard work of solving their own problem and stop attempting to manufacture “real world solutions” with .mjs.
- snek 8y agoIf you feel you can come up with a better solution for module syntax ambiguity please feel free. Over the last three years we (including hundreds of devs like you who come in guns blazing calling us all idiots) haven't been able to come up with something that works better.
- through 8y agoProposing or fixing a bug in a third party product should not affect a web standard. If you want my personal help to assist the NodeJS fix, yeah, I would be good there, but I can’t fix the toxic bikeshed culture that you obviously enjoy. MJS is NodeJS solution. It is not a web solution.
- through 8y agoYou could support .es file types which would work just as well and is more succinct than .mjs, but AFAIK the group won’t because of “branding” issues. Bull. Use .es. Standard semantics for ECMSScript which is the name of the language, no games or pretence.
- ricardobeat 8y agoThat is a self-inflicted problem. Step away from the ambiguity by dropping cross-compatibility as everyone has been begging for.
- jdd 8y agoNode will likely end up with multiple ways to disambiguate modules. Until then, you might dig https://npmjs.com/esm https://npmjs.com/esm. No .mjs required.
- snek 8y agoThis is a pretty good breakdown but to the author: i would like to see a better explanation of defaults. Node will ship with the defaults needed to perform tasks logically and unambiguously. This however doesn't mean that a user's use case is locked away. ESM in node has hooks which allow people to override resolution behaviour, which means you could put esm in `.esm` files, or cjs in `.sadness` files, or have all files in a specific directory be esm, or read package.json and pull metadata out of it, etc.
- styfle 8y agoIs this documented anywhere?
- donateee 8y agobabel/register: https://babeljs.io/docs/en/babel-register https://babeljs.io/docs/en/babel-register
- keepingscore 8y agoDid I read that correctly we are going to lose the ability to require paths and modules eg import {} from './services' or import {} from 'lodash'?
- jkrems 8y agoFirst: none of the exact rules are written yet. But current discussions seem to suggest "Yes, this will go away" for "./services" being resolved to anything but an absolute location ending in "/services". But importing so-called bare specifiers ("lodash") will almost definitely be supported.
- snek 8y ago> current discussions seem to suggest "Yes, this will go away" for "./services" being resolved to anything but an absolute location ending in "/services" to be clear, this is still heavily in contention. node made a good choice to support requiring something without knowing exactly what type of thing it is, and a lot of people feel we shouldn't throw that away.
- BigJono 8y agoCan you give an actual use case for this? Because it seems insane. How can you use something without knowing what it is?
- snek 8y agoNo you did not.
- jagthebeetle 8y agoI'm not strongly opinionated or informed about this, so why does Node need file extensions when Other languages don't? Is it because of non-JS imports? I know that when Ryan dahl announced Deno, he mentioned it as an obviously good change, but, I'm not sure what the problems with extension-less imports are. (Edit for context: at work, we either use Closure or put everything under a global root, and only import JS, so maybe I've been shielded from the state of the world.)
- snek 8y ago> why does Node need file extensions when Other languages don't? Node allows you to require things that aren't commonjs javascript. (json, native extensions, etc). > when Ryan dahl announced Deno, he mentioned [removing extensionless resolution] as an obviously good change I think a lot of people disagree with ryan dahl in this case (I certainly do). The ability to have a dependency expose an interface without knowing what type of thing that dependency is is very powerful, and i think it is one of the reasons people like require so much.
- ng12 8y ago> The ability to have a dependency expose an interface without knowing what type of thing that dependency is is very powerful Is it? I honestly can't say I've actually used that feature -- although I can say I have gotten burned by it by accidentally importing the wrong thing.
- austincheney 8y agoJavaScript is environment agnostic, such that JavaScript doesn't know what a directory is. Many other languages have a standard library that define these things, but JavaScript does not deliberately to avoid presumptions upon specific environmental constraints that apply in some conditions but not others. A directory is a convention of a file system. When executing in a web browser there is no file system. If they wish to achieve convention and function compatibility with code that works in the browser certain constraints are necessary to prevent assumptions that work in one place and not another. Also, consider the work that happens when resolving files from a directory: 1. You have to determine if the thing is a directory or a file. 2. Get list of files in directory. 3. Read each file 4. Determine if that file is a ESM. 5. Load modules only if the file is ESM. These steps would not work in a browser.
- apo 8y agoMeanwhile, Deno (alternative server-side JS built from scratch by the creator of Node) keeps chugging along: https://github.com/denoland/deno https://github.com/denoland/deno It will be interesting to see which happens first: developers migrate to Deno or Node finalizes its ES module implementation. I think both groups are doing excellent work, but the Node team seems to be at a disadvantage having to continue support for some decisions made early on in Node: https://www.youtube.com/watch?v=M3BM9TB-8yA&vl=en https://www.youtube.com/watch?v=M3BM9TB-8yA&vl=en
- tobr 8y agoThanks for reminding me about Deno. It really feels like some of Node’s homegrown solutions to things like modules and async is starting to become a burden. > It will be interesting to see which happens first: developers migrate to Deno or Node finalizes its ES module implementation. I guess that depends on what you mean with "happens". Only one of them has a clear point where they're done happening.
- sickcodebruh 8y agoThis is my first time seeing it but so much of it looks great. I don’t think I’m a fan with how it handles dependencies, though. It feels like they’ve just reinvented GOPATH. “Deno caches remote imports in a special directory specified by the $DENO_DIR environmental variable. It default to $HOME/.deno if $DENO_DIR is not specified. ... Production software should always bundle its dependencies. In Deno this is done by checking the $DENO_DIR into your source control system, and specifying that path as the $DENO_DIR environmental variable at runtime.“ https://github.com/denoland/deno/blob/master/Docs.md https://github.com/denoland/deno/blob/master/Docs.md The suggestion to import and reexport dependencies in a central `package.ts` file sounds absurd to me. Clearly, I’m not a user and am just now reading this for the first time. Has there been discussion about this in the community? Are there plans to correct it? Go has been digging out of this hole for years, I can’t imagine why they’d start in the same place.
- bartlomieju 8y agoDisclaimer: I contribute to Deno. Contrary to Go, when using Deno you don't need to invoke any command to install a package. You just put your import statement with full URL (you can pin version there, eg. https://unpkg.com/liltest@0.0.5/dist/liltest.js https://unpkg.com/liltest@0.0.5/dist/liltest.js) and Deno downloads it on first run and caches for later use. Changing $DENO_DIR is somehow equivalent to changing node_modules location. This eliminates need for package manager.
- ianwalter 8y agoI appreciate the work going into this but the timeline is a bit disappointing.
- strictnein 8y agofrom https://nodejs.org/api/esm.html https://nodejs.org/api/esm.html > For now, only modules using the file: protocol can be loaded. Is it the plan to allow modules to be loaded from any protocol? Because that seems dangerous, from a security viewpoint. I've been trying to find mention of anything that mentions SRI support, or something similar, but I haven't found it yet.
- OnlyRepliesToBS 8y agodon't masturbate over sugar tho
- chmln 8y agoInstead of forcing this joke of a file extension, it would be far better to make a breaking change in a semver major release and do away with commonjs. There is no usecase for having two module syntaxes at the same time. All the big projects already use babel or typescript. The community would catch up eventually.
- through 8y agoI completely agree. As I pointed out below, following the path you suggested, if the filetype has to change to fix the require regression, they could use .es. Would it be surprising to use .es with ECMAScript? No.
- chmln 8y agoRequiring users to adopt a new extension is a massive change and as an end-user I have nothing to gain from this - I'm already using babel and typescript along with a huge part of the community. If this creates regressions like you said, let users deal with their code that deletes things from frozen objects. Allowing bad behavior to continue and stall the module evolution is questionable at best.
- russellbeattie 8y agoLet me state some things which I think are true: 1) A vast majority of developers hate '.mjs'. 2) In 5 years, most developers will have moved to import statements and stopped using CommonJS' require(). 3) Importing from a directory using index.js is both common in popular projects, and a useful way to organize code like components. 4) We've already been waiting years for a standard solution and many of us are getting annoyed at the delays. 5) With transpilers the norm nowadays, it seems NodeJS could make a clean break from the past and simply provide tools to help convert older projects. Am I assuming too much?
- BillinghamJ 8y agoTo be honest I don't mind `.mjs` - we renamed everything in our system to that about a year ago, but I do think it's important for it to be consistent with browser standards. The index.js thing is nice, but I think consistency with browsers is more important and I'd be willing to lose it.
- BigJono 8y agoI actually don't see why the whole index.js thing is nice. What happened to prefering explicitness? All it does is save a couple of characters, it doesn't make anything clearer. It's just one more noob trap that's neither here nor there for experienced devs.
- BillinghamJ 8y agoOne nice benefit was that you could change from a module being defined as a single file to making it a folder with the logic split across multiple files without any other files needing to be changed
- BigJono 8y agoMehhhh, to quote Joel Spolsky: "This is the kind of thing you solve in 5 minutes with a macro in emacs".
- 8y ago
- z3t4 8y agocan a script using es6 modules start with something besides a import statement ? then it would be easy to know if to use es5 or es6 module syntax. why was the es6 module system made to break the es5 module system ? why not just use es5 modules ? why is es5 modules so slow? are es6 modules faster ?
- shurley 8y agoNo one will use this Interoperability hack in light of the fact that Rollup and Webpack perform this for you seamlessly: import {createRequireFromPath as createRequire} from 'module'; import {fileURLToPath as fromPath} from 'url'; const require = createRequire(fromPath(import.meta.url)); const cjsModule = require('./cjs-module.js'); Bundlers will not likely be able to produce efficient static code for that bizarre interop code.
- pvorb 8y agoCan anybody explain to me why ES didn't simply adopt the require syntax from Node.js?
- yuchi 8y agoImport statements must be asynchronous in the latency-controlled environments (mostly browsers). CJS is by definition synchronous and has different order semantics when treating execution of an asynchronous loader.
- pvorb 8y agoBut it could simply be implemented using the same routine in browsers like `import`. I don't see why `require` can't be asynchronous. There's an implicit join after the last import. Maybe it would have been enough to restrict calls to `require` to the top of a file.
- keithwhor 8y agoIs there any reason we can't just have a `'use esm';` declaration at the top of a file instead of a janky filename extension? It's just as arbitrary, but there's precedence with `'use strict';`, and instead of renaming all of my files I just prepend them appropriately.
- jdd 8y agoThe `esm` loader allows specifying a parse goal pragma as one of several ways to disambiguate source – https://npmjs.com/esm https://npmjs.com/esm. Node will likely end up needing more than just an extension to disambiguate as well.
- writepub 8y agoEven with new features, node chooses to do things differently. Take worker threads for instance. Node added worker support much later than the browser, and chose to mysteriously use an API that's significantly different. I wish interop with the browser becomes more Central for upcoming features. Where the browser is lacking, node can supplement, but to supplant those APIs is just inconvenient