4 ms·
The simplest solution is to stop requiring the top level package to be a module. Allow es modules to be require()-ed. It's be synchronous and slow and ideologic
by bastawhiz 2y ago
The simplest solution is to stop requiring the top level package to be a module. Allow es modules to be require()-ed. It's be synchronous and slow and ideologically impure, but it'll make it so that millions of projects (mostly private!) will be able to start adopting es modules without a massive refactor. Anyone with a project of significant age or size looks at ES modules now and thinks "fuck it". There's no return on investment to convert (other than less pain while trying to upgrade or adopt new libraries). It's a big undertaking with loads of risk (modifying import order isn't safe!) and the payoff is "it's the shiny new thing".
I had been using adminjs at work. Their new major version was ESM-only, and it was easier to _write a new admin panel from scratch_ than it was to refactor our entire codebase to be ESM just so we could upgrade one library. I expect that's the situation at hundreds of thousands of other companies.
Like for all the belly aching that happened over async functions (and the whole function color rant), synchronous and asynchronous functions work together just fine through plain old promises. You can easily use async functions alongside functions that use old fashioned callbacks. ESM vs CJS is a file coloring (versus function coloring) problem, but there's no interoperability. There's no escape hatch when you just need one file to use another file but their colors are incompatible.
- afavour 2y ago> it was easier to _write a new admin panel from scratch_ than it was to refactor our entire codebase to be ESM just so we could upgrade one library A middle ground answer would be using import() in CJS to asynchronously require the ESM module. It would require some hacking around your loading flow to ensure the module loads before you try to do anything with it but would still be preferable to rebuilding the entire thing.
- bastawhiz 2y agoI don't disagree! But is there really an argument (other than an ideological one) for why it needs to be an awaited import? Why does it matter that it's async?
- eyelidlessness 2y agoESM (as standardized, not necessarily as implemented in Node) treats import specifiers as URLs. That alone would inherently be async because network requests are async across basically all JS runtimes. ESM was also later extended to support top level await, making module execution itself async. Node has recently taken a more pragmatic approach, by treating local specifiers as potentially synchronous, allowing sync require() of ESM… which then fails if the required module uses top level await.
- WorldMaker 2y ago"Sometimes works, but can also blow up at runtime after any upgrade" seems to me like a "pragmatic cure" that is worse than the existing "disease" of needing to treat all ESM imports as async and/or migrating all uses of `require()` to `await import()`. If incrementally moving a CJS code base to use `await import()` in more places is too hard then you probably have other problems you've overlooked for too long (bad promise usage, callback waterfalls, etc). (But I'm a hardline "all you need is type=module" sort and think a CJS=>TS emitting ESM "rip off the bandaid" approach to CJS legacy libraries works and is fundamentally easier than most of the "pragmatic" solutions to CJS/ESM interop. It is past time to jettison CJS out the airlock.)
- zarzavat 2y agoFetching code at runtime over the network makes sense in a browser, it doesn’t make sense outside of a browser and would actually be a huge security risk without all of the security features that browsers have. So it just reinforces the perception that ESM was only designed for browsers. Both TC39 and node utterly cocked this up. The design of ESM is bad, async imports are a stupid idea, and node’s implementation has been beset by ideology.
- tqwhite 2y agoI'd say it's exactly the opposite. Neither require() or import require an async code structure. The biggest reason I avoid ESM like a plague is that many of my projects are not, and do not need to be, async. Adding import() is a huge hassle because it is async. Adding an importSync() function would be a perfect solution.
- IggleSniggle 2y agoThat was my first thought, and what I attempted first in my project. But it turned out that rewriting everything, while being incredibly tedious, was significantly easier to reason about and ensure continuity than developing my own bespoke dependency loader in a large project where not all portions are well understood.
- ComputerGuru 2y agoI have a custom loader that can operate in both sync and async mode with the same syntax; it started off as a single loader to load AMD and CommonJS but I’ve been meaning to add ESM support as well (it was more opaque than I liked when I last tried, required too much browser magic). https://github.com/mqudsi/loader https://github.com/mqudsi/loader
- deleted 2y ago[deleted]
- microflash 2y ago> Allow es modules to be require()-ed. It's be synchronous and slow and ideologically impure, but it'll make it so that millions of projects (mostly private!) will be able to start adopting es modules without a massive refactor. This is already available in Node.js 22 [1] with ` --experimental-require-module` flag. [1]: https://nodejs.org/en/blog/announcements/v22-release-announce#support-requireing-synchronous-esm-graphs https://nodejs.org/en/blog/announcements/v22-release-announc...
- jauntywundrkind 2y agoBefore Node had any/experimental module support (April 2019, https://nodejs.medium.com/announcing-a-new-experimental-modules-1be8d2d6c2ff https://nodejs.medium.com/announcing-a-new-experimental-modu...), I'd been using std-things/esm loader for Node.js, which seemed to have no problem allowing CJS and ESM to interop, require and import each other. In my experience it worked flawlessly. https://github.com/standard-things/esm https://github.com/standard-things/esm It's wild to think about: esm has taken nearly a decade to be feasible for folks to use, without heavy encumbrance; it was the key feature of es-2015/es6! It took many more years for import-maps to become a thing, such that we could start using modules in a modular fashion; we've had to use bundlers/codemods to re-resplve dependencies, unless you've been going to go Deno's original path of just hitting absolute urls wherever they be. There's still no standing proposal for how we get import-maps on service-workers & shared workers; modules still are not a ubiquitous feature of the core web & JavaScript ecosystems. 9 years latter. https://github.com/WICG/import-maps/issues/2 https://github.com/WICG/import-maps/issues/2 It's been hard being a webdevs, when the future remains so partially implemented. The article's exploration of the bomb ecosystem is telling. And is much of the reason I hope JavaScript Registry (jsr) project might succeed. Npm will never escape it's past. CJS is going to be in there for a long time. Being on a registry where everything is esm & typescript, that sounds divine. https://deno.com/blog/jsr-is-not-another-package-manager https://deno.com/blog/jsr-is-not-another-package-manager
- z3t4 2y agoDid anyone actually want ES modules, and are anyone using ES modules in production today? I don't think so except for transpiling languages that compile and bundle the JS. The only argument against lazy loading is that single threaded UI's will freeze for a few ms when the code loads, but if you compare that to the time it takes to fully load a modern website that point is moot.
- tyleo 2y agoBy god do I agree with this. I feel like all my most diabolical ESM issues are beating the tsconfig.json, webpack.js, and package.json into shape to get ESM and legacy working.
- TheRealPomax 2y agoAlthough on a payoff note, it's not even the shiny new thing, it's almost a decade old. We really need a Python 2.7 style "no: we're done with this. Don't uplift your code as much as you deem necessary, but we're moving on. You've had a decade by now" event for Node.
- bastawhiz 2y ago> it's almost a decade old The syntax is a decade old, and people have been using CJS with ES6 import syntax this whole time without major issues. Node ESM support is an entirely different self-inflicted problem that's been around for just a couple years. Don't gaslight folks into thinking this was something that the ecosystem was planning for a decade. It simply wasn't.
- TheRealPomax 2y agoIf by "just a couple years" you mean five years. There is no gas lighting here: I am directly blaming Node for hurting the uptake of ESM by not planning to switch to ESM-first (with CJS second) as part of the ESM support work, and I'm blaming Node even more for then also not starting that planning work when ESM in Node was considered mature enough to not live behind an experimental feature flag anymore. Node was, from the get-go, strongly tied to V8's development roadmap. They (like everyone else in the JS language space at the time) were watching the ESM debate, and had every opportunity to plan for a "few years out" switchover, starting the moment TC39 accepted ESM into the official language spec, with V8 following suit. They didn't, but thankfully it's never too late: Node should absolutely still go "we're finally deprecating CJS. You should use ESM, and for the next 2 years it's business as usual but after that we won't support CJS unless your project marks itself as such in package.json, and 2 years after that, we're removing CJS support entirely".
- bastawhiz 2y ago> Node should absolutely still go "we're finally deprecating CJS. You should use ESM, and for the next 2 years it's business as usual but after that we won't support CJS unless your project marks itself as such in package.json, and 2 years after that, we're removing CJS support entirely". So this has two consequences: 1. NPM is split in two. All the packages that don't upgrade are useless. If you rely on one, you either need to do the work to port it (and all of its dependencies) before you can even consider migrating your own code. Or pray that someone else will do it. 2. Companies simply won't upgrade. The last version of Node to support CJS will persist for a decade or more. It might get forked and make it harder for Node to actually drive the ecosystem forward. If you think io.js won't happen again as a result of a decision like this, you're mistaken.
- hn_throwaway_99 2y agoThe whole thing feels like IPv6 adoption to me: Make v2 of a thing that is undeniably better, but downplay/ignore the migration strategy and then just complain that people are lazy if they haven't migrated yet.
- cxr 2y ago> There's no return on investment to convert Nothing except doing things the standard way so that standards-compliant language implementations (and other tools) can work with them instead of targetting IE^H^H NodeJS's non-standard quirks. > the payoff is "it's the shiny new thing" NodeJS's reality distortion field has claimed another. Ecmascript modules are going on 10 years old at this point. (For comparison, that's more than the amount of time that NodeJS existed when modules weren't a part of the language.)
- JasonSage 2y ago… so what’s the return on investment exactly? It sounds like nothing tangible at all.
- cxr 2y agoBut why male models?
- bastawhiz 2y ago> Ecmascript modules are going on 10 years old at this point. False. Ecmascript defined the syntax for imports but not the mechanism for module loading. I've been using ES6 imports with CJS modules for almost as long as the syntax has existed (without problems!); the incompatibility you're describing is ones of years old. > Nothing except doing things the standard way so that standards-compliant language implementations Node's way of handling modules isn't the same as the way the browser works (and it can't be: see node_modules). There's no singular all-encompassing "JavaScript ESM" standard. There's not even consistency between server-side JS runtimes. But more importantly, saying "the way your code worked for fifteen years is now shit and won't work going forward, you need to rewrite it because there's a new thing now" is an incredibly fucked way to run an ecosystem. See: Python 3.
- cxr 2y ago> Node's way of handling modules isn't the same as the way the browser works That's a Node problem. 100% foreseeable (and avoidable).* There's a reason why I compared NodeJS to Internet Explorer in my first comment—because it's extremely apt. Too bad most NPM programmers have blinders such that their entire worldview consists of NPM+NodeJS (not unlike the way programmers who were used to exclusively targeting Microsoft tech 20 years ago lived in their own parochial world) and an expectation that everyone else should bend over backwards to accommodate their way of life. If you as a programmer want not to be burned by IE-/NodeJS-like shenanigans, then don't run arms outstretched into those flames. * And, no, it's not an anachronism to say that. There's a reason why the notion of "forward compatibility" exists—at least among the wise. For an example of how to approach things sensibly (read: avoid getting burned exactly like one should otherwise expect to), have a look at the position/approach to forward compatibility that the TypeScript team (initially a Microsoft undertaking, funnily enough) has taken. There's no reason anyone should expect to be able to throw caution to the wind, YOLO their way forward with their hands covering their ears while shouting "la-la-la" and for everything to turn out okay. That is profoundly unwise. > an incredibly fucked way to run an ecosystem Right. Which makes it doubly crazy the way that folks underneath the NodeJS carnival tent have taken to approaching things for the last 15 years—and the last 10 years in particular. (Again, to emphasize: 10 years!) But of course I'm being facetious, because when you said "an ecosystem" you didn't narrowly mean "the NodeJS ecosystem". You meant "the JS ecosystem"—leaning heavily on the very same merger doctrine of the-NodeJS-ecosystem-is-the-JS-ecosystem-which-is-the-NodeJS-ecosystem that I mentioned NPMers believing in before. > See: Python 3. Sure, and Python is proprietary—with a BDFL, a community rallying for the most part behind a single implementation (where you get what you pay for if you go astray), and no strong commitment to standards/compatibility for most of its existence; the Python project can do whatever the Python project wants to do. Likewise Perl. Likewise, you know... Dart. That's how those things go. JS is fundamentally different, and that's what you and countless other NPM programmers seem not to be getting. See also my earlier comment: The difference is the language/standard in question neither originated with NodeJS nor is NodeJS now nor has it ever been led by the people behind the language/standard <https://news.ycombinator.com/item?id=40744081 https://news.ycombinator.com/item?id=40744081> The node_modules ecosystem is, in fact, not the whole world, nor is it synonymous with "JS". (Heck, it's not even synonymous with CommonJS[1].) NPMers can ignore this fact at their peril, or they can wise up and accept that they can't simply will things to go the way they apparently think they should be able to make them go despite mountains of evidence and wisdom to the contrary. 1. <https://news.ycombinator.com/item?id=31528800 https://news.ycombinator.com/item?id=31528800>
- akira2501 2y ago> It's be synchronous and slow and ideologically impure Sure. It also "just works" at the CLI. Sometimes I just want to test code quickly at the CLI. I'm not into "architecting" this part of my code as much as ES thinks I want to.
- tqwhite 2y agoThank you for saying this. I got 'flagged' when I said it more irritably. The solution is so simple and the payoff so great that it seems to me that the standards bodies are a bad process. Me? I'd prefer importSync() to having require() cover ESM files. I think it would be more clear but, either way, it would do exactly what you say, open the CJS world to migration.
- bastawhiz 2y ago> I'd prefer importSync() to having require() cover ESM files. My only criticism is that in order to do incremental conversion, both need to work. At that point, you'd might as well just eliminate the cognitive overhead and have one function.
- throwAGIway 2y agoWhat about the createRequire function?
- bastawhiz 2y agoWhy should I need to go out of my way to do this? Why do I need to care what kind of module I'm importing and reach into the stdlib to run code? It should just work.
- throwAGIway 2y agoTo be frank I don't care about these questions, perhaps you're right and it should, but whatever. It's a solution I used in the three cases it was necessary that blocked moving my codebase to ESM.
- sureIy 2y agoYou’re missing the point. Of course createRequire is a solution, it’s always been. The problem are tools and sub-dependencies that you can’t touch.