6 ms·

Adding JavaScript modules to the web platform

123 points by sonnyp 10y ago

MatthewPhillips 10y ago
If you don't find JavaScript modules confusing, you should. I work on a module loader [1] and I find it quite confusing. To clarify we have 3 things: 1. import and export syntax, this is defined in ES2015. It's just syntax, it doesn't explain how modules should be loaded. 2. The Loader spec (http://whatwg.github.io/loader/ http://whatwg.github.io/loader/) which defines a full loader including hooks to do just about anything you can possibly need. This allows you to do really crazy things like define modules dynamically and import then dynamically (which the static import/export syntax doesn't allow). We use this extensively in StealJS (or at least an older version of it) and it's pretty powerful. 3. import type=module is what this article is referring to. It defines a tag that allows you to import modules in the browser. You can't use a regular script tag, modules are different; they are strict mode by default for example. If you read the GitHub issue there was some discussion of whether import type=module should use the Loader spec. It probably should. But the Loader spec isn't done so... reading the spec and seeing things like the module map [2] it sounds like import type=script defines a miniature sized loader. I don't blame them for doing this, Loader has taken too long and people are clamoring for something. The Loader spec has a weird history; it was close to being done in ES2015 but when it was ejected it sit for a long time without much work. Then around last summer it was heavily worked on and got pretty far. But now as it sits, no commits like the last 2 and a half months [3] At this point I wouldn't be surprised to see Loader go away. It's probably too controversially and too big to ever get done. I imagine the web will just have the import type=module loader and Node will do it's own this. This is going to make writing portable JavaScript as annoying going forward as it's always been. [1] http://stealjs.com/ http://stealjs.com/ [2] https://html.spec.whatwg.org/multipage/webappapis.html#module-map https://html.spec.whatwg.org/multipage/webappapis.html#modul... [3] https://github.com/whatwg/loader/commits https://github.com/whatwg/loader/commits
- CraigJPerry 10y ago
  >> This allows you to do really crazy things like define modules dynamically In python there's a library called "sh" and it has a novelty interface: from sh import ls, pwd, my_unique_command print ls('/dir') ... I wondered (not enough to check right enough) if something similar could be done in JS. Although due to the async nature of such a call, it might not end up looking so familiar with JS in practice i guess.
  - redka 10y ago
    https://github.com/shelljs/shelljs https://github.com/shelljs/shelljs Edit: \/ ahh, yes you're right!
    - foota 10y ago
      I think if I'm grokking correctly that you can import "anything" from sh, without it having to be explicitly in the package.
- spankalee 10y ago
  One detail that's often lost in the discussion (that you might have been hinting at) is that without a loader modules can only be referred to by URL. Node-style name resolution won't work at all. This means that modules will either still have to be compiled with something like browserify that resolves modules names into URLs ahead of time (a solution I don't particularly like), or modules will just have to use URLs to import other modules, leading to a convention of assuming that modules are all siblings so that you import like this: import * as jquery from '../jquery/jquery.js'; Despite the extra verbosity, I prefer this since it requires that modules are flat and de-duplicated and doesn't require any resolution step at all.
  - MatthewPhillips 10y ago
    Modules are not flat in reality though, any non-trivial sized app likely has multiple versions of the same module being used. I don't see any reason to think that's going to change in the future, extreme modularity is the norm in JavaScript these days.
    - josteink 10y ago
      It may be the norm right now (since npm came around), but that's no reason to excuse it for what it is: a mess. A mess we should be able to do without. If we can have an official module-implementation which somehow forces developers to get a grip on their dependencies (which is the exact opposite of what "modern" js developers are doing), that can only be a good thing. The cowboy days must surely soon be over?
      - jessaustin 10y ago
        Every language and platform ever has had serious problems with multiple library versions. Anyone who has used an OS without a package manager knows this. I won't claim that npm doesn't also have problems, but at least its problems are different. Humanity can survive the existence of more than one paradigm for dealing with versioned libraries.
    - spankalee 10y ago
nawitus 10y ago
Can you use <script type="module"> with web components/Polymer (in a sane way, e.g. without duplicates being loaded like with the old <script> tag)?
- spankalee 10y ago
  You will be able to (somewhat) soon! Support is going to roll out in 3 steps: 1. A system that converts <script type=module> to HTML imports so that loading and ordering is consistent across JS and HTML imports. The converter will run in `polytool build` (polytool is the Polymer CLI in development) and in `polytool serve` (the dev server). 2a. Chrome's implementation of <script type=module> will work correctly with HTML imports, so that HTML imports wait on any JS modules and their transitive dependencies before signaling ready. 2b. The HTML imports polyfill will wait for JS modules to load (this might require no work). 3. HTML imports itself is being re-imagined as a layer on top of the JS module loader, so that HTML is just a different file type that can be loaded and participate in the dependency graph. This idea of "HTML Modules" has positive feedback from other browser vendors so far.
  - thegayngler 10y ago
    Thanks! Where can I read more about this?
  - deleted 10y ago
    [deleted]
- domenicd 10y ago
  Deduplication/memoization is built in to the spec, if that helps: https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-single-module-script https://html.spec.whatwg.org/multipage/webappapis.html#fetch... steps 2 and 3.
GroSacASacs 10y ago
There are many ways to load and execute JavaScript, static/dynamic adding a script tag to the html, html imports, manual fetch + eval , concatenation of javascript files in 1 file, in the future maybe the import syntax ? and now <script module> ? Do we really want to add more ways ? Complexity is an enemy of security, it will add more confusion. Maybe we should first disable/remove ways to load and execute JavaScript. We should also think, if what really need is a client solution or a better tool/system to transform scripts, in a script server side. Why, what should be clearly answered before thinking in terms of how. Sometimes having the bigger picture in your head helps.
- esailija 10y ago
  You basically cannot ever remove anything as it will break websites which will just make people switch to another browser where the websites work.
  - GroSacASacs 10y ago
    By remove, I meant "encourage to not do that". Like JSLint or "use strict"; didn't remove JavaScript possibilities, it encouraged us to not write insane JavaScript. They should consider making a document that lists all possibilities to organize code in modules and explain why some are betters than others, then in the same document explain the benefit for new syntax.
awalGarg 10y ago
This comment is gonna be bikeshedding, and definitely not a popular opinion in today's "modern" JS community, but here it goes anyways... I don't understand why TC39 had to define a new syntax, new semantics, and new restrictions for modules in JS land when the JS community had already created solutions for modules in JS. People were using modules in JS before they knew that "loader" or "static import/exports" were a thing to be landed in ES6 in future. And frankly, what the JS community made for themselves was a lot more saner than what is coming natively to JS, and even more so compared to the native solutions for module loading in some other languages (compare to Python for example). I don't think any JS programmer wanted modules natively in JS. And with HTTP2, they would have just stopped bundling and a newer async implementation of `require` would have been conjured. And all this "tree-shaking" etc. that is claimed to be possible only with the static syntax is blatantly incorrect. It is entirely possible to eliminate unused code with the commonjs/amd/umd way with a slightly more intelligent bundler. And the new "syntax" that they created is just a variant of destructuring. You can achieve nearly the same with ES6 destructuring and `require`. sigh
- matthewbauer 10y ago
  Well, the biggest issue with using "require" (commonjs) is it is by definition synchronous. That works okay for server side when files are accessible (not best practice though) but it gets really difficult with network resources. JS modules are supposed to be a way to get the advantages with aynchronous module loading (as in require.js) without all of the callback hell.
  - awalGarg 10y ago
    Again, that is just an issue with the current implementation of the commonjs require. You can, for example, have a require implementation which runs your code inside a `GeneratorFunction` instead of a regular `Function` and use `yield require(...)` instead of just `require`, where `require` returns a promise and is async, and use something like `co` or bluebird's `coroutine` etc.. There is also AMD which is by definition async and gives you the same static export/import as ES6, leave for the new syntax. It really is just that JS land hacked up something together and it worked great. Had the community went to lengths as much as people in the TC, we would have had much better everything, but it is a problem already solved enough that people don't care much.
nostrademons 10y ago
This is huge. Finalization of the module loader spec is usually what's cited as the last remaining blocker for full implementation of modules in V8 [1]. Combined with HTTP2 being available nearly everywhere [2] and native ES6 support soon reaching most mainstream browsers [3], it means that web developers will soon be able to drop a large portion of the current JS stack, including spriting, Webpack, and Babel. [1] https://bugs.chromium.org/p/v8/issues/detail?id=1569 https://bugs.chromium.org/p/v8/issues/detail?id=1569 [2] http://caniuse.com/#search=http2 http://caniuse.com/#search=http2 [3] https://kangax.github.io/compat-table/es6/ https://kangax.github.io/compat-table/es6/
- M4v3R 10y ago
  Re [3]: Nice, it seems that both Blink and Webkit are nearing 100% support for ES6. Safari Technology Preview already has 98%, so next major version of Safari (presumably both on desktop and on mobile) will have it too. So this autumn we can expect to have ES6 in every browser that matters!
  - nostrademons 10y ago
    Node 6.0 is also supposed to come out in late April [1], targeting v8 5.0, which IIRC corresponds to Chrome 50 on the compatibility table. Not quite as nice as the Chrome 52 release, but still covers most of the ES6 features I actually use. FF48 and Edge 14 don't do so badly either. If you drill down into their missing test cases, it's largely features that few people care about. [1] https://github.com/nodejs/node/issues/5766 https://github.com/nodejs/node/issues/5766
  - mmebane 10y ago
    > So this autumn we can expect to have ES6 in every browser that matters! I wish I could afford to ignore Internet Explorer... if Microsoft leaves Windows 7 to rot with IE 11, corporate users are going to keep holding back the web for a few more years.
  - kuschku 10y ago
    > So this autumn we can expect to have ES6 in every browser that matters! Since when does FF not matter anymore?

123 points by sonnyp 10y ago

If you don't find JavaScript modules confusing, you should. I work on a module loader [1] and I find it quite confusing. To clarify we have 3 things: 1. import and export syntax, this is defined in ES2015. It's just syntax, it doesn't explain how modules should be loaded. 2. The Loader spec (http://whatwg.github.io/loader/ http://whatwg.github.io/loader/) which defines a full loader including hooks to do just about anything you can possibly need. This allows you to do really crazy things like define modules dynamically and import then dynamically (which the static import/export syntax doesn't allow). We use this extensively in StealJS (or at least an older version of it) and it's pretty powerful. 3. import type=module is what this article is referring to. It defines a tag that allows you to import modules in the browser. You can't use a regular script tag, modules are different; they are strict mode by default for example. If you read the GitHub issue there was some discussion of whether import type=module should use the Loader spec. It probably should. But the Loader spec isn't done so... reading the spec and seeing things like the module map [2] it sounds like import type=script defines a miniature sized loader. I don't blame them for doing this, Loader has taken too long and people are clamoring for something. The Loader spec has a weird history; it was close to being done in ES2015 but when it was ejected it sit for a long time without much work. Then around last summer it was heavily worked on and got pretty far. But now as it sits, no commits like the last 2 and a half months [3] At this point I wouldn't be surprised to see Loader go away. It's probably too controversially and too big to ever get done. I imagine the web will just have the import type=module loader and Node will do it's own this. This is going to make writing portable JavaScript as annoying going forward as it's always been. [1] http://stealjs.com/ http://stealjs.com/ [2] https://html.spec.whatwg.org/multipage/webappapis.html#module-map https://html.spec.whatwg.org/multipage/webappapis.html#modul... [3] https://github.com/whatwg/loader/commits https://github.com/whatwg/loader/commits

CraigJPerry 10y ago
>> This allows you to do really crazy things like define modules dynamically In python there's a library called "sh" and it has a novelty interface: from sh import ls, pwd, my_unique_command print ls('/dir') ... I wondered (not enough to check right enough) if something similar could be done in JS. Although due to the async nature of such a call, it might not end up looking so familiar with JS in practice i guess.
- redka 10y ago
  https://github.com/shelljs/shelljs https://github.com/shelljs/shelljs Edit: \/ ahh, yes you're right!
  - foota 10y ago
    I think if I'm grokking correctly that you can import "anything" from sh, without it having to be explicitly in the package.
spankalee 10y ago
One detail that's often lost in the discussion (that you might have been hinting at) is that without a loader modules can only be referred to by URL. Node-style name resolution won't work at all. This means that modules will either still have to be compiled with something like browserify that resolves modules names into URLs ahead of time (a solution I don't particularly like), or modules will just have to use URLs to import other modules, leading to a convention of assuming that modules are all siblings so that you import like this: import * as jquery from '../jquery/jquery.js'; Despite the extra verbosity, I prefer this since it requires that modules are flat and de-duplicated and doesn't require any resolution step at all.
- MatthewPhillips 10y ago
  Modules are not flat in reality though, any non-trivial sized app likely has multiple versions of the same module being used. I don't see any reason to think that's going to change in the future, extreme modularity is the norm in JavaScript these days.
  - josteink 10y ago
    It may be the norm right now (since npm came around), but that's no reason to excuse it for what it is: a mess. A mess we should be able to do without. If we can have an official module-implementation which somehow forces developers to get a grip on their dependencies (which is the exact opposite of what "modern" js developers are doing), that can only be a good thing. The cowboy days must surely soon be over?
    - jessaustin 10y ago
      Every language and platform ever has had serious problems with multiple library versions. Anyone who has used an OS without a package manager knows this. I won't claim that npm doesn't also have problems, but at least its problems are different. Humanity can survive the existence of more than one paradigm for dealing with versioned libraries.
  - spankalee 10y ago

nawitus 10y ago

Can you use <script type="module"> with web components/Polymer (in a sane way, e.g. without duplicates being loaded like with the old <script> tag)?

spankalee 10y ago
You will be able to (somewhat) soon! Support is going to roll out in 3 steps: 1. A system that converts <script type=module> to HTML imports so that loading and ordering is consistent across JS and HTML imports. The converter will run in `polytool build` (polytool is the Polymer CLI in development) and in `polytool serve` (the dev server). 2a. Chrome's implementation of <script type=module> will work correctly with HTML imports, so that HTML imports wait on any JS modules and their transitive dependencies before signaling ready. 2b. The HTML imports polyfill will wait for JS modules to load (this might require no work). 3. HTML imports itself is being re-imagined as a layer on top of the JS module loader, so that HTML is just a different file type that can be loaded and participate in the dependency graph. This idea of "HTML Modules" has positive feedback from other browser vendors so far.
- thegayngler 10y ago
  Thanks! Where can I read more about this?
- deleted 10y ago
  [deleted]
domenicd 10y ago
Deduplication/memoization is built in to the spec, if that helps: https://html.spec.whatwg.org/multipage/webappapis.html#fetch-a-single-module-script https://html.spec.whatwg.org/multipage/webappapis.html#fetch... steps 2 and 3.

GroSacASacs 10y ago

There are many ways to load and execute JavaScript, static/dynamic adding a script tag to the html, html imports, manual fetch + eval , concatenation of javascript files in 1 file, in the future maybe the import syntax ? and now <script module> ? Do we really want to add more ways ? Complexity is an enemy of security, it will add more confusion. Maybe we should first disable/remove ways to load and execute JavaScript. We should also think, if what really need is a client solution or a better tool/system to transform scripts, in a script server side. Why, what should be clearly answered before thinking in terms of how. Sometimes having the bigger picture in your head helps.

esailija 10y ago
You basically cannot ever remove anything as it will break websites which will just make people switch to another browser where the websites work.
- GroSacASacs 10y ago
  By remove, I meant "encourage to not do that". Like JSLint or "use strict"; didn't remove JavaScript possibilities, it encouraged us to not write insane JavaScript. They should consider making a document that lists all possibilities to organize code in modules and explain why some are betters than others, then in the same document explain the benefit for new syntax.

awalGarg 10y ago

This comment is gonna be bikeshedding, and definitely not a popular opinion in today's "modern" JS community, but here it goes anyways... I don't understand why TC39 had to define a new syntax, new semantics, and new restrictions for modules in JS land when the JS community had already created solutions for modules in JS. People were using modules in JS before they knew that "loader" or "static import/exports" were a thing to be landed in ES6 in future. And frankly, what the JS community made for themselves was a lot more saner than what is coming natively to JS, and even more so compared to the native solutions for module loading in some other languages (compare to Python for example). I don't think any JS programmer wanted modules natively in JS. And with HTTP2, they would have just stopped bundling and a newer async implementation of `require` would have been conjured. And all this "tree-shaking" etc. that is claimed to be possible only with the static syntax is blatantly incorrect. It is entirely possible to eliminate unused code with the commonjs/amd/umd way with a slightly more intelligent bundler. And the new "syntax" that they created is just a variant of destructuring. You can achieve nearly the same with ES6 destructuring and `require`. sigh

matthewbauer 10y ago
Well, the biggest issue with using "require" (commonjs) is it is by definition synchronous. That works okay for server side when files are accessible (not best practice though) but it gets really difficult with network resources. JS modules are supposed to be a way to get the advantages with aynchronous module loading (as in require.js) without all of the callback hell.
- awalGarg 10y ago
  Again, that is just an issue with the current implementation of the commonjs require. You can, for example, have a require implementation which runs your code inside a `GeneratorFunction` instead of a regular `Function` and use `yield require(...)` instead of just `require`, where `require` returns a promise and is async, and use something like `co` or bluebird's `coroutine` etc.. There is also AMD which is by definition async and gives you the same static export/import as ES6, leave for the new syntax. It really is just that JS land hacked up something together and it worked great. Had the community went to lengths as much as people in the TC, we would have had much better everything, but it is a problem already solved enough that people don't care much.

nostrademons 10y ago

This is huge. Finalization of the module loader spec is usually what's cited as the last remaining blocker for full implementation of modules in V8 [1]. Combined with HTTP2 being available nearly everywhere [2] and native ES6 support soon reaching most mainstream browsers [3], it means that web developers will soon be able to drop a large portion of the current JS stack, including spriting, Webpack, and Babel. [1] https://bugs.chromium.org/p/v8/issues/detail?id=1569 https://bugs.chromium.org/p/v8/issues/detail?id=1569 [2] http://caniuse.com/#search=http2 http://caniuse.com/#search=http2 [3] https://kangax.github.io/compat-table/es6/ https://kangax.github.io/compat-table/es6/

M4v3R 10y ago
Re [3]: Nice, it seems that both Blink and Webkit are nearing 100% support for ES6. Safari Technology Preview already has 98%, so next major version of Safari (presumably both on desktop and on mobile) will have it too. So this autumn we can expect to have ES6 in every browser that matters!
- nostrademons 10y ago
  Node 6.0 is also supposed to come out in late April [1], targeting v8 5.0, which IIRC corresponds to Chrome 50 on the compatibility table. Not quite as nice as the Chrome 52 release, but still covers most of the ES6 features I actually use. FF48 and Edge 14 don't do so badly either. If you drill down into their missing test cases, it's largely features that few people care about. [1] https://github.com/nodejs/node/issues/5766 https://github.com/nodejs/node/issues/5766
- mmebane 10y ago
  > So this autumn we can expect to have ES6 in every browser that matters! I wish I could afford to ignore Internet Explorer... if Microsoft leaves Windows 7 to rot with IE 11, corporate users are going to keep holding back the web for a few more years.
- kuschku 10y ago
  > So this autumn we can expect to have ES6 in every browser that matters! Since when does FF not matter anymore?