5 ms·

How we run NPM packages in the browser

97 points by judofyr 9y ago

spankalee 9y ago
Hopefully all this madness will be over soon when people start publishing JavaScript modules to npm so we don't need WebPack for development anymore.
- ricardobeat 9y ago
  Can you explain? Webpack actually uses modules from npm. Distribution is a separate matter to bundling. Not sure what you’re getting at.
  - denisw 9y ago
    I think spankalee meant NPM packages being distributed as ES6 modules rather than ES5 source files that use require() or are pre-bundled with something like Webpack or Rollup (as is the case for many frontend packages). Node.js has support for ES6 modules since version 8.5.0 [1], so this will slowly become feasible. [1]: http://2ality.com/2017/09/native-esm-node.html http://2ality.com/2017/09/native-esm-node.html
    - ricardobeat 9y ago
      That doesn’t change the need for bundlers like webpack, at least not until we have http2 everywhere.
      - breatheoften 9y ago
        I will believe http2 can efficiently replace a module bundling step when I see it ... unless esm also magically replaces the small module mentality of the npm ecosystem there will remain a ridiculously large number of small packages in the module graph of apps using npm packages ... turning module import into an async operation without a bundler is going to require a ridiculous number of round trips ... somehow dynamically building the bundling mechanism into n*m server layers sounds like even more of a nightmare to me than webpack ... Also — even if you make every module import async — why would you want to get rid of the ahead of time bundler? Walking the module graph ahead of time is enormously useful for static analysis ... it seems to me like asynchronously loading a module is a special case file size optimization not the normal usage of a module in a program ... Do people really see mjs and http2 as removing the need for module bundlers? I ask because I’m honestly extremely concerned — I thought that javascript modules were eventually going to grow a new syntax for async module loading not turn every import into an async operation. I don’t understand the vision being pursued here ...
      - spankalee 9y ago
        It means we don't need them at development time. Just use modules (and an http/2 dev server for better multiplexing), then do bundling and other optimizations only for production. It also means that project like in the article can mostly just go away - you don't need a custom loader.
xixixao 9y ago
This looks amazing, but trying the Playground there seems to be an issue when trying to rerun programs, giving: "TransactionInactiveError: A request was placed against a transaction which is currently not active, or which is finished."
- somebee 9y ago
  I posted an issue about it here: https://github.com/scrimba/community/issues/110 https://github.com/scrimba/community/issues/110. We'll look into it :)
porsager 9y ago
This was a great read, and a pleasure to follow your path to the end result. I think it would be great if you would Open Source it. I've been working some time on a web playground ( https://flems.io https://flems.io ). It is based on an open source single file module (https://github.com/porsager/flems https://github.com/porsager/flems) that anyone can use to make a live showcase for their documentation / examples and what not. Flems.io simply uses unpkg.com directly. Unpkg.com does it's own resolution by first looking for an `unpkg` field, then `browser` and last `main` in package.json. This works great since most browser packages also comes with a browser distribution. If the package doesn't point to a browser compatible bundle you can always go directly to a working one, like in the case of react, which would be used like this: https://goo.gl/b4546q https://goo.gl/b4546q
- pygy_ 9y ago
  Mithriliites assemble! (Flems is built with Mithril, and Per Harald Borgen from Scrimba digs the framework as well)
TAForObvReasons 9y ago
> The project is called "mrmanager" and is currently not open-sourced, but if there's interest we can make it happen. If the performance profile is as asserted "From a hot cache, mrmanager is capable of executing React (with ReactDOM) in 9ms instead of SystemJS' 150ms.", I and many others would likely benefit from making the loader open source
- glibgil 9y ago
  Yes, please open source this. Also, it wasn't clear from the article exactly which part needs to run on the server. Does anyone understand that part?
tarr11 9y ago
JavaScript modules are supported by default in Chrome 61 https://www.chromestatus.com/feature/5365692190687232 https://www.chromestatus.com/feature/5365692190687232
js4all 9y ago
Am I the only one who thinks running scripts on a site directly from npm is a bad idea? What happens when an incompatible change in any of the dependencies is pushed. What if a package owner had transferred his ownership und the new owner pushes whatever he wants.
- pygy_ 9y ago
  For one off examples it is just convenient. For long-lasting demo, it would be wise to pin the versions.
- porsager 9y ago
  Using unpkg.com as an example they redirect from the root URL of a package to the latest version. For instance https://unpkg.com/mithril https://unpkg.com/mithril will redirect to https://unpkg.com/mithril@1.1.4/mithril.js https://unpkg.com/mithril@1.1.4/mithril.js For example if adding a URL on https://flems.io https://flems.io it will store the xhr.responseURL to take advantage of this to pin the version. I would guess guess Scrimba does something similar?
  - pygy_ 9y ago
    Wow, so you auto-pin, that's great!
wcr3 9y ago
"Yarn is a package manager for NPM" HAHAHAHAHHAH
- ianjsikes 9y ago
  What's so funny? Yarn uses the npm package registry, so it is a package manager for npm.
  - owebmaster 9y ago
    A package manager for a package manager is indeed a funny thing that shows the overengineering of the JS ecosystem. Facebook could improve npm but they would not profit too much from it so yarn.
    - favadi 9y ago
      Npm is both a package management and a package registry.
      - owebmaster 9y ago
        Exactly! But mainly, a package manager. If you create a project that tries to replace it and at the same time needs it to exist, I guess funny is a minimum you can say about.
        allover 9y ago
        It's not funny at all. It's totally logical that you'd have competing clients.
    - allover 9y ago
      > A package manager for a package manager It's not, it's a package manager for a package registry. > that shows the overengineering of the JS ecosystem Nope. > Facebook could improve npm but they would not profit too much from it so yarn Explain? 1. I don't see how Facebook profits from Yarn. 2. The npm cli is hard to contribute to, the npm team themselves admit that, so it's fairly obvious to me why it made more sense to build a new client at the time, given the design goals of Yarn.
tuukkah 9y ago
An increasingly relevant followup: how to run NPM packages in a React Native app?
- pygy_ 9y ago
  Why not use a bunlder? Webpack, Rollup, Browserify... You'd want to load it dynamically over the network? That sounds like a very bad plan (tying your app uptime to the one of the NPM servers).
  - tuukkah 9y ago
    It's not as simple as using a bundler because the React Native runtime is unlike Node.js or common browsers regarding available APIs.
grizzles 9y ago
Just thinking out loud here. What about a stdlib framework that emulates node.js (eg. using BrowserFS) in the browser along with an optional stdlib minifier/zapper based on package.json. It seems like that approach would eliminate quite a bit of complexity.

97 points by judofyr 9y ago

Hopefully all this madness will be over soon when people start publishing JavaScript modules to npm so we don't need WebPack for development anymore.

ricardobeat 9y ago
Can you explain? Webpack actually uses modules from npm. Distribution is a separate matter to bundling. Not sure what you’re getting at.
- denisw 9y ago
  I think spankalee meant NPM packages being distributed as ES6 modules rather than ES5 source files that use require() or are pre-bundled with something like Webpack or Rollup (as is the case for many frontend packages). Node.js has support for ES6 modules since version 8.5.0 [1], so this will slowly become feasible. [1]: http://2ality.com/2017/09/native-esm-node.html http://2ality.com/2017/09/native-esm-node.html
  - ricardobeat 9y ago
    That doesn’t change the need for bundlers like webpack, at least not until we have http2 everywhere.
    - breatheoften 9y ago
      I will believe http2 can efficiently replace a module bundling step when I see it ... unless esm also magically replaces the small module mentality of the npm ecosystem there will remain a ridiculously large number of small packages in the module graph of apps using npm packages ... turning module import into an async operation without a bundler is going to require a ridiculous number of round trips ... somehow dynamically building the bundling mechanism into n*m server layers sounds like even more of a nightmare to me than webpack ... Also — even if you make every module import async — why would you want to get rid of the ahead of time bundler? Walking the module graph ahead of time is enormously useful for static analysis ... it seems to me like asynchronously loading a module is a special case file size optimization not the normal usage of a module in a program ... Do people really see mjs and http2 as removing the need for module bundlers? I ask because I’m honestly extremely concerned — I thought that javascript modules were eventually going to grow a new syntax for async module loading not turn every import into an async operation. I don’t understand the vision being pursued here ...
    - spankalee 9y ago
      It means we don't need them at development time. Just use modules (and an http/2 dev server for better multiplexing), then do bundling and other optimizations only for production. It also means that project like in the article can mostly just go away - you don't need a custom loader.

xixixao 9y ago

This looks amazing, but trying the Playground there seems to be an issue when trying to rerun programs, giving: "TransactionInactiveError: A request was placed against a transaction which is currently not active, or which is finished."

somebee 9y ago
I posted an issue about it here: https://github.com/scrimba/community/issues/110 https://github.com/scrimba/community/issues/110. We'll look into it :)

porsager 9y ago

This was a great read, and a pleasure to follow your path to the end result. I think it would be great if you would Open Source it. I've been working some time on a web playground ( https://flems.io https://flems.io ). It is based on an open source single file module (https://github.com/porsager/flems https://github.com/porsager/flems) that anyone can use to make a live showcase for their documentation / examples and what not. Flems.io simply uses unpkg.com directly. Unpkg.com does it's own resolution by first looking for an `unpkg` field, then `browser` and last `main` in package.json. This works great since most browser packages also comes with a browser distribution. If the package doesn't point to a browser compatible bundle you can always go directly to a working one, like in the case of react, which would be used like this: https://goo.gl/b4546q https://goo.gl/b4546q

pygy_ 9y ago
Mithriliites assemble! (Flems is built with Mithril, and Per Harald Borgen from Scrimba digs the framework as well)

TAForObvReasons 9y ago

> The project is called "mrmanager" and is currently not open-sourced, but if there's interest we can make it happen. If the performance profile is as asserted "From a hot cache, mrmanager is capable of executing React (with ReactDOM) in 9ms instead of SystemJS' 150ms.", I and many others would likely benefit from making the loader open source

glibgil 9y ago
Yes, please open source this. Also, it wasn't clear from the article exactly which part needs to run on the server. Does anyone understand that part?

tarr11 9y ago

JavaScript modules are supported by default in Chrome 61 https://www.chromestatus.com/feature/5365692190687232 https://www.chromestatus.com/feature/5365692190687232

js4all 9y ago

Am I the only one who thinks running scripts on a site directly from npm is a bad idea? What happens when an incompatible change in any of the dependencies is pushed. What if a package owner had transferred his ownership und the new owner pushes whatever he wants.

pygy_ 9y ago
For one off examples it is just convenient. For long-lasting demo, it would be wise to pin the versions.
porsager 9y ago
Using unpkg.com as an example they redirect from the root URL of a package to the latest version. For instance https://unpkg.com/mithril https://unpkg.com/mithril will redirect to https://unpkg.com/mithril@1.1.4/mithril.js https://unpkg.com/mithril@1.1.4/mithril.js For example if adding a URL on https://flems.io https://flems.io it will store the xhr.responseURL to take advantage of this to pin the version. I would guess guess Scrimba does something similar?
- pygy_ 9y ago
  Wow, so you auto-pin, that's great!

wcr3 9y ago

"Yarn is a package manager for NPM" HAHAHAHAHHAH

ianjsikes 9y ago
What's so funny? Yarn uses the npm package registry, so it is a package manager for npm.
- owebmaster 9y ago
  A package manager for a package manager is indeed a funny thing that shows the overengineering of the JS ecosystem. Facebook could improve npm but they would not profit too much from it so yarn.
  - favadi 9y ago
    Npm is both a package management and a package registry.
    - owebmaster 9y ago
      Exactly! But mainly, a package manager. If you create a project that tries to replace it and at the same time needs it to exist, I guess funny is a minimum you can say about.
      - allover 9y ago
        It's not funny at all. It's totally logical that you'd have competing clients.
  - allover 9y ago
    > A package manager for a package manager It's not, it's a package manager for a package registry. > that shows the overengineering of the JS ecosystem Nope. > Facebook could improve npm but they would not profit too much from it so yarn Explain? 1. I don't see how Facebook profits from Yarn. 2. The npm cli is hard to contribute to, the npm team themselves admit that, so it's fairly obvious to me why it made more sense to build a new client at the time, given the design goals of Yarn.

tuukkah 9y ago

An increasingly relevant followup: how to run NPM packages in a React Native app?

pygy_ 9y ago
Why not use a bunlder? Webpack, Rollup, Browserify... You'd want to load it dynamically over the network? That sounds like a very bad plan (tying your app uptime to the one of the NPM servers).
- tuukkah 9y ago
  It's not as simple as using a bundler because the React Native runtime is unlike Node.js or common browsers regarding available APIs.

grizzles 9y ago

Just thinking out loud here. What about a stdlib framework that emulates node.js (eg. using BrowserFS) in the browser along with an optional stdlib minifier/zapper based on package.json. It seems like that approach would eliminate quite a bit of complexity.