7 ms·

PR converts the Three.js codebase to ES2015 modules

134 points by based2 10y ago

Fifer82 10y ago
FFS "Rollup", I gather there was not enough choice in this area, so why not create YET ANOTHER ONE.
moron4hire 10y ago
> The solution I've used here is to put the KeyframeTrack prototype and constructor function in separate files, which allows for a stable sort. Thanks. I've always wanted to organize my JavaScript projects like C projects. It's an interesting exercise, but now we have to learn how to use Rollup to hack on Three.js (I run my own fork with optimizations built in for stereo rendering, and know a few other people who do similar things). I also fundamentally disagree that plugin repos are a good idea: part of the appeal of Three.js is that it is a drop-in file that you can use anywhere for 3D graphics.
- nawitus 10y ago
  Why do you need to learn Rollup, assuming that the build system is already configured for you?
- rich_harris 10y ago
  > I also fundamentally disagree that plugin repos are a good idea Part of the reason I was motivated to do this work is that I love Three's batteries-included approach – I don't need to spend a long time configuring my project or trying to figure out how to glue different libraries together (I've written more about this here https://medium.com/@Rich_Harris/small-modules-it-s-not-quite-that-simple-3ca532d65de4#.sckmgwnd6 https://medium.com/@Rich_Harris/small-modules-it-s-not-quite...). Nothing has changed vis-a-vis Three being a drop-in file. But it's a double-edged sword – batteries-included historically means including code that you're not using in your project. ES modules give us the best of both worlds via tree-shaking. And once you've learned them (you're not learning Rollup, you're learning JavaScript – this is a standard we're talking about) you'll find it's much more logical than the alternative. Trust me! The KeyframeTrack thing is just a special case resulting from the codebase's history. It could easily be fixed more elegantly, it was just outside the scope of this PR.
  - moron4hire 10y ago
    So serious questions here, because I actually want to solve these problems, but I haven't found a good solution yet. I build a framework that runs on top of Three.js. I need to not only consume Three.js and make a bundle for demos, but I also need to create a library bundle that other people can consume. Every time I've looked into Webpack or Browserify to try to do this, the only documented use cases I've seen have been for consuming, not publishing libraries. What would you suggest as a workflow for providing drop-in packages? Most of my users are not very technical and usually don't have Node installed. They might be semi-comfortable getting a basic, essentially static site going (though they are often running on RoR), or they're hacking on scripts almost exclusively on CodePen. They want to be able to script-tag include JS files and get to work on their projects.
seanwilson 10y ago
Is there a way to automated these kinds of conversions? A lot of the line changes are adding "THREE." in front of the right identifier.
- cjcenizal 10y ago
  https://github.com/facebook/codemod https://github.com/facebook/codemod
- bhouston 10y ago
  It was done via a script in this PR. :)
bhouston 10y ago
It should be clear that this conversion was primarily done via a script. So it is mostly an automated conversion. If you are running forks of ThreeJS, like we are with https://Clara.io https://Clara.io, you can run the same conversion script on your repo and then merge in the new ThreeJS. I haven't done it yet, hopefully in the coming week, but it should mostly work. Next up I hope is moving ThreeJS to full ES6 classes. :)
- zfedoran 10y ago
  It's actually a lot of fun to write a script like the one they used. I highly recommend it if you have a dependency that is large and not split up into modules. I dabbled with converting Three.js into modules through a script a couple years ago, took only an afternoon to get it working initially. I'll have to take a look to see how the official version compares to my hack of an attempt. If Inyone else is curious: https://github.com/zfedoran/modulite-three.js/blob/master/auto-modulite.js https://github.com/zfedoran/modulite-three.js/blob/master/au...
apo 10y ago
I've used Rollup and it worked extremely well for bundling JS that uses modules into a single file. The documentation is very good and the community is quite responsive. The only thing that would be better is not needing Rollup in the first place. What's the current status of browser support for ES6 modules?
- maga 10y ago
  In development in all browsers[1] meaning none has them yet. IIRC, one of the major roadblocks was the need to define <script type="module"> in the HTML standard[2] as well as the whole loading routine. Hence, the feature is coming later than the rest in ES2015. [1] https://www.chromestatus.com/features/5365692190687232 https://www.chromestatus.com/features/5365692190687232 [2] https://github.com/whatwg/html/pull/443 https://github.com/whatwg/html/pull/443
- MatthewPhillips 10y ago
  It's available in Edge if you have Windows Insider (the Edge equivalent of Chrome Canary). There are some bugs there though. Note that the browser's module support is stricter than Node's, you have to point your imports to the specific file you want to import and include `.js` on the end, for example this won't work: https://github.com/mrdoob/three.js/blob/dev/src/materials/Material.js#L1 https://github.com/mrdoob/three.js/blob/dev/src/materials/Ma... in script[type=module]. So there will still need to be a refactor of THREE.js even after browser support starts coming in.
- dexwiz 10y ago
  We probably won't see module, or a good module, until big HTTP2 support. Having a single file has some huge gains for HTTP, mostly in the overhead cost of a request. HTTP2 will get rid of some of that overhead via streaming files and header compression. Sending a ton of different javascript files is too costly right now.
sergiotapia 10y ago
Rollup huh, I guess Webpack is now too de jour and will join Gulp in the pit that is Old Javascript.
- mrgalaxy 10y ago
  Actually Webpack and Rollup go hand-in-hand. Rollup just compiles JavaScript ES6 modules into a single file, dealing with only `import` and `export` statements and some tree-shaking. Webpack handles everything else, like ES6-to-ES5 conversion, bundling dependencies, styles and everything else you need to get your code shippable to browsers and node.
  - sergiotapia 10y ago
    >Actually Webpack and Rollup go hand-in-hand. Rollup just compiles JavaScript ES6 modules into a single file Isn't that webpacks gig. Or what that gulp? Or was that browserify? Or was that Jello? Or was that...
    - draw_down 10y ago
      Doesn't seem so terrible to me for people to have a choice in what tool they use.
    - nilliams 10y ago
      You seem to be complaining that innovation is happening. If you don't want to deal with the inevitable 'options' that are going to be offered up by a community as large as the JS community, you could 1. pick a smaller community, 2. stick to old tech (e.g. at our company we still have an app running on RequireJS that runs just fine -- RequireJS is over 6 years old). 3. calmly re-assess tech choices every year or so.
      - sergiotapia 10y ago
        >You seem to be complaining that innovation is happening. Not at all. I'm complaining that all of these seemingly similar tools keep superseding each other and solve...what exactly.
superswordfish 10y ago
I wonder what justifies turning normal code into a string to eval(): https://github.com/mrdoob/three.js/pull/9310/files#diff-2ac7ae14bafb96167d6b480d86c885feL5 https://github.com/mrdoob/three.js/pull/9310/files#diff-2ac7... becomes https://github.com/mrdoob/three.js/pull/9310/files#diff-ff6e5f22a9c7e66987b19c0199636480R4 https://github.com/mrdoob/three.js/pull/9310/files#diff-ff6e... On the surface it seems bad.
- espadrine 10y ago
  This does not look like an eval. It seems to be a script that generates a JS file. The string is not eval'ed, it is concatenated.
  - superswordfish 10y ago
    Good point, either way it is ugly that normal code is now a string. It is in rollup.config.js so maybe this is imposed by "Rollup", but I'm not about to go find how the "outro" option differs from the "footer" option in yet another JS build tool.
- rich_harris 10y ago
  AudioContext is a special case – unlike most of the THREE.* exports, THREE.AudioContext is a getter (which initializes a Web Audio context). ES module namespaces can't have getters, so for backwards compatibility in the generated UMD bundle it was necessary to add that snippet to the bottom.
erikpukinskis 10y ago
Why is this a good thing? Just seems like different syntax to do the same thing. You can theoretically dynamically resolve the different exports asynhronously, but you could do that by exporting functions before. ES6 makes me think I am going crazy. Everyone seems so excited to have new sugar and a ton of busywork upgrading code and resolving runtime incompatibilities. I have yet to see any feature that seems like an actual benefit to the conservative working programmer. Maybe destructuring.
- lhnz 10y ago
  Smaller file sizes.
- bsou 10y ago
  eslint also provides static analysis for dependency resolution when using import statements instead of node require calls
  - jacobr 10y ago
    eslint-plugin-import found several bugs for us when we enabled it. Can't recommend it enough. https://github.com/benmosher/eslint-plugin-import https://github.com/benmosher/eslint-plugin-import
- smitherfield 10y ago
  I'd say constants are pretty big.
gedy 10y ago
Awesome, virtually all of Rich Harris' work is pure gold - this, Rollup, RactiveJS, etc, etc. Pragmatic and solves real development problems.
- typpytyper 10y ago
  Don't forget his babel replacement, buble: https://gitlab.com/Rich-Harris/buble https://gitlab.com/Rich-Harris/buble Zero config, small and fast as heck.
stevebmark 10y ago
It's hard to see this as anything other than an ad for rollup. There's been many attempts to modernize the Three.js codebase. Not sure why this is the one that went through. I'm all for bringing Three.js up to date and applying modern JS tools to it. The maintainers of Three usually push back against these efforts. This PR still doesn't address the many fundamental problems with the quality of Three's source code and the attitudes of the maintainers. I wouldn't personally use Three on a large project, the low quality ecosystem will eventually harm you. It's designed as library to spin up small projects quickly. It's good at that. Beyond that I wouldn't recommend it. For the record I believe Babylon (another WebGL utility library) has similar problems with the source code. I don't have a suggestion for the "best" WebGL library to use.
- TheRealPomax 10y ago
  Is it, though? If "Hi, I'm the author of the ONLY tool that does bundling based on how ES2015 modules work, and applied that to your codebase, which made it both more useful to future developers and made it smaller as bundle" is an ad, I'd like to see more ads everywhere. Other than rollup there are no bundlers that perform ES2015 module bundling, which means there is no efficient tree shaking, no load-order independency, etc. I suspect the reason it went through is because it _actually_ modernized the codebase: unlike previous ideas, just swapping ES5 notation for a newer slightly more efficient ES6 notation, this refactor actually changes the load, parse, and bundle efficiency of the codebase, from a non-standard module style to what we've now collectively agreed on JS-native module should look like; something that before ES6 simply did not exist. There were choices made by individuals or organizations (commonjs, umd, etc), but only as of ES6 are modules _actually_ a formal construct in JS, with an official spec that formalizes how imports work. Any code changes that bring older codebases in line with the new spec are fantastic: this codebase just got massively future-proofed without any impact whatsoever on its integrity. (the one example that didn't run turned out to be a bug in the current example, not in the refactor)

134 points by based2 10y ago

FFS "Rollup", I gather there was not enough choice in this area, so why not create YET ANOTHER ONE.

> The solution I've used here is to put the KeyframeTrack prototype and constructor function in separate files, which allows for a stable sort. Thanks. I've always wanted to organize my JavaScript projects like C projects. It's an interesting exercise, but now we have to learn how to use Rollup to hack on Three.js (I run my own fork with optimizations built in for stereo rendering, and know a few other people who do similar things). I also fundamentally disagree that plugin repos are a good idea: part of the appeal of Three.js is that it is a drop-in file that you can use anywhere for 3D graphics.

nawitus 10y ago
Why do you need to learn Rollup, assuming that the build system is already configured for you?
rich_harris 10y ago
> I also fundamentally disagree that plugin repos are a good idea Part of the reason I was motivated to do this work is that I love Three's batteries-included approach – I don't need to spend a long time configuring my project or trying to figure out how to glue different libraries together (I've written more about this here https://medium.com/@Rich_Harris/small-modules-it-s-not-quite-that-simple-3ca532d65de4#.sckmgwnd6 https://medium.com/@Rich_Harris/small-modules-it-s-not-quite...). Nothing has changed vis-a-vis Three being a drop-in file. But it's a double-edged sword – batteries-included historically means including code that you're not using in your project. ES modules give us the best of both worlds via tree-shaking. And once you've learned them (you're not learning Rollup, you're learning JavaScript – this is a standard we're talking about) you'll find it's much more logical than the alternative. Trust me! The KeyframeTrack thing is just a special case resulting from the codebase's history. It could easily be fixed more elegantly, it was just outside the scope of this PR.
- moron4hire 10y ago
  So serious questions here, because I actually want to solve these problems, but I haven't found a good solution yet. I build a framework that runs on top of Three.js. I need to not only consume Three.js and make a bundle for demos, but I also need to create a library bundle that other people can consume. Every time I've looked into Webpack or Browserify to try to do this, the only documented use cases I've seen have been for consuming, not publishing libraries. What would you suggest as a workflow for providing drop-in packages? Most of my users are not very technical and usually don't have Node installed. They might be semi-comfortable getting a basic, essentially static site going (though they are often running on RoR), or they're hacking on scripts almost exclusively on CodePen. They want to be able to script-tag include JS files and get to work on their projects.

seanwilson 10y ago

Is there a way to automated these kinds of conversions? A lot of the line changes are adding "THREE." in front of the right identifier.

cjcenizal 10y ago
https://github.com/facebook/codemod https://github.com/facebook/codemod
bhouston 10y ago
It was done via a script in this PR. :)

bhouston 10y ago

It should be clear that this conversion was primarily done via a script. So it is mostly an automated conversion. If you are running forks of ThreeJS, like we are with https://Clara.io https://Clara.io, you can run the same conversion script on your repo and then merge in the new ThreeJS. I haven't done it yet, hopefully in the coming week, but it should mostly work. Next up I hope is moving ThreeJS to full ES6 classes. :)

zfedoran 10y ago
It's actually a lot of fun to write a script like the one they used. I highly recommend it if you have a dependency that is large and not split up into modules. I dabbled with converting Three.js into modules through a script a couple years ago, took only an afternoon to get it working initially. I'll have to take a look to see how the official version compares to my hack of an attempt. If Inyone else is curious: https://github.com/zfedoran/modulite-three.js/blob/master/auto-modulite.js https://github.com/zfedoran/modulite-three.js/blob/master/au...

apo 10y ago

I've used Rollup and it worked extremely well for bundling JS that uses modules into a single file. The documentation is very good and the community is quite responsive. The only thing that would be better is not needing Rollup in the first place. What's the current status of browser support for ES6 modules?

maga 10y ago
In development in all browsers[1] meaning none has them yet. IIRC, one of the major roadblocks was the need to define <script type="module"> in the HTML standard[2] as well as the whole loading routine. Hence, the feature is coming later than the rest in ES2015. [1] https://www.chromestatus.com/features/5365692190687232 https://www.chromestatus.com/features/5365692190687232 [2] https://github.com/whatwg/html/pull/443 https://github.com/whatwg/html/pull/443
MatthewPhillips 10y ago
It's available in Edge if you have Windows Insider (the Edge equivalent of Chrome Canary). There are some bugs there though. Note that the browser's module support is stricter than Node's, you have to point your imports to the specific file you want to import and include `.js` on the end, for example this won't work: https://github.com/mrdoob/three.js/blob/dev/src/materials/Material.js#L1 https://github.com/mrdoob/three.js/blob/dev/src/materials/Ma... in script[type=module]. So there will still need to be a refactor of THREE.js even after browser support starts coming in.
dexwiz 10y ago
We probably won't see module, or a good module, until big HTTP2 support. Having a single file has some huge gains for HTTP, mostly in the overhead cost of a request. HTTP2 will get rid of some of that overhead via streaming files and header compression. Sending a ton of different javascript files is too costly right now.

sergiotapia 10y ago

Rollup huh, I guess Webpack is now too de jour and will join Gulp in the pit that is Old Javascript.

mrgalaxy 10y ago
Actually Webpack and Rollup go hand-in-hand. Rollup just compiles JavaScript ES6 modules into a single file, dealing with only `import` and `export` statements and some tree-shaking. Webpack handles everything else, like ES6-to-ES5 conversion, bundling dependencies, styles and everything else you need to get your code shippable to browsers and node.
- sergiotapia 10y ago
  >Actually Webpack and Rollup go hand-in-hand. Rollup just compiles JavaScript ES6 modules into a single file Isn't that webpacks gig. Or what that gulp? Or was that browserify? Or was that Jello? Or was that...
  - draw_down 10y ago
    Doesn't seem so terrible to me for people to have a choice in what tool they use.
  - nilliams 10y ago
    You seem to be complaining that innovation is happening. If you don't want to deal with the inevitable 'options' that are going to be offered up by a community as large as the JS community, you could 1. pick a smaller community, 2. stick to old tech (e.g. at our company we still have an app running on RequireJS that runs just fine -- RequireJS is over 6 years old). 3. calmly re-assess tech choices every year or so.
    - sergiotapia 10y ago
      >You seem to be complaining that innovation is happening. Not at all. I'm complaining that all of these seemingly similar tools keep superseding each other and solve...what exactly.

superswordfish 10y ago

I wonder what justifies turning normal code into a string to eval(): https://github.com/mrdoob/three.js/pull/9310/files#diff-2ac7ae14bafb96167d6b480d86c885feL5 https://github.com/mrdoob/three.js/pull/9310/files#diff-2ac7... becomes https://github.com/mrdoob/three.js/pull/9310/files#diff-ff6e5f22a9c7e66987b19c0199636480R4 https://github.com/mrdoob/three.js/pull/9310/files#diff-ff6e... On the surface it seems bad.

espadrine 10y ago
This does not look like an eval. It seems to be a script that generates a JS file. The string is not eval'ed, it is concatenated.
- superswordfish 10y ago
  Good point, either way it is ugly that normal code is now a string. It is in rollup.config.js so maybe this is imposed by "Rollup", but I'm not about to go find how the "outro" option differs from the "footer" option in yet another JS build tool.
rich_harris 10y ago
AudioContext is a special case – unlike most of the THREE.* exports, THREE.AudioContext is a getter (which initializes a Web Audio context). ES module namespaces can't have getters, so for backwards compatibility in the generated UMD bundle it was necessary to add that snippet to the bottom.

erikpukinskis 10y ago

Why is this a good thing? Just seems like different syntax to do the same thing. You can theoretically dynamically resolve the different exports asynhronously, but you could do that by exporting functions before. ES6 makes me think I am going crazy. Everyone seems so excited to have new sugar and a ton of busywork upgrading code and resolving runtime incompatibilities. I have yet to see any feature that seems like an actual benefit to the conservative working programmer. Maybe destructuring.

lhnz 10y ago
Smaller file sizes.
bsou 10y ago
eslint also provides static analysis for dependency resolution when using import statements instead of node require calls
- jacobr 10y ago
  eslint-plugin-import found several bugs for us when we enabled it. Can't recommend it enough. https://github.com/benmosher/eslint-plugin-import https://github.com/benmosher/eslint-plugin-import
smitherfield 10y ago
I'd say constants are pretty big.

gedy 10y ago

Awesome, virtually all of Rich Harris' work is pure gold - this, Rollup, RactiveJS, etc, etc. Pragmatic and solves real development problems.

typpytyper 10y ago
Don't forget his babel replacement, buble: https://gitlab.com/Rich-Harris/buble https://gitlab.com/Rich-Harris/buble Zero config, small and fast as heck.

stevebmark 10y ago

It's hard to see this as anything other than an ad for rollup. There's been many attempts to modernize the Three.js codebase. Not sure why this is the one that went through. I'm all for bringing Three.js up to date and applying modern JS tools to it. The maintainers of Three usually push back against these efforts. This PR still doesn't address the many fundamental problems with the quality of Three's source code and the attitudes of the maintainers. I wouldn't personally use Three on a large project, the low quality ecosystem will eventually harm you. It's designed as library to spin up small projects quickly. It's good at that. Beyond that I wouldn't recommend it. For the record I believe Babylon (another WebGL utility library) has similar problems with the source code. I don't have a suggestion for the "best" WebGL library to use.

TheRealPomax 10y ago
Is it, though? If "Hi, I'm the author of the ONLY tool that does bundling based on how ES2015 modules work, and applied that to your codebase, which made it both more useful to future developers and made it smaller as bundle" is an ad, I'd like to see more ads everywhere. Other than rollup there are no bundlers that perform ES2015 module bundling, which means there is no efficient tree shaking, no load-order independency, etc. I suspect the reason it went through is because it _actually_ modernized the codebase: unlike previous ideas, just swapping ES5 notation for a newer slightly more efficient ES6 notation, this refactor actually changes the load, parse, and bundle efficiency of the codebase, from a non-standard module style to what we've now collectively agreed on JS-native module should look like; something that before ES6 simply did not exist. There were choices made by individuals or organizations (commonjs, umd, etc), but only as of ES6 are modules _actually_ a formal construct in JS, with an official spec that formalizes how imports work. Any code changes that bring older codebases in line with the new spec are fantastic: this codebase just got massively future-proofed without any impact whatsoever on its integrity. (the one example that didn't run turned out to be a bug in the current example, not in the refactor)