5 ms·

"The trend started by minifiers like UglifyJS and continued by transpilers like Babel will only accelerate." Those were predated by Closure Compiler, which was

"The trend started by minifiers like UglifyJS and continued by transpilers like Babel will only accelerate." Those were predated by Closure Compiler, which was started with Gmail, and Closure Compiler in some ways still doesn't have an equal. Besides being a fully optimizing compiler, it also has a module system for code splitting, and optimization passes designed to move code at the method/property level from initially loaded code into deferred loaded code if it is detected to only be used later. You can see this in action with photos.google.com which splits up every UI component, model, and controller into a dependency tree of modules, which are aggressively optimized. When an action needs to fire, only the transitive dependency of reachable code needed to handle it is loaded. The closest external version of this is Malte Ubl's"Splittables" for Closure Compiler (https://medium.com/@cramforce/introducing-splittable-1c882babca7e https://medium.com/@cramforce/introducing-splittable-1c882ba...)

kolme 9y ago
Those features are supported in Webpack and other bundlers. Code splitting: https://webpack.js.org/guides/code-splitting/ https://webpack.js.org/guides/code-splitting/ Tree shaking: https://webpack.js.org/guides/tree-shaking/ https://webpack.js.org/guides/tree-shaking/
- cromwellian 9y ago
  It's not quite the same. It has some aspects of Google's module system in terms of de-duping (our internal version automatically splits up all code referenced in more than one place into a tree of synthetic modules) But Closure's optimizations go far beyond the "tree shaking" described here, which is really about pruning unused module dependencies. Closure does pruning of methods, properties, etc at a fine grained level, moves code at a per-method level between modules, it computes whether functions have side effects and elides unused side-effect-free code, it does type-based optimizations (https://github.com/google/closure-compiler/wiki/Type-Based-Property-Renaming https://github.com/google/closure-compiler/wiki/Type-Based-P...), the list goes on and on. It's really like having GCC or LLVM for Javascript code. That's one reason why it's slow, because it does far more than the transpilers and minimizers out there. (As many in another, GWT does many of the same things, being a fully optimizing true compiler)
  - lennelpennel 9y ago
    I want to add that you can easily write your own compile pass for the closure compiler as well. All the optimisations described above are just compile passes.
- coldtea 9y ago
  Only in a very primitive way.
- pas 9y ago
  https://github.com/webpack/webpack/issues/2867 https://github.com/webpack/webpack/issues/2867 - "Tree shaking completely broken?" It turns out that it basically supported "skip loading of unnecessary ES6 modules", so kind of like a "warn: unused import", but turned into a silence and no-op. Now with Uglify getting the "pure" annotation, and libraries sprinkling that onto their code, it'll help a bit.
- fnordsensei 9y ago
  React moved some parts from Uglify to Closure recently. They're only compiling using "simple" ("none" and "advanced" are the other options), but they're still seeing benefits: https://github.com/facebook/react/pull/10236 https://github.com/facebook/react/pull/10236
bananicorn 9y ago
The Closure Compiler is the only reason I have Java installed, it's that good. And IMHO, a lot of the bloated JS projects nowadays would really profit from its dead code removal...
- JBiserkov 9y ago
  In case you haven't heard, you can now use it without Java https://developers.googleblog.com/2016/08/closure-compiler-in-javascript.html https://developers.googleblog.com/2016/08/closure-compiler-i...
  - bananicorn 9y ago
    That's actually really amazing - and I didn't know that, thank you!
    - kevmo314 9y ago
      It's extremely slow compared to the Java version though.
      - paulddraper 9y ago
        I think it's about 2x slower.
        leeoniya 9y ago
        they're the same in name only. but not in perf and not in output. https://github.com/Rich-Harris/butternut/issues/35 https://github.com/Rich-Harris/butternut/issues/35
        bpicolo 9y ago
        I don't see where in that issue it says output changes from java to js? Other than the runtime option
hbbio 9y ago
This was also predated by the Opa "framework" which is indeed a compiler for the web platform, announced in 2009 and released a bit later. At the time, people didn't want to hear about a new language so we had to hide behind a framework. But one of the ideas was that there is no sense in bundling libraries at runtime instead of generating exactly the application code that should run. Cf. http://opalang.org http://opalang.org (disclaimer: I was the project creator)
- cromwellian 9y ago
  Yup, Opa is pretty nice. It just highlights the differences between "compilers" and "transpilers" which came up in another thread recently. There's a number of transpilers that are fairly simplistic transforms of source languages, or JS->JS translation (lowering ES7/6 to ES5, etc) which do limited whole world optimization, and mostly consist of peephole passes. Closure, Opa, GWT, Dart, and a few others (ScalaJS?) run fully optimizing compiler passes like a traditional old school compiler. For example, IIRC, Dart transforms (or used to) to Hydrogen IR/SSA internally, runs lots of optimizations, and then transforms back before JS output. This is not to bash transpilers, but I don't consider Webpack/Babel/Uglify in the same category.
- sambe 9y ago
  I was just thinking about Opa the other day - it looked nice but I never used it. People were worried about debugging through the layers of abstraction. Is it still "alive"? You write in the past tense...
  - hbbio 9y ago
    It is not really alive as of now. But there is still space for a full stack language and Opa is relevant today. MLstate, the startup behind Opa, shifted to secure communication platforms (built with Opa) and was acquired last year but without Opa itself. Many things we did became hype later (implemented in OCaml, a functional language, JSX before React - I know Jordan played with Opa before building React) and we still have bits that are missing is today's stacks. The ideal next step would be to join a foundation so that development on the project could resume.
gwt4life 9y ago
In the beginning there was GWT. But along with Java, GWT has a steep learning curve so its appeal is limited.
JBiserkov 9y ago
The Google Closure compiler and library are used as targets by the ClojureScript compiler https://clojurescript.org/about/closure https://clojurescript.org/about/closure
paulddraper 9y ago
Closure Compiler is very, very good at producing small code. It also takes a fair bit of manual effort unless it's handled for you (e.g. ClojureScript, Scala.js). Output modules in particular demand some very particular attention.
curun1r 9y ago
I remember the first publicly available being Dojo's minifier which used Mozilla's Rhino to determine which symbols could be safely obfuscated. It's primitive by today's standards, but it was a huge step forward at the time.
tootie 9y ago
Douglas Crockford is the guy who pushed JavaScript into this world. He wrote the first jslint which put JavaScript in the build pipeline on it's own. Also the YUI compressor and minifier are old enough to be deprecated.
kylecordes 9y ago
This seems to be something of a trend in the software library and tool world: A problem emerges. Some early, sharp thinkers create tools which attack the problem in a high level, abstract, powerful way... but also with the rough edges that come with being early. These tools get a few adopters, but not that many because it is difficult to see the value versus the obvious rough edges. Then other sharp thinkers, put off by those rough edges, create N new waves of tools which attack the problem less effectively, but more accessibly. Then eventually, some years later, common practice of the waves of tools will circle back around and start doing the things that that early rave started. It's easy to criticize the "daily new JavaScript framework" phenomenon along these lines, but I don't think it is specific to JavaScript, nor specific to 2017, and not even specific to software development. It's just part of how we humans work.
- pzs 9y ago
  Reading your comment reminded me of a classic piece [1] about a similar phenomenon. [1] http://discuss.joelonsoftware.com/default.asp?joel.3.219431.12 http://discuss.joelonsoftware.com/default.asp?joel.3.219431....
- nostrademons 9y ago
  Somewhat relatedly - when a new technology wave gets started, it's usually proprietary, because there are big competitive advantages that can be had by being the only one with possession of a new technology. As details leak out and enterprising independent hackers get interested, commodity or open-source clones emerge, and they get much wider distribution simply because they're cheap or free. The original innovator is caught flat-footed, loses mindshare, and eventually has to adopt the open-source solution when it becomes dominant in the open-source world. Altair -> Apple -> IBM -> Dell & clones. UNIX -> System V -> BSD -> Linux. MapReduce -> Hadoop. GMail -> Closure Compiler -> Traceur -> Babel. Google datacenters -> AWS -> Docker -> Kubernetes.
  - bsimpson 9y ago
    This is a very strange example. Gmail, Closure, and Traceur are all Google products.
dunham 9y ago
The compiler for GWT has done a lot of this for years (and it's functionality might have fed into closure). It's arguably different because it compiles from Java to Javascript as a first pass. It then does a lot of full-program optimization, including dead code elimination, lifting virtual method calls to static, inlining, reordering code to compress better, and analyzing the call graph to split code. (You have to be explicit about the split points in the Java code, so the analysis is mainly for doing the code splitting and for coalescing components that are are small or would force a lot of code into the common segment.) I believe GWT is moving on to closure for the latter optimization passes in future releases.

kolme 9y ago

Those features are supported in Webpack and other bundlers. Code splitting: https://webpack.js.org/guides/code-splitting/ https://webpack.js.org/guides/code-splitting/ Tree shaking: https://webpack.js.org/guides/tree-shaking/ https://webpack.js.org/guides/tree-shaking/

cromwellian 9y ago
It's not quite the same. It has some aspects of Google's module system in terms of de-duping (our internal version automatically splits up all code referenced in more than one place into a tree of synthetic modules) But Closure's optimizations go far beyond the "tree shaking" described here, which is really about pruning unused module dependencies. Closure does pruning of methods, properties, etc at a fine grained level, moves code at a per-method level between modules, it computes whether functions have side effects and elides unused side-effect-free code, it does type-based optimizations (https://github.com/google/closure-compiler/wiki/Type-Based-Property-Renaming https://github.com/google/closure-compiler/wiki/Type-Based-P...), the list goes on and on. It's really like having GCC or LLVM for Javascript code. That's one reason why it's slow, because it does far more than the transpilers and minimizers out there. (As many in another, GWT does many of the same things, being a fully optimizing true compiler)
- lennelpennel 9y ago
  I want to add that you can easily write your own compile pass for the closure compiler as well. All the optimisations described above are just compile passes.
coldtea 9y ago
Only in a very primitive way.
pas 9y ago
https://github.com/webpack/webpack/issues/2867 https://github.com/webpack/webpack/issues/2867 - "Tree shaking completely broken?" It turns out that it basically supported "skip loading of unnecessary ES6 modules", so kind of like a "warn: unused import", but turned into a silence and no-op. Now with Uglify getting the "pure" annotation, and libraries sprinkling that onto their code, it'll help a bit.
fnordsensei 9y ago
React moved some parts from Uglify to Closure recently. They're only compiling using "simple" ("none" and "advanced" are the other options), but they're still seeing benefits: https://github.com/facebook/react/pull/10236 https://github.com/facebook/react/pull/10236

bananicorn 9y ago

The Closure Compiler is the only reason I have Java installed, it's that good. And IMHO, a lot of the bloated JS projects nowadays would really profit from its dead code removal...

JBiserkov 9y ago
In case you haven't heard, you can now use it without Java https://developers.googleblog.com/2016/08/closure-compiler-in-javascript.html https://developers.googleblog.com/2016/08/closure-compiler-i...
- bananicorn 9y ago
  That's actually really amazing - and I didn't know that, thank you!
  - kevmo314 9y ago
    It's extremely slow compared to the Java version though.
    - paulddraper 9y ago
      I think it's about 2x slower.
      - leeoniya 9y ago
        they're the same in name only. but not in perf and not in output. https://github.com/Rich-Harris/butternut/issues/35 https://github.com/Rich-Harris/butternut/issues/35
        bpicolo 9y ago
        I don't see where in that issue it says output changes from java to js? Other than the runtime option

hbbio 9y ago

This was also predated by the Opa "framework" which is indeed a compiler for the web platform, announced in 2009 and released a bit later. At the time, people didn't want to hear about a new language so we had to hide behind a framework. But one of the ideas was that there is no sense in bundling libraries at runtime instead of generating exactly the application code that should run. Cf. http://opalang.org http://opalang.org (disclaimer: I was the project creator)

cromwellian 9y ago
Yup, Opa is pretty nice. It just highlights the differences between "compilers" and "transpilers" which came up in another thread recently. There's a number of transpilers that are fairly simplistic transforms of source languages, or JS->JS translation (lowering ES7/6 to ES5, etc) which do limited whole world optimization, and mostly consist of peephole passes. Closure, Opa, GWT, Dart, and a few others (ScalaJS?) run fully optimizing compiler passes like a traditional old school compiler. For example, IIRC, Dart transforms (or used to) to Hydrogen IR/SSA internally, runs lots of optimizations, and then transforms back before JS output. This is not to bash transpilers, but I don't consider Webpack/Babel/Uglify in the same category.
sambe 9y ago
I was just thinking about Opa the other day - it looked nice but I never used it. People were worried about debugging through the layers of abstraction. Is it still "alive"? You write in the past tense...
- hbbio 9y ago
  It is not really alive as of now. But there is still space for a full stack language and Opa is relevant today. MLstate, the startup behind Opa, shifted to secure communication platforms (built with Opa) and was acquired last year but without Opa itself. Many things we did became hype later (implemented in OCaml, a functional language, JSX before React - I know Jordan played with Opa before building React) and we still have bits that are missing is today's stacks. The ideal next step would be to join a foundation so that development on the project could resume.

gwt4life 9y ago

In the beginning there was GWT. But along with Java, GWT has a steep learning curve so its appeal is limited.

JBiserkov 9y ago

The Google Closure compiler and library are used as targets by the ClojureScript compiler https://clojurescript.org/about/closure https://clojurescript.org/about/closure

paulddraper 9y ago

Closure Compiler is very, very good at producing small code. It also takes a fair bit of manual effort unless it's handled for you (e.g. ClojureScript, Scala.js). Output modules in particular demand some very particular attention.

curun1r 9y ago

I remember the first publicly available being Dojo's minifier which used Mozilla's Rhino to determine which symbols could be safely obfuscated. It's primitive by today's standards, but it was a huge step forward at the time.

tootie 9y ago

Douglas Crockford is the guy who pushed JavaScript into this world. He wrote the first jslint which put JavaScript in the build pipeline on it's own. Also the YUI compressor and minifier are old enough to be deprecated.

kylecordes 9y ago

This seems to be something of a trend in the software library and tool world: A problem emerges. Some early, sharp thinkers create tools which attack the problem in a high level, abstract, powerful way... but also with the rough edges that come with being early. These tools get a few adopters, but not that many because it is difficult to see the value versus the obvious rough edges. Then other sharp thinkers, put off by those rough edges, create N new waves of tools which attack the problem less effectively, but more accessibly. Then eventually, some years later, common practice of the waves of tools will circle back around and start doing the things that that early rave started. It's easy to criticize the "daily new JavaScript framework" phenomenon along these lines, but I don't think it is specific to JavaScript, nor specific to 2017, and not even specific to software development. It's just part of how we humans work.

pzs 9y ago
Reading your comment reminded me of a classic piece [1] about a similar phenomenon. [1] http://discuss.joelonsoftware.com/default.asp?joel.3.219431.12 http://discuss.joelonsoftware.com/default.asp?joel.3.219431....
nostrademons 9y ago
Somewhat relatedly - when a new technology wave gets started, it's usually proprietary, because there are big competitive advantages that can be had by being the only one with possession of a new technology. As details leak out and enterprising independent hackers get interested, commodity or open-source clones emerge, and they get much wider distribution simply because they're cheap or free. The original innovator is caught flat-footed, loses mindshare, and eventually has to adopt the open-source solution when it becomes dominant in the open-source world. Altair -> Apple -> IBM -> Dell & clones. UNIX -> System V -> BSD -> Linux. MapReduce -> Hadoop. GMail -> Closure Compiler -> Traceur -> Babel. Google datacenters -> AWS -> Docker -> Kubernetes.
- bsimpson 9y ago
  This is a very strange example. Gmail, Closure, and Traceur are all Google products.

dunham 9y ago

The compiler for GWT has done a lot of this for years (and it's functionality might have fed into closure). It's arguably different because it compiles from Java to Javascript as a first pass. It then does a lot of full-program optimization, including dead code elimination, lifting virtual method calls to static, inlining, reordering code to compress better, and analyzing the call graph to split code. (You have to be explicit about the split points in the Java code, so the analysis is mainly for doing the code splitting and for coalescing components that are are small or would force a lot of code into the common segment.) I believe GWT is moving on to closure for the latter optimization passes in future releases.