8 ms·

JavaScript Modularity Shaming

140 points by fogus 12y ago

serve_yay 12y ago
This is a strange post, it's about two completely separate things. I do agree with the "modularity shaming" idea though, the JS community does seem to be overzealous about that. However I think it better to err in that direction than the other direction.
amelius 12y ago
Why don't we just let browsers treat javascript files as modules, just like we have shared libraries on our OS? This way, libraries such as JQuery, could be cached, and shared between websites.
- lomnakkus 12y ago
  You can sort-of already do that with CDNs. Browser still has to parse+run all of that JS though. Because JS-in-browsers is antimodular in that all previous JS on a given page influences all subsequently loaded JS, even if using just script tags with a URL. In a way this is very similar to the pains experienced by the C++ community for many years now.
  - amelius 12y ago
    Could you elaborate on the C++ part?
    - lomnakkus 12y ago
      The short answer is that "it's complicated". The most accessible long answer I'm aware of for non-C++'ers is at [1] (video). If you prefer text, the standardese version is at [2]. Unfortunately I cannot find a concise text version for laymen except perhaps [3]. [1]: https://www.youtube.com/watch?v=4Xo9iH5VLQ0 https://www.youtube.com/watch?v=4Xo9iH5VLQ0 [2]: http://www.open-std.org/Jtc1/sc22/wg21/docs/papers/2014/n4047.pdf http://www.open-std.org/Jtc1/sc22/wg21/docs/papers/2014/n404... [3]: http://clang.llvm.org/docs/Modules.html http://clang.llvm.org/docs/Modules.html
    - DiThi 12y ago
      C++ doesn't have a "load this module" statement. I does have an "include all the contents of X file here", and file X has to deal with possible duplication and cyclic dependencies, usually with something like #ifndef _SOME_FILE_H #define _SOME_FILE_H // code goes here #endif Also any "module" can define any global variable, so name clashes (esp. with C libraries) are possible. Most languages with proper modules the "global" variables are module-wide. An equally nasty issue happens with JS, where many libraries must be loaded in the proper order and not duplicated. There are a bunch of module systems but none is part of the spec. The good thing is that module systems are easy to make thanks to closures.
  - jffry 12y ago
    FYI you can add an async attribute [1] to a <script> tag to allow it to load and run asynchronously without blocking the rest of the page [1]: http://caniuse.com/#feat=script-async http://caniuse.com/#feat=script-async
    - lomnakkus 12y ago
      Good point. Should've mentioned that. Unfortunately, it's often hard to know if "async" or "defer" will work reliably.
- smrtinsert 12y ago
  This is a dead end in a sufficiently large and complicated app (think beyond facebook/gmail etc) especially if the app supported multiple versions of 3rd party code such as charting libs. besides writing it yourself (cue contractors going this is the only viable route) reuse and dead code elimination are necessary foundations to the next level of web apps for productivity of dev and responsive runtimes.
- j_baker 12y ago
  That's basically what ES6 will do: http://www.2ality.com/2014/09/es6-modules-final.html http://www.2ality.com/2014/09/es6-modules-final.html
imslavko 12y ago
Same people who shame other's work for not being modular enough for their taste, would worship a one-line module[0] that is modular down to every subroutine. [0]: https://github.com/blakeembrey/is-upper-case/blob/master/is-upper-case.js https://github.com/blakeembrey/is-upper-case/blob/master/is-...
- cmpb 12y ago
  That's intense and really quite silly. It makes sense to modularize large or complex (or some better modifier?) portions of the code base, but there's a point at which it just becomes too deep of a rabbit hole. I don't believe there is a formal definition of how deep down the rabbit hole that mark is, but I'm sure it would be a function of the module's (and the project's) size, readability and complexity.
  - mkal_tsr 12y ago
    When the rabbits become turtles, you've gone too deep.
  - vog 12y ago
    Lewis and Fowler provide a nice definition in their Microservices article. [1] They drive modularity through the pattern of change, so their judgement is based on the development, rather than the code itself. "You want to keep things that change at the same time in the same module. Parts of a system that change rarely should be in different services to those that are currently undergoing lots of churn. If you find yourself repeatedly changing two services together, that's a sign that they should be merged." Although that article is about services, believe it can be applied to any type of modules. [1] http://martinfowler.com/articles/microservices.html http://martinfowler.com/articles/microservices.html
- geetee 12y ago
  I thought this was a joke library. Then I checked the NPM stats. 6,597 downloads in the last day 28,883 downloads in the last week 134,084 downloads in the last month
ianbicking 12y ago
Is the Closure Compiler specifically good at dead code elimination in Closure libraries? Theoretically I would imagine that it could be applied to anything, though I'm sure there are many Javascript tricks that would make dead code detection ineffective.
- swannodette 12y ago
  Closure Compiler requires authors to write code in a very specific static style. It's somewhat tedious, but for some libs worth the effort. In the case of ClojureScript we can of course automate the tedium for you through the compiler :) For random JS libs Closure can't really do much better than Uglify.
  - craigching 12y ago
    > For random JS libs Closure can't really do much better than Uglify. I wish I could use ClojureScript in my day-to-day, but for now it's out of the question. So, for us, AMD [1] has been a god send. We can defer loading other major parts of our UI until the user actually uses that functionality. Doing this on our own would be impossible, but using AMD has really worked well for us. But, here's to hoping I can actually use ClojureScript at some future point :) [1] -- http://requirejs.org/docs/whyamd.html http://requirejs.org/docs/whyamd.html
  - nostrademons 12y ago
    Yeah, I tried for a couple years to get Google Search to adopt JQuery, but JQuery without dead-code elimination would've doubled the SRP latency, and JQuery with dead-code elimination doesn't actually eliminate any dead code. The problem is that it's written in a style that dynamically assigns methods to the JQuery prototype, and so it's not possible to analyze which code actually exists in the library without executing the whole of the library.
    - dmethvin 12y ago
      You could build a custom jQuery copy that eliminated whatever methods you don't need. Since plugins use unknown parts of jQuery and Closure Compiler can't always detect that, you really need to do some manual labor to pull in what you need. In the 1.8 time frame the jQuery devs made a call for people who wanted to use Closure Compiler's ADVANCED_OPTIMIZATIONS to participate, but just didn't get any community interest. CCAO style is not typical JavaScript and it doesn't seem that a lot of people use it.
juandopazo 12y ago
This is an area where the new syntax for modules in EcmaScript 6 does really well. Since exports are static, a dead code elimination tool can figure out which exports from a module are being used and remove the ones that are not being used. As mentioned in comments about how the Closure Compiler works, you can't really do this with purely dynamic code, but you can do it with static imports/exports.
skybrian 12y ago
Of course GWT and Dart also do dead code elimination (also known as tree-shaking). There's a reason Google's JavaScript-targeted compilers implement this. On the other hand, the consistent dedication to writing tiny libraries in JavaScript ecosystem is not a bug. Tree-shaking compilers make it somewhat harder to tell where the bloat is coming from, and that encourages people writing libraries to get sloppy.
- swannodette 12y ago
  My experience so far is that writing code that defeats tree shaking is somewhat more difficult when adopting the simple functional style of the Closure Library base libs. Fortunately for ClojureScript, this style is already idiomatic.
nwienert 12y ago
Relevant discussion thread on github[1]. Max Ogden's last comment irks me. After a long and fairly productive discussion, he generalizes / calls out people, and doesn't explicate on what he's trying to point out. [1] https://gist.github.com/substack/68f8d502be42d5cd4942 https://gist.github.com/substack/68f8d502be42d5cd4942
- sehr 12y ago
  It's not exactly out of character for 'high profile' node community members is it. I love the work they're doing, and the community they've fostered might be one of the most progressive in the industry, but they can be insanely pretentious at times. (irony of responding to a comment about generalization noted)
thomasreggi 12y ago
I had a twitter convo with David DeSandro about this exact thing today https://twitter.com/thomasreggi/status/552903079142883329 https://twitter.com/thomasreggi/status/552903079142883329. I've really taken to projects like https://github.com/tjmehta/101 https://github.com/tjmehta/101 where you are forced (there's an error if you include the index) to include the files / functions that you need directly. He raised some good points.
city41 12y ago
I'm a fan of ClojureScript and one reason being how easy it makes it to take advantage of the Closure compiler. But it can be a double edged sword. At times the advanced compilation mode breaks your code, and it can be extremely difficult to track down why. The combination of symbol renaming and dead code elimination usually means final JavaScript that looks utterly nothing like the ClojureScript you started out with. Just today I had an issue where the Closure compiler decided my call to (set!) wasn't necessary, and so it removed it, completely breaking my app. The best solution I could find was a work around involving a pretty large let block. I was just happy to get it working, debugging optimized Closure compiled code is not fun at all.
- swannodette 12y ago
  Debugging advanced compiled Closure in ClojureScript is actually pretty straightforward if you know which knobs to turn - :pseudo-names true, :pretty-print true build options are pretty much all you need. I'm somewhat skeptical about a set! being eliminated unless you were setting a property of an object from a random JavaScript library you haven't provided an extern for. The above compiler settings would have showed this immediately.
  - city41 12y ago
    I did not know about :pseudo-names, that looks to be quite helpful. Thanks. This is the method I was dealing with: https://github.com/city41/bookends/blob/master/site/src/cljs/demo/knex.cljs#L6 https://github.com/city41/bookends/blob/master/site/src/cljs... In that let expression I'm digging my way down into a native JS object to monkey patch it. The equivalent (set! (.. knex -client -Runner -prototype -debug) ...) was being completely removed as far as I can tell. Same with an equivalent aset. That channel no longer got set up and my app stopped working. If I simply put that method back to set!, then it breaks again, very reproducible. When I added in a (.log js/console "hello") to the method, I could see that console log inlined where the method call was, but no other remnant of the method could be found. I will play with :pretty-print and look into creating a small repro of what I'm seeing. But if I don't succeed with a smaller repro, then at the very least this app is already quite small. EDIT: here is a little gist showing the difference I'm seeing if anyone is curious: https://gist.github.com/city41/12099b91e0fbb526b0dc https://gist.github.com/city41/12099b91e0fbb526b0dc
mattdesl 12y ago
I responded a while back in another thread about "small modules" in npm. Filesize is only one (rather small) facet of why some people prefer this approach to the "put everything under the same hood". https://news.ycombinator.com/item?id=8830058 https://news.ycombinator.com/item?id=8830058
zubairq 12y ago
https://www.youtube.com/watch?v=dtnQ3Mwp0KQ https://www.youtube.com/watch?v=dtnQ3Mwp0KQ

140 points by fogus 12y ago

This is a strange post, it's about two completely separate things. I do agree with the "modularity shaming" idea though, the JS community does seem to be overzealous about that. However I think it better to err in that direction than the other direction.

amelius 12y ago

Why don't we just let browsers treat javascript files as modules, just like we have shared libraries on our OS? This way, libraries such as JQuery, could be cached, and shared between websites.

lomnakkus 12y ago
You can sort-of already do that with CDNs. Browser still has to parse+run all of that JS though. Because JS-in-browsers is antimodular in that all previous JS on a given page influences all subsequently loaded JS, even if using just script tags with a URL. In a way this is very similar to the pains experienced by the C++ community for many years now.
- amelius 12y ago
  Could you elaborate on the C++ part?
  - lomnakkus 12y ago
    The short answer is that "it's complicated". The most accessible long answer I'm aware of for non-C++'ers is at [1] (video). If you prefer text, the standardese version is at [2]. Unfortunately I cannot find a concise text version for laymen except perhaps [3]. [1]: https://www.youtube.com/watch?v=4Xo9iH5VLQ0 https://www.youtube.com/watch?v=4Xo9iH5VLQ0 [2]: http://www.open-std.org/Jtc1/sc22/wg21/docs/papers/2014/n4047.pdf http://www.open-std.org/Jtc1/sc22/wg21/docs/papers/2014/n404... [3]: http://clang.llvm.org/docs/Modules.html http://clang.llvm.org/docs/Modules.html
  - DiThi 12y ago
    C++ doesn't have a "load this module" statement. I does have an "include all the contents of X file here", and file X has to deal with possible duplication and cyclic dependencies, usually with something like #ifndef _SOME_FILE_H #define _SOME_FILE_H // code goes here #endif Also any "module" can define any global variable, so name clashes (esp. with C libraries) are possible. Most languages with proper modules the "global" variables are module-wide. An equally nasty issue happens with JS, where many libraries must be loaded in the proper order and not duplicated. There are a bunch of module systems but none is part of the spec. The good thing is that module systems are easy to make thanks to closures.
- jffry 12y ago
  FYI you can add an async attribute [1] to a <script> tag to allow it to load and run asynchronously without blocking the rest of the page [1]: http://caniuse.com/#feat=script-async http://caniuse.com/#feat=script-async
  - lomnakkus 12y ago
    Good point. Should've mentioned that. Unfortunately, it's often hard to know if "async" or "defer" will work reliably.
smrtinsert 12y ago
This is a dead end in a sufficiently large and complicated app (think beyond facebook/gmail etc) especially if the app supported multiple versions of 3rd party code such as charting libs. besides writing it yourself (cue contractors going this is the only viable route) reuse and dead code elimination are necessary foundations to the next level of web apps for productivity of dev and responsive runtimes.
j_baker 12y ago
That's basically what ES6 will do: http://www.2ality.com/2014/09/es6-modules-final.html http://www.2ality.com/2014/09/es6-modules-final.html

imslavko 12y ago

Same people who shame other's work for not being modular enough for their taste, would worship a one-line module[0] that is modular down to every subroutine. [0]: https://github.com/blakeembrey/is-upper-case/blob/master/is-upper-case.js https://github.com/blakeembrey/is-upper-case/blob/master/is-...

cmpb 12y ago
That's intense and really quite silly. It makes sense to modularize large or complex (or some better modifier?) portions of the code base, but there's a point at which it just becomes too deep of a rabbit hole. I don't believe there is a formal definition of how deep down the rabbit hole that mark is, but I'm sure it would be a function of the module's (and the project's) size, readability and complexity.
- mkal_tsr 12y ago
  When the rabbits become turtles, you've gone too deep.
- vog 12y ago
  Lewis and Fowler provide a nice definition in their Microservices article. [1] They drive modularity through the pattern of change, so their judgement is based on the development, rather than the code itself. "You want to keep things that change at the same time in the same module. Parts of a system that change rarely should be in different services to those that are currently undergoing lots of churn. If you find yourself repeatedly changing two services together, that's a sign that they should be merged." Although that article is about services, believe it can be applied to any type of modules. [1] http://martinfowler.com/articles/microservices.html http://martinfowler.com/articles/microservices.html
geetee 12y ago
I thought this was a joke library. Then I checked the NPM stats. 6,597 downloads in the last day 28,883 downloads in the last week 134,084 downloads in the last month

ianbicking 12y ago

Is the Closure Compiler specifically good at dead code elimination in Closure libraries? Theoretically I would imagine that it could be applied to anything, though I'm sure there are many Javascript tricks that would make dead code detection ineffective.

swannodette 12y ago
Closure Compiler requires authors to write code in a very specific static style. It's somewhat tedious, but for some libs worth the effort. In the case of ClojureScript we can of course automate the tedium for you through the compiler :) For random JS libs Closure can't really do much better than Uglify.
- craigching 12y ago
  > For random JS libs Closure can't really do much better than Uglify. I wish I could use ClojureScript in my day-to-day, but for now it's out of the question. So, for us, AMD [1] has been a god send. We can defer loading other major parts of our UI until the user actually uses that functionality. Doing this on our own would be impossible, but using AMD has really worked well for us. But, here's to hoping I can actually use ClojureScript at some future point :) [1] -- http://requirejs.org/docs/whyamd.html http://requirejs.org/docs/whyamd.html
- nostrademons 12y ago
  Yeah, I tried for a couple years to get Google Search to adopt JQuery, but JQuery without dead-code elimination would've doubled the SRP latency, and JQuery with dead-code elimination doesn't actually eliminate any dead code. The problem is that it's written in a style that dynamically assigns methods to the JQuery prototype, and so it's not possible to analyze which code actually exists in the library without executing the whole of the library.
  - dmethvin 12y ago
    You could build a custom jQuery copy that eliminated whatever methods you don't need. Since plugins use unknown parts of jQuery and Closure Compiler can't always detect that, you really need to do some manual labor to pull in what you need. In the 1.8 time frame the jQuery devs made a call for people who wanted to use Closure Compiler's ADVANCED_OPTIMIZATIONS to participate, but just didn't get any community interest. CCAO style is not typical JavaScript and it doesn't seem that a lot of people use it.

juandopazo 12y ago

This is an area where the new syntax for modules in EcmaScript 6 does really well. Since exports are static, a dead code elimination tool can figure out which exports from a module are being used and remove the ones that are not being used. As mentioned in comments about how the Closure Compiler works, you can't really do this with purely dynamic code, but you can do it with static imports/exports.

skybrian 12y ago

Of course GWT and Dart also do dead code elimination (also known as tree-shaking). There's a reason Google's JavaScript-targeted compilers implement this. On the other hand, the consistent dedication to writing tiny libraries in JavaScript ecosystem is not a bug. Tree-shaking compilers make it somewhat harder to tell where the bloat is coming from, and that encourages people writing libraries to get sloppy.

swannodette 12y ago
My experience so far is that writing code that defeats tree shaking is somewhat more difficult when adopting the simple functional style of the Closure Library base libs. Fortunately for ClojureScript, this style is already idiomatic.

nwienert 12y ago

Relevant discussion thread on github[1]. Max Ogden's last comment irks me. After a long and fairly productive discussion, he generalizes / calls out people, and doesn't explicate on what he's trying to point out. [1] https://gist.github.com/substack/68f8d502be42d5cd4942 https://gist.github.com/substack/68f8d502be42d5cd4942

sehr 12y ago
It's not exactly out of character for 'high profile' node community members is it. I love the work they're doing, and the community they've fostered might be one of the most progressive in the industry, but they can be insanely pretentious at times. (irony of responding to a comment about generalization noted)

thomasreggi 12y ago

I had a twitter convo with David DeSandro about this exact thing today https://twitter.com/thomasreggi/status/552903079142883329 https://twitter.com/thomasreggi/status/552903079142883329. I've really taken to projects like https://github.com/tjmehta/101 https://github.com/tjmehta/101 where you are forced (there's an error if you include the index) to include the files / functions that you need directly. He raised some good points.

city41 12y ago

I'm a fan of ClojureScript and one reason being how easy it makes it to take advantage of the Closure compiler. But it can be a double edged sword. At times the advanced compilation mode breaks your code, and it can be extremely difficult to track down why. The combination of symbol renaming and dead code elimination usually means final JavaScript that looks utterly nothing like the ClojureScript you started out with. Just today I had an issue where the Closure compiler decided my call to (set!) wasn't necessary, and so it removed it, completely breaking my app. The best solution I could find was a work around involving a pretty large let block. I was just happy to get it working, debugging optimized Closure compiled code is not fun at all.

swannodette 12y ago
Debugging advanced compiled Closure in ClojureScript is actually pretty straightforward if you know which knobs to turn - :pseudo-names true, :pretty-print true build options are pretty much all you need. I'm somewhat skeptical about a set! being eliminated unless you were setting a property of an object from a random JavaScript library you haven't provided an extern for. The above compiler settings would have showed this immediately.
- city41 12y ago
  I did not know about :pseudo-names, that looks to be quite helpful. Thanks. This is the method I was dealing with: https://github.com/city41/bookends/blob/master/site/src/cljs/demo/knex.cljs#L6 https://github.com/city41/bookends/blob/master/site/src/cljs... In that let expression I'm digging my way down into a native JS object to monkey patch it. The equivalent (set! (.. knex -client -Runner -prototype -debug) ...) was being completely removed as far as I can tell. Same with an equivalent aset. That channel no longer got set up and my app stopped working. If I simply put that method back to set!, then it breaks again, very reproducible. When I added in a (.log js/console "hello") to the method, I could see that console log inlined where the method call was, but no other remnant of the method could be found. I will play with :pretty-print and look into creating a small repro of what I'm seeing. But if I don't succeed with a smaller repro, then at the very least this app is already quite small. EDIT: here is a little gist showing the difference I'm seeing if anyone is curious: https://gist.github.com/city41/12099b91e0fbb526b0dc https://gist.github.com/city41/12099b91e0fbb526b0dc

mattdesl 12y ago

I responded a while back in another thread about "small modules" in npm. Filesize is only one (rather small) facet of why some people prefer this approach to the "put everything under the same hood". https://news.ycombinator.com/item?id=8830058 https://news.ycombinator.com/item?id=8830058

zubairq 12y ago

https://www.youtube.com/watch?v=dtnQ3Mwp0KQ https://www.youtube.com/watch?v=dtnQ3Mwp0KQ