6 ms·

CommonJS Is The Future For Rails JavaScript Packaging

75 points by maccman 15y ago

exogen 15y ago
"If you've ever use Node or Python you've used CommonJS modules" Huh? Python, really?
- joeyespo 15y ago
  I could be wrong, but I imagine he's referring to the module concept. Not CommonJS itself. (Or perhaps referring to skulpt.org, but I doubt that.)
  - maccman 15y ago
    Yes, that's correct. Importing is similar to require(), although there's no explicit exporting.
    - MBlume 15y ago
      setting module-level __all__ can modify what gets pulled in a wild import, but that's about it.
      - masklinn 15y ago
        It's also used by `help()` (and pydoc): if there's an __all__, help will only pull the docstrings of the stuff in __all__. `dir()`, on the other hand, does not care.
- JonnieCache 15y ago
  CommonJS modules are modelled on python's I think.
- snprbob86 15y ago
  With CoffeeScript's destructuring assignment, you can even simulate Python's "from" clause: foo = require 'foo' {Bar} = require 'bar' {X, f} = require 'abc' Compiles to: var Bar, X, f, foo, _ref; foo = require('foo'); Bar = require('bar').Bar; _ref = require('abc'), X = _ref.X, f = _ref.f;
bobfunk 15y ago
Looks very cool, will play around with it. Seems like a big missed opportunity that Sprockets doesn't speak CommonJS.
- shubber 15y ago
  Sprockets, as it exists, can't do module loading: it's a Javascript concatenator. All it does it copy the contents of <filename.js> in place of //= require 'filename.js'. Gotta, say, I just spent a chunk of time converting a project from one to the other (actually RequireJS), and I really think it was the right move.
bronson 15y ago
Didn't Ryan Dahl say he wish he used something else for packaging Node modules? (I don't know what he had in mind other than CommonJS and I don't remember where I heard it so this is pretty hazy...)
- telemachos 15y ago
  Yes, it's in this interview: http://bostinnovation.com/2011/01/31/node-js-interview-4-questions-with-creator-ryan-dahl/ http://bostinnovation.com/2011/01/31/node-js-interview-4-que... BostInno: Is there anything you wish you had done differently with node? Dahl: Yes – many things. For example, I wish I had not used the CommonJS module system. It’s far too complex and wildly different from how the browser works. In another interview (which I can't find), I recall him worrying a bit that CommonJS is not async, though admitting that a few lapses in async purity might be fine. In any case, I'm not aware that Node actually has any plans to move away from CommonJS. At this point, a whole hell of a lot of code relies on that style of require. Perhaps it could be swapped out in some backward compatible way, but it seems like a lot of work. (Also from the point of view of someone who just uses CommonJS style rather than implements it in something like Node, I have to say, it's dead easy and intuitive. Why fix what isn't broken?) Wow, wtf? I got voted down for finding someone a reference? I'm not making an anti- (or pro-) CommonJS argument here. I just found the reference.
  - maccman 15y ago
    This may go some ways to solving the issue (mostly with asynchronous module loading) - https://github.com/joyent/node/commit/9967c369c9272335bb0343558673b689725c6d7c https://github.com/joyent/node/commit/9967c369c9272335bb0343... Like you say, I don't see any issues with the current system (and don't like wrapping everything manually in define function). You'll always need to get a server involved at some point for concatenation and performance reasons - and I think the current system works fine.
    - telemachos 15y ago
      Ah that makes sense. (As I say, I remember that he said elsewhere he wasn't thrilled about requires being synchronous.)
shubber 15y ago
If only it were true. I started in on Sprockets to get my feet wet for Rails 3.1, and have to say I was disappointed. Sprockets is a textual concatenator, which makes it appropriate when you're writing short snippets throughout a project. By contrast, CommonJS (and requireJS and a slew of other implementations of asynchronous modules) implements well scoped modules, which have all kinds of boons.
sstephenson 15y ago
As the creator of both Sprockets, which underlies Rails 3.1's asset pipeline, and Stitch, which the poster of this article ported beautifully from Node.js to Ruby, I feel obligated to reply here :) While CommonJS modules are undeniably elegant, they carry with them a high conceptual overhead. It would be unreasonable to expect Rails developers to rewrite all their existing JavaScript in order to take advantage of the asset pipeline. Most people do not want to manually import every dependency into every source file (and in fact Rails goes to great lengths to make use of Ruby's "require" method largely unnecessary). Sprockets solves this with the "require_tree" directive that automatically pulls in all source files in a given directory. If you've been keeping your JavaScript source files in public/javascripts, it's trivial to upgrade your app to 3.1 and take advantage of the asset pipeline — just move your source files into app/assets/javascripts. Then when you do need more specific control over source file ordering, the "require" and "include" directives are there for you. Combine this with the implicit closure around each CoffeeScript source file and Sprockets gives you 80% of the benefit of CommonJS modules with 20% of the complexity. Stitch (and CommonJS in general) does not address the problem of CSS preprocessing or concatenation, and does not provide a mechanism for managing other assets like images and sounds. Sprockets supports Sass and SCSS, and provides a global load path which lets you depend on JS, CSS and images from a single logical package — even a Ruby gem — with essentially no configuration. I hope this clarifies why we chose to go with the Sprockets approach rather than CommonJS for Rails 3.1.
- rasmusrn 15y ago
  Sprockets can be used for JS, CSS, images, and so on, because it is so simple: it is "just" concatenation. I understand and agree on the benefit/complexity argument. But the fact that this method works universally, is also the reason it cannot accomodate individual needs for each "asset type" (css, js): * For Javascript we cannot use CommonJS (or something similar) * For SCSS we cannot use variables, mixins For me, this means a lot. Traditionally, Rails has been giving me the best web environment experience I could imagine. With Rails 3.1, I can't help thinking: Why did they leave some of the best parts out? I know I actually CAN use SCSS variables and mixins - I just have to use SCSS' @import rather than Sprockets own =require. But that leads to my stance in this matter: If we need to bypass Sprockets in order to get the cool features, then why use it in the first place? We should use a system designed specifically for JS to management our JS, and a system specifically for CSS to manage CSS. Stitch and SASS comes to mind of course. While I understand the benefits of the "Sprockets way", I'd like to see Rails use specific tools for specific asset types in order to give developers easy access to all of the available features instead of just 80% of them.

75 points by maccman 15y ago

exogen 15y ago

"If you've ever use Node or Python you've used CommonJS modules" Huh? Python, really?

joeyespo 15y ago
I could be wrong, but I imagine he's referring to the module concept. Not CommonJS itself. (Or perhaps referring to skulpt.org, but I doubt that.)
- maccman 15y ago
  Yes, that's correct. Importing is similar to require(), although there's no explicit exporting.
  - MBlume 15y ago
    setting module-level __all__ can modify what gets pulled in a wild import, but that's about it.
    - masklinn 15y ago
      It's also used by `help()` (and pydoc): if there's an __all__, help will only pull the docstrings of the stuff in __all__. `dir()`, on the other hand, does not care.
JonnieCache 15y ago
CommonJS modules are modelled on python's I think.
snprbob86 15y ago
With CoffeeScript's destructuring assignment, you can even simulate Python's "from" clause: foo = require 'foo' {Bar} = require 'bar' {X, f} = require 'abc' Compiles to: var Bar, X, f, foo, _ref; foo = require('foo'); Bar = require('bar').Bar; _ref = require('abc'), X = _ref.X, f = _ref.f;

bobfunk 15y ago

Looks very cool, will play around with it. Seems like a big missed opportunity that Sprockets doesn't speak CommonJS.

shubber 15y ago
Sprockets, as it exists, can't do module loading: it's a Javascript concatenator. All it does it copy the contents of <filename.js> in place of //= require 'filename.js'. Gotta, say, I just spent a chunk of time converting a project from one to the other (actually RequireJS), and I really think it was the right move.

bronson 15y ago

Didn't Ryan Dahl say he wish he used something else for packaging Node modules? (I don't know what he had in mind other than CommonJS and I don't remember where I heard it so this is pretty hazy...)

telemachos 15y ago
Yes, it's in this interview: http://bostinnovation.com/2011/01/31/node-js-interview-4-questions-with-creator-ryan-dahl/ http://bostinnovation.com/2011/01/31/node-js-interview-4-que... BostInno: Is there anything you wish you had done differently with node? Dahl: Yes – many things. For example, I wish I had not used the CommonJS module system. It’s far too complex and wildly different from how the browser works. In another interview (which I can't find), I recall him worrying a bit that CommonJS is not async, though admitting that a few lapses in async purity might be fine. In any case, I'm not aware that Node actually has any plans to move away from CommonJS. At this point, a whole hell of a lot of code relies on that style of require. Perhaps it could be swapped out in some backward compatible way, but it seems like a lot of work. (Also from the point of view of someone who just uses CommonJS style rather than implements it in something like Node, I have to say, it's dead easy and intuitive. Why fix what isn't broken?) Wow, wtf? I got voted down for finding someone a reference? I'm not making an anti- (or pro-) CommonJS argument here. I just found the reference.
- maccman 15y ago
  This may go some ways to solving the issue (mostly with asynchronous module loading) - https://github.com/joyent/node/commit/9967c369c9272335bb0343558673b689725c6d7c https://github.com/joyent/node/commit/9967c369c9272335bb0343... Like you say, I don't see any issues with the current system (and don't like wrapping everything manually in define function). You'll always need to get a server involved at some point for concatenation and performance reasons - and I think the current system works fine.
  - telemachos 15y ago
    Ah that makes sense. (As I say, I remember that he said elsewhere he wasn't thrilled about requires being synchronous.)

shubber 15y ago

If only it were true. I started in on Sprockets to get my feet wet for Rails 3.1, and have to say I was disappointed. Sprockets is a textual concatenator, which makes it appropriate when you're writing short snippets throughout a project. By contrast, CommonJS (and requireJS and a slew of other implementations of asynchronous modules) implements well scoped modules, which have all kinds of boons.

sstephenson 15y ago

As the creator of both Sprockets, which underlies Rails 3.1's asset pipeline, and Stitch, which the poster of this article ported beautifully from Node.js to Ruby, I feel obligated to reply here :) While CommonJS modules are undeniably elegant, they carry with them a high conceptual overhead. It would be unreasonable to expect Rails developers to rewrite all their existing JavaScript in order to take advantage of the asset pipeline. Most people do not want to manually import every dependency into every source file (and in fact Rails goes to great lengths to make use of Ruby's "require" method largely unnecessary). Sprockets solves this with the "require_tree" directive that automatically pulls in all source files in a given directory. If you've been keeping your JavaScript source files in public/javascripts, it's trivial to upgrade your app to 3.1 and take advantage of the asset pipeline — just move your source files into app/assets/javascripts. Then when you do need more specific control over source file ordering, the "require" and "include" directives are there for you. Combine this with the implicit closure around each CoffeeScript source file and Sprockets gives you 80% of the benefit of CommonJS modules with 20% of the complexity. Stitch (and CommonJS in general) does not address the problem of CSS preprocessing or concatenation, and does not provide a mechanism for managing other assets like images and sounds. Sprockets supports Sass and SCSS, and provides a global load path which lets you depend on JS, CSS and images from a single logical package — even a Ruby gem — with essentially no configuration. I hope this clarifies why we chose to go with the Sprockets approach rather than CommonJS for Rails 3.1.

rasmusrn 15y ago
Sprockets can be used for JS, CSS, images, and so on, because it is so simple: it is "just" concatenation. I understand and agree on the benefit/complexity argument. But the fact that this method works universally, is also the reason it cannot accomodate individual needs for each "asset type" (css, js): * For Javascript we cannot use CommonJS (or something similar) * For SCSS we cannot use variables, mixins For me, this means a lot. Traditionally, Rails has been giving me the best web environment experience I could imagine. With Rails 3.1, I can't help thinking: Why did they leave some of the best parts out? I know I actually CAN use SCSS variables and mixins - I just have to use SCSS' @import rather than Sprockets own =require. But that leads to my stance in this matter: If we need to bypass Sprockets in order to get the cool features, then why use it in the first place? We should use a system designed specifically for JS to management our JS, and a system specifically for CSS to manage CSS. Stitch and SASS comes to mind of course. While I understand the benefits of the "Sprockets way", I'd like to see Rails use specific tools for specific asset types in order to give developers easy access to all of the available features instead of just 80% of them.