6 ms·

ClojureScript Is Not An Island: Integrating Node Modules

193 points by swannodette 9y ago

nickbauman 9y ago
The thing I like about Clojurescript is the ability to escape the node/NPM ecosystem entirely. It's feature, not a bug, that I don't want to interoperate with node/NPM.
- anmonteiro90 9y ago
  This functionality is opt-in, so you can still live on your island if you'd like :-)
- swannodette 9y ago
  I would say you are in a distinct minority here with that viewpoint :) I've been using / working on ClojureScript since the very beginning and the halfway integration with Node and NPM never seemed particularly desirable to me.
  - nickbauman 9y ago
    I hear you, David, and completely understand your point of view. And let me take this moment to say how much your work is loved and admired. I agree I am in the minority. And I also understand why people love a language like JavaScript: they love it so much that they routinely compile it back into JavaScript before running it o_0 ;)
- y2bd 9y ago
  As someone looking to get into CLJS, where do I find "native" CLJS libraries in case I wish to avoid using NPM?
  - anmonteiro90 9y ago
    Native CLJS libraries are regular JARs that you add to your classpath. Most people publish them to Clojars[1] (which also has Clojure libs). Maven Central may also have some. [1] http://clojars.org http://clojars.org
  - juhoteperi 9y ago
    Clojure Toolbox (https://www.clojure-toolbox.com/ https://www.clojure-toolbox.com/) lists libraries for both Clojure and ClojureScript.
swannodette 9y ago
This feature is really, really a long time coming. Myself and other ClojureScript core contributors are happy to answer any questions people may have.
- Royalaid 9y ago
  It sounds like this handles extern inference if possible. Is that the case? I have wanted to play around with Webrtc for a while now but it basically requires adapter.js (https://github.com/webrtc/adapter https://github.com/webrtc/adapter) and as a result a ton of externs. It would be cool if I could just plug and play!
  - swannodette 9y ago
    There's no extern inference required here as all the original sources are passed directly through Google Closure Compiler. As a benchmark for feasibility when we started we chose React.js as it's non-trivial and heavily depended upon in the ClojureScript ecosystem. We discovered that over all of React's sources we only need 4 extern definitions for 4 generated names in EventsPlugin. So as long as the JS library in question isn't relying on string-based meta programming everything should work. Granted this rules out some libraries, there are plenty of libraries written in a simple straightforward style that will work just fine. In particular I am bullish about fantastic results with ES6+ libraries written with ES6 classes with static import/export. Closure will eat that stuff up.
    - anmonteiro90 9y ago
      Just to add to this statement, even in cases where externs will still be necessary, adding CLJSJS packages[1] to your project's classpath will result in those externs being used even if you don't use the libraries that the packages provide. http://github.com/cljsjs/packages http://github.com/cljsjs/packages
    - sjrd 9y ago
      That's a surprisingly positive result. Usually I expect most JavaScript libraries to be destroyed by Closure's Advanced optimizations, unless they've been written specifically with Closure in mind (which they're not). Anyway, congrats on those features :-) Scala.js doesn't do the part where Closure is applied on JS dependencies; it's only applied to Scala.js code (though with the advantage that we never need any externs).
mtnygard 9y ago
This is a nice step forward for developer experience. I struggled to get NPM and Clojurescript to play nicely. Ultimately, I had two different build systems at play in one project. (Three if you count Webpack.) It'll be nice to have such an easy integration.
fiatjaf 9y ago
How does Clojurescript lookup for a module you declare? For example, in https://github.com/omcljs/om/blob/c68e668a73cc534ecfdc71d6314abc3af911657e/src/main/om/core.cljs https://github.com/omcljs/om/blob/c68e668a73cc534ecfdc71d631... (couldn't think of any Clojurescript library), you have `[om.dom`, `[cljsjs.react`, `[goog.dom`. Where does the compiler look for these things? How does it knows one is a local module and the other an external package? I need to know this so I can implement it in https://github.com/fiatjaf/module-linker https://github.com/fiatjaf/module-linker
- juhoteperi 9y ago
  This is quite a large question to answer. From viewpoint of compiler, there is no much distinction between local and external packages. Clojure (and so ClojureScript compiler) is built on to of JVM and uses Maven dependencies so that is where most of the dependencies come from. From viewpoint of compiler, all files are in JVM classpath (or in directories specified in Cljs compiler options). Om declares it's dependencies in project.clj file: https://github.com/omcljs/om/blob/master/project.clj#L13 https://github.com/omcljs/om/blob/master/project.clj#L13 - om.dom, based on the name, compiler will look for om/dom.cljs/cljc files in classpath, and in this case the file is local to the project - cljsjs.react is foreign-library (https://clojurescript.org/reference/packaging-foreign-deps https://clojurescript.org/reference/packaging-foreign-deps) provided by external cljsjs/react package. Compiler indexes all these foreign-libraries at start, so when it encounters require to one of them knows what to do - goog.dom is Closure module provided by Google Closure library (https://github.com/google/closure-library https://github.com/google/closure-library), which is dependency of ClojureScript. Compiler will look for goog/dom.js file if no .cljs or .cljc by the name is found.
kcorbitt 9y ago
This is a great step for ClojureScript. It's extremely difficult for a compile-to-JS language to bootstrap a large enough ecosystem to stand alone. A good interop story neatly sidesteps that issue.
- priornix 9y ago
  That's very true, one of the key aspects that I use for picking a language is the huge ecosystem. Enough cannot be mentioned about the ecosystem of libraries and tools. With Clojure, you can leverage it to lift the heavy server-side stuff like Spark, Hadoop, and use the same syntax basically for the browser UI and mobile.
Scarbutt 9y ago
Scary, any newbie will read this and run away. Is passing through the complexity, pain and all the caveats one can encounter from this really worth it for using Clojurescript? Serious question, now that with have ES6+ and today babel/webpack are solid tools. I mean from all the modules mess, I didn't get what's the correct way to do javascript interop right now.
- swannodette 9y ago
  I fail to see how Closure Complier is any more complicated, more painful, and has any more caveats than Babel/Webpack. With Closure Compiler at 8 years old and building some of the most sophisticated JS properties on the web (GDocs anyone?) it seems pretty "solid" to me. Perhaps it not clear from the post but that's a big part of why this works, we're not doing the heavy lifting.
  - Scarbutt 9y ago
    Its not about closure, is all the Clojurescript interop issues/gotchas, what's the correct way to do interop today in Clojurescript?
    - swannodette 9y ago
      Nothing has changed about ClojureScript interop in 6 years so I don't really know what you're getting at. We believe ClojureScript interop is pretty good and our users pretty consistently agree.
- prodtorok 9y ago
  Amazing you consider an unstable, undecided, and hectic JavaScript world (language, 3rd party libraries, many competing tools, etc.) against the sanity of a singular language that has been stable and ahead of game since it's creation.
lilactown 9y ago
This is awesome. CLJS is amazingly close to having as good host-interop with JavaScript as plain Clojure does with Java. Will be interesting to see how this impacts things like the React Native and Node.js service-side stories.
priornix 9y ago
One thing I really like about Clojure is how the language has basically stayed the same, but I can use it on the heavy server-side stuffs like working with Hadoop, Spark, ElasticSearch to the front-end stuff like React, React Native, to CLR and now into nodejs, while staying sane [1] in the process. It feels like comparing a nice tropical island paradise to a crowded, hectic, mutating market, where everyone is competing for your attention and selling their newest wares. [1] https://hackernoon.com/how-it-feels-to-learn-javascript-in-2016-d3a717dd577f https://hackernoon.com/how-it-feels-to-learn-javascript-in-2...
bribri 9y ago
Do I still need externs?
- swannodette 9y ago
  The whole point of passing everything through Closure is to avoid needing externs. However there are caveats, see me comment above about building React.js directly from sources.
curryingyou 9y ago
How did clojurescript use node modules / APIs before?
- swannodette 9y ago
  Using NPM for client libraries required using Webpack to bundle those things. If you were targeting Node.js then we just didn't support idiomatic usage of those libraries (users could just call require themselves directly).
xiaoma 9y ago
What's the best resource for learning CLJS as of mid 2017?
- priornix 9y ago
  Depends whether you learn best by reading [1], answering some interactive questions/koans [2], or downloading and playing around with a ToDoMVC app example [3][4][5] or a mixture of the above. Feel free to join the #clojurian Slack channel [6] if you need help. As a beginner, I recommend you looking at the Reagent [7] or Rum [8] React library first, than you can look at something more complex and full-featured like Om [9] later (whose author is also a ClojureScript core contributor @swannodette). PS: You can also learn by implementing a Dashboard like this [10]. Disclaimer: I am the author of this library. [1] http://funcool.github.io/clojurescript-unraveled/ http://funcool.github.io/clojurescript-unraveled/ [2] http://clojurescriptkoans.com/ http://clojurescriptkoans.com/ [3] http://todomvc.com/examples/reagent/ http://todomvc.com/examples/reagent/ [4] https://github.com/reagent-project/reagent/tree/master/examples/todomvc https://github.com/reagent-project/reagent/tree/master/examp... [5] https://github.com/gadfly361/cljs-todomvc https://github.com/gadfly361/cljs-todomvc [6] http://clojurians.net/ http://clojurians.net/ [7] https://reagent-project.github.io/ https://reagent-project.github.io/ [8] https://github.com/tonsky/rum https://github.com/tonsky/rum [9] https://github.com/omcljs/om https://github.com/omcljs/om [10] https://github.com/priornix/antizer https://github.com/priornix/antizer
  - xiaoma 9y ago
    I like small books that build progressively more complex projects (Dave Thomas's Elixir book was good that way, Realm of Racket was my favorite). Short video tutorials are nice as a supplement are nice, too. Daily Drip's elm videos were fantastic. The #1 thing I'm usually looking for is something that eliminates any time-wasting hassles related to getting set up, and all the major language features in some set of working examples. After that, I usually write little command line apps, play with graphics a bit and convert some simple games into the language.
    - VexorLoophole 9y ago
fernandohur 9y ago
I just want to thank the Clojure(Script) team for such an amazing work. It really is a joy to work in Clojure(Script) and improvements like this make it an even better tool to work with. Keep up the good work!

193 points by swannodette 9y ago

The thing I like about Clojurescript is the ability to escape the node/NPM ecosystem entirely. It's feature, not a bug, that I don't want to interoperate with node/NPM.

anmonteiro90 9y ago
This functionality is opt-in, so you can still live on your island if you'd like :-)
swannodette 9y ago
I would say you are in a distinct minority here with that viewpoint :) I've been using / working on ClojureScript since the very beginning and the halfway integration with Node and NPM never seemed particularly desirable to me.
- nickbauman 9y ago
  I hear you, David, and completely understand your point of view. And let me take this moment to say how much your work is loved and admired. I agree I am in the minority. And I also understand why people love a language like JavaScript: they love it so much that they routinely compile it back into JavaScript before running it o_0 ;)
y2bd 9y ago
As someone looking to get into CLJS, where do I find "native" CLJS libraries in case I wish to avoid using NPM?
- anmonteiro90 9y ago
  Native CLJS libraries are regular JARs that you add to your classpath. Most people publish them to Clojars[1] (which also has Clojure libs). Maven Central may also have some. [1] http://clojars.org http://clojars.org
- juhoteperi 9y ago
  Clojure Toolbox (https://www.clojure-toolbox.com/ https://www.clojure-toolbox.com/) lists libraries for both Clojure and ClojureScript.

swannodette 9y ago

This feature is really, really a long time coming. Myself and other ClojureScript core contributors are happy to answer any questions people may have.

Royalaid 9y ago
It sounds like this handles extern inference if possible. Is that the case? I have wanted to play around with Webrtc for a while now but it basically requires adapter.js (https://github.com/webrtc/adapter https://github.com/webrtc/adapter) and as a result a ton of externs. It would be cool if I could just plug and play!
- swannodette 9y ago
  There's no extern inference required here as all the original sources are passed directly through Google Closure Compiler. As a benchmark for feasibility when we started we chose React.js as it's non-trivial and heavily depended upon in the ClojureScript ecosystem. We discovered that over all of React's sources we only need 4 extern definitions for 4 generated names in EventsPlugin. So as long as the JS library in question isn't relying on string-based meta programming everything should work. Granted this rules out some libraries, there are plenty of libraries written in a simple straightforward style that will work just fine. In particular I am bullish about fantastic results with ES6+ libraries written with ES6 classes with static import/export. Closure will eat that stuff up.
  - anmonteiro90 9y ago
    Just to add to this statement, even in cases where externs will still be necessary, adding CLJSJS packages[1] to your project's classpath will result in those externs being used even if you don't use the libraries that the packages provide. http://github.com/cljsjs/packages http://github.com/cljsjs/packages
  - sjrd 9y ago
    That's a surprisingly positive result. Usually I expect most JavaScript libraries to be destroyed by Closure's Advanced optimizations, unless they've been written specifically with Closure in mind (which they're not). Anyway, congrats on those features :-) Scala.js doesn't do the part where Closure is applied on JS dependencies; it's only applied to Scala.js code (though with the advantage that we never need any externs).

mtnygard 9y ago

This is a nice step forward for developer experience. I struggled to get NPM and Clojurescript to play nicely. Ultimately, I had two different build systems at play in one project. (Three if you count Webpack.) It'll be nice to have such an easy integration.

fiatjaf 9y ago

How does Clojurescript lookup for a module you declare? For example, in https://github.com/omcljs/om/blob/c68e668a73cc534ecfdc71d6314abc3af911657e/src/main/om/core.cljs https://github.com/omcljs/om/blob/c68e668a73cc534ecfdc71d631... (couldn't think of any Clojurescript library), you have `[om.dom`, `[cljsjs.react`, `[goog.dom`. Where does the compiler look for these things? How does it knows one is a local module and the other an external package? I need to know this so I can implement it in https://github.com/fiatjaf/module-linker https://github.com/fiatjaf/module-linker

juhoteperi 9y ago
This is quite a large question to answer. From viewpoint of compiler, there is no much distinction between local and external packages. Clojure (and so ClojureScript compiler) is built on to of JVM and uses Maven dependencies so that is where most of the dependencies come from. From viewpoint of compiler, all files are in JVM classpath (or in directories specified in Cljs compiler options). Om declares it's dependencies in project.clj file: https://github.com/omcljs/om/blob/master/project.clj#L13 https://github.com/omcljs/om/blob/master/project.clj#L13 - om.dom, based on the name, compiler will look for om/dom.cljs/cljc files in classpath, and in this case the file is local to the project - cljsjs.react is foreign-library (https://clojurescript.org/reference/packaging-foreign-deps https://clojurescript.org/reference/packaging-foreign-deps) provided by external cljsjs/react package. Compiler indexes all these foreign-libraries at start, so when it encounters require to one of them knows what to do - goog.dom is Closure module provided by Google Closure library (https://github.com/google/closure-library https://github.com/google/closure-library), which is dependency of ClojureScript. Compiler will look for goog/dom.js file if no .cljs or .cljc by the name is found.

kcorbitt 9y ago

This is a great step for ClojureScript. It's extremely difficult for a compile-to-JS language to bootstrap a large enough ecosystem to stand alone. A good interop story neatly sidesteps that issue.

priornix 9y ago
That's very true, one of the key aspects that I use for picking a language is the huge ecosystem. Enough cannot be mentioned about the ecosystem of libraries and tools. With Clojure, you can leverage it to lift the heavy server-side stuff like Spark, Hadoop, and use the same syntax basically for the browser UI and mobile.

Scarbutt 9y ago

Scary, any newbie will read this and run away. Is passing through the complexity, pain and all the caveats one can encounter from this really worth it for using Clojurescript? Serious question, now that with have ES6+ and today babel/webpack are solid tools. I mean from all the modules mess, I didn't get what's the correct way to do javascript interop right now.

swannodette 9y ago
I fail to see how Closure Complier is any more complicated, more painful, and has any more caveats than Babel/Webpack. With Closure Compiler at 8 years old and building some of the most sophisticated JS properties on the web (GDocs anyone?) it seems pretty "solid" to me. Perhaps it not clear from the post but that's a big part of why this works, we're not doing the heavy lifting.
- Scarbutt 9y ago
  Its not about closure, is all the Clojurescript interop issues/gotchas, what's the correct way to do interop today in Clojurescript?
  - swannodette 9y ago
    Nothing has changed about ClojureScript interop in 6 years so I don't really know what you're getting at. We believe ClojureScript interop is pretty good and our users pretty consistently agree.
prodtorok 9y ago
Amazing you consider an unstable, undecided, and hectic JavaScript world (language, 3rd party libraries, many competing tools, etc.) against the sanity of a singular language that has been stable and ahead of game since it's creation.

lilactown 9y ago

This is awesome. CLJS is amazingly close to having as good host-interop with JavaScript as plain Clojure does with Java. Will be interesting to see how this impacts things like the React Native and Node.js service-side stories.

priornix 9y ago

One thing I really like about Clojure is how the language has basically stayed the same, but I can use it on the heavy server-side stuffs like working with Hadoop, Spark, ElasticSearch to the front-end stuff like React, React Native, to CLR and now into nodejs, while staying sane [1] in the process. It feels like comparing a nice tropical island paradise to a crowded, hectic, mutating market, where everyone is competing for your attention and selling their newest wares. [1] https://hackernoon.com/how-it-feels-to-learn-javascript-in-2016-d3a717dd577f https://hackernoon.com/how-it-feels-to-learn-javascript-in-2...

bribri 9y ago

Do I still need externs?

swannodette 9y ago
The whole point of passing everything through Closure is to avoid needing externs. However there are caveats, see me comment above about building React.js directly from sources.

curryingyou 9y ago

How did clojurescript use node modules / APIs before?

swannodette 9y ago
Using NPM for client libraries required using Webpack to bundle those things. If you were targeting Node.js then we just didn't support idiomatic usage of those libraries (users could just call require themselves directly).

xiaoma 9y ago

What's the best resource for learning CLJS as of mid 2017?

priornix 9y ago
Depends whether you learn best by reading [1], answering some interactive questions/koans [2], or downloading and playing around with a ToDoMVC app example [3][4][5] or a mixture of the above. Feel free to join the #clojurian Slack channel [6] if you need help. As a beginner, I recommend you looking at the Reagent [7] or Rum [8] React library first, than you can look at something more complex and full-featured like Om [9] later (whose author is also a ClojureScript core contributor @swannodette). PS: You can also learn by implementing a Dashboard like this [10]. Disclaimer: I am the author of this library. [1] http://funcool.github.io/clojurescript-unraveled/ http://funcool.github.io/clojurescript-unraveled/ [2] http://clojurescriptkoans.com/ http://clojurescriptkoans.com/ [3] http://todomvc.com/examples/reagent/ http://todomvc.com/examples/reagent/ [4] https://github.com/reagent-project/reagent/tree/master/examples/todomvc https://github.com/reagent-project/reagent/tree/master/examp... [5] https://github.com/gadfly361/cljs-todomvc https://github.com/gadfly361/cljs-todomvc [6] http://clojurians.net/ http://clojurians.net/ [7] https://reagent-project.github.io/ https://reagent-project.github.io/ [8] https://github.com/tonsky/rum https://github.com/tonsky/rum [9] https://github.com/omcljs/om https://github.com/omcljs/om [10] https://github.com/priornix/antizer https://github.com/priornix/antizer
- xiaoma 9y ago
  I like small books that build progressively more complex projects (Dave Thomas's Elixir book was good that way, Realm of Racket was my favorite). Short video tutorials are nice as a supplement are nice, too. Daily Drip's elm videos were fantastic. The #1 thing I'm usually looking for is something that eliminates any time-wasting hassles related to getting set up, and all the major language features in some set of working examples. After that, I usually write little command line apps, play with graphics a bit and convert some simple games into the language.
  - VexorLoophole 9y ago

fernandohur 9y ago

I just want to thank the Clojure(Script) team for such an amazing work. It really is a joy to work in Clojure(Script) and improvements like this make it an even better tool to work with. Keep up the good work!