7 ms·

Maybe stop using Grunt?

58 points by NTH 12y ago

jakswa 12y ago
I switched to skimming the article at: [If] you just want to set up your single project with a task workflow, then go ahead and use Grunt (or gulp). They will serve you just fine.
- NTH 12y ago
  Yep, my article is aimed at people with a very specific use case, and if that's not your use case, the article will not be as relevant to you.
  - jakswa 12y ago
    Maybe the title should reflect that?
tinco 12y ago
I'm a bit confused, what are they using grunt for? We're using gulp, but it's basically the same software. It just compiles our coffeescript, tags some assets and moves some files around, perhaps run a minifier somewhere. In what situation do you need your build toolchain to do more than that?
- geetee 12y ago
  I think I found myself in a similar situation as the OP. We have multiple different, but architecturally similar, products which each get their own Gruntfile. If we stop here, we're solid, perfect. Unfortunately, each product has multiple flavors which each need to build into their own separate output directories. For us, that's where the Grunt patterns break down and get awkward. It's actually been good enough for almost 2 years now, but I can definitely see the cracks.
  - mikegioia 12y ago
    Why don't you just write shell scripts?
    - deleted 12y ago
      [deleted]
    - geetee 12y ago
      I will probably go that route if we ever rebuild those tools.
  - NTH 12y ago
    Yes, that sounds very much like the scenario we see ourselves in. For us, we have too many projects for copy/pasting the Gruntfile between them to be a viable option, and the Gruntfiles themselves would have been too complex. I'm not convinced the shell scripts are a good alternative, though. (Maybe they are for your scenario.) I would like to write a follow-up post talking about this.
badthingfactory 12y ago
This article should be titled "Maybe you should stop using Grunt for things it was never meant to do to begin with"
- NTH 12y ago
  Yes, I acknowledge in the article that Grunt is not designed for this, and if you use Grunt for what it is designed for, it works pretty well. However, it wasn't obvious to me the using Grunt to build an SDK was outside of its scope when I started the project, so I figured it may not be obvious to other people as well.
jessaustin 12y ago
...I would like the task system to provide an abstraction for pipelining tasks. (Perhaps gulp helps with this?) Hmmm, perhaps someone needs to look at any gulpfile ever written, anywhere. gulp.src('src') .pipe(this()) .pipe(that()); Maybe it's frustrating for anyone who spent "18 months" hacking grunt, but gulp really is better. Both tools are built on node for Pete's sake; async really shouldn't be that hard.
- NTH 12y ago
  I am excited to look into gulp!
- nostrademons 12y ago
  There is also the 'scripts' section of your package.json, 'npm run', and UNIX pipes: scripts: { "do_something": "this | that" } $ npm run do_something See also http://blog.keithcirkel.co.uk/why-we-should-stop-using-grunt/ http://blog.keithcirkel.co.uk/why-we-should-stop-using-grunt...
  - jessaustin 12y ago
    Yes while gulp > grunt, in many cases npm by itself is what you want. gulp.watch(), in particular, seems more difficult to use than the watch facility built into many (but not all) node modules. Also something like Browserify must be somewhat shoehorned into gulp. It might be worth it to look at what gulp has that npm run is missing? Maybe it doesn't need to worry about streaming files, but the notion of some tasks depending on arbitrary sets of other tasks (a generalization of npm's "pre-" and "post-") seems pretty handy.
- asadlionpk 12y ago
  yes I found gulp to be more easy to implement than grunt. Maybe I haven't mastered grunt but I am happy with gulp.
greggman 12y ago
I probably just don't have enough experience. I like the idea that Grunt and Gulp are written in JS so using JS to build JS seems like a win. But...., aren't build systems supposed to do dependency checking and just built the minimal amount of stuff? Maybe few JS projects are big enough for it to matter or maybe I missed where the dependencies are checked but AFAICT the default is to build everything always.
- skybrian 12y ago
  First the build tool needs to load the build files that declare the dependencies, so it knows which dependencies to check. For big, distributed projects, this step can be slow.
  - cdnsteve 12y ago
    Could this not be cached? If deps aren't changing, why do them all from scratch?
- jdlshore 12y ago
  Grunt doesn't do dirty-file dependency checking. Gulp does, sort of, but it's awkward. There are many, many build tools in JS-land. Grunt and Gulp happen to be the most popular. (This isn't really due to technical merit. They're popular mostly because they're popular.) My personal favorite is Jake, which is a classic Rake-like (or Make-like) tool.
  - acdlite 12y ago
    Or instead of a Make-like tool you could just use Make itself :) Works really well.
    - jdlshore 12y ago
      You could, but I much prefer more expressive tools. Make's DSL is spectacularly crappy.
- digitalzombie 12y ago
falcolas 12y ago
I keep seeing people mention "a mess of shell scripts", and then use replacements which are dozens if not hundreds of lines of custom-configuration files. Shell scripts, when you're not dealing with autotools generated script, aren't really that big, or even that complicated. Not to mention that the ability to chain disparate tools and subshells can be incredibly elegant and powerful.
- tieTYT 12y ago
  > Shell scripts, when you're not dealing with autotools generated script, aren't really that big, or even that complicated. They're not always portable. Some people develop on windows.
  - falcolas 12y ago
    Windows offers shell scripts as well. Powershell, instead of Bash/Zsh/..., but they still offer many of the same opportunities. Considering that teams frequently homogenize on one type of machine - bet it windows or Mac or Linux, shell scripts are typically portable enough for these use cases.
    - onion2k 12y ago
      Msysgit on Windows comes with a bash emulator (for running hooks) that could be used in a cross-platform build process.
- jeswin 12y ago
  I agree mostly. But when you're in the node ecosystem, often times you need to run JS as part of the build. For example, you might need to create some JSON data as part of the build. I ended up writing my own build library (npm install crankshaft). Docs aren't up yet, but here's what build scripts look like - https://github.com/jeswin/hitchslap/blob/master/build.js https://github.com/jeswin/hitchslap/blob/master/build.js (simple example) and https://github.com/jeswin/fora/blob/master/www-client/build-config.js https://github.com/jeswin/fora/blob/master/www-client/build-... (more complex, with dependent tasks)
- 12y ago
pothibo 12y ago
I couldn't refrain myself for posting something I just wrote http://pothibo.com/2015/3/churn http://pothibo.com/2015/3/churn I think JavaScript people might need to read it.
MichaelGG 12y ago
How are all of you using any npm-based system combined with a continuous integration or clean build environment? It seems that a simple yeoman/grunt setup that just preprocesses some HTML and JS takes... a very long time. A minute or two just for the actual work. Plus 5-15 minutes for npm to make a thousand requests and write 10K+ files to the filesystem. I hack around it by keeping node_modules around as a special resource on a custom build server and symlinking it in, but that wouldn't work with a proper "start from zero" build system.
- lucaspiller 12y ago
  Why does it matter if your build system takes a while? You shouldn't be pushing it to there until you are happy it works locally, at which point you can move on to the next task and forget about it.
- tieTYT 12y ago
  > It seems that a simple yeoman/grunt setup that just preprocesses some HTML and JS takes... a very long time. I have not experienced this. I built a 15k+ line CoffeeScript app. Every time a file changed it would compile it to JS and run thousands of unit tests in ~1 second. As for the clean NPM... I never deleted that directory. I believe I read an article somewhere that you should actually commit your module dir. Maybe that's no longer in vogue though?
- Kudos 12y ago
  Both Travis CI and Circle CI cache your node_modules directory between builds. If you don't change your package.json, it just uses that cache instead.
- doublerebel 12y ago
  1) We have our own private npm that can optionally include a caching layer. This has been invaluable since the beginning of our company to manage internal dependencies. 2) We use StriderCD, the self-hosted CD/CI, and it too supports caching node_modules. 2 minutes is a 'long' time for us from git push to autodeploy. 3) We use gulp and npm always. Grunt and bower are a mess. EDIT: Also, we npm shinkwrap projects. It avoids numerous issues and makes module caching a natural part of the process.
mike_hearn 12y ago
I feel I must be missing something here, but ... The inability to specify config asynchronously turned into a disaster. Our primary need for async config was specifying temp directories for tasks to operate in. We used tmp, which does not have a synchronous api ...... I would like the task system to provide an abstraction for pipelining tasks Is this abstraction not normally just called a program? Where you can do something like: val tmp = makeTempDir() generateFoo(outputDir = tmp) generateBar(inputDir = tmp) etc? How much of this complexity is fundamental and how much is people attempting to work around Node's inherent limitations and fully asynchronous APIs? Perhaps the author should check out something like Gradle instead?
- woah 12y ago
  Gulp has no temporary directories.
andrewchambers 12y ago
Maybe I am missing something, but it seems like most of theses tools are working around the fact the windows doesn't come with make by default. They are essentially reinventing the wheel for web developers who have never worked from the command line before.
- falcolas 12y ago
  There's a lot of distaste for make as well, typically centered on the limitations of the dependency and .PHONY targets. I rarely find them to be limitations for my projects, but there's certainly a reason so many different build systems have been built, even for Linux.
- brandonmenc 12y ago
  I've written both makefiles and gulpfiles. Yeah, I guess you can get make to do the exact same things as gulp, but gulp already does what a web developer needs, in a language they already know, with a massive plugin ecosystem you don't have to re-implement in shell.
  - andrewchambers 12y ago
    A plug in system is nice. I didn't say it was totally bad, I just think its interesting.
mcx 12y ago
I switched to webpack (the config is like ~100 loc) and it has replaced the need for a grunt or gulp file entirely.
gadr90 12y ago
I have had similar problems with my team, and I went with the "meta-project" approach. I created an external project with all default configs[1]. It works well-ish. Main complaint is speed, grunt really is dumb to not lazy load... [1]: http://github.com/vtex/grunt-vtex http://github.com/vtex/grunt-vtex
nmalaguti 12y ago
Check out Laravel Elixer[0]. It's built on top of gulp and exposes a very simple SDK for compiling the various parts of your app. [0]: http://laravel.com/docs/5.0/elixir http://laravel.com/docs/5.0/elixir
mikejholly 12y ago
grunt < gulp < webpack < make
- jessaustin 12y ago
  I'm willing to believe the first two comparisons even though I haven't used webpack, but I have tried to use make on multiple occasions over the course of decades. (That is, I've attempted to write new Makefiles and to modify existing ones to fit a purpose; I'm not talking about just using a Makefile someone else already wrote.) make is not nice. I admire anyone who can use it! To be specific, what advantages does webpack have over gulp that webpack doesn't also have over make? (I would also accept advantages that gulp has over grunt that gulp doesn't also have over make.)
- pluma 12y ago
  * < npm scripts
latchkey 12y ago
I started with grunt, then moved to gulp because I like the plugin support more and it felt less magical to me. Then, I found that working with many different projects, gulp wasn't DRY enough for me. I kept copying any pasting my tasks across projects and forgetting to go back and improve them as I went. I also disliked the chain of require statements and devDependencies in my package.json. I was spending too much time coding my build. So, I started a project [1] to move gulp tasks into re-usable functions and hide the devDependencies. Now I'm pretty happy with the result, my gulpfile [2] is now mostly configuration and very little code. [1] https://github.com/lookfirst/gulp-helpers/ https://github.com/lookfirst/gulp-helpers/ [2] https://github.com/lookfirst/systemjs-seed/blob/master/gulpfile.js https://github.com/lookfirst/systemjs-seed/blob/master/gulpf...

58 points by NTH 12y ago

I switched to skimming the article at: [If] you just want to set up your single project with a task workflow, then go ahead and use Grunt (or gulp). They will serve you just fine.

NTH 12y ago
Yep, my article is aimed at people with a very specific use case, and if that's not your use case, the article will not be as relevant to you.
- jakswa 12y ago
  Maybe the title should reflect that?

tinco 12y ago

I'm a bit confused, what are they using grunt for? We're using gulp, but it's basically the same software. It just compiles our coffeescript, tags some assets and moves some files around, perhaps run a minifier somewhere. In what situation do you need your build toolchain to do more than that?

geetee 12y ago
I think I found myself in a similar situation as the OP. We have multiple different, but architecturally similar, products which each get their own Gruntfile. If we stop here, we're solid, perfect. Unfortunately, each product has multiple flavors which each need to build into their own separate output directories. For us, that's where the Grunt patterns break down and get awkward. It's actually been good enough for almost 2 years now, but I can definitely see the cracks.
- mikegioia 12y ago
  Why don't you just write shell scripts?
  - deleted 12y ago
    [deleted]
  - geetee 12y ago
    I will probably go that route if we ever rebuild those tools.
- NTH 12y ago
  Yes, that sounds very much like the scenario we see ourselves in. For us, we have too many projects for copy/pasting the Gruntfile between them to be a viable option, and the Gruntfiles themselves would have been too complex. I'm not convinced the shell scripts are a good alternative, though. (Maybe they are for your scenario.) I would like to write a follow-up post talking about this.

badthingfactory 12y ago

This article should be titled "Maybe you should stop using Grunt for things it was never meant to do to begin with"

NTH 12y ago
Yes, I acknowledge in the article that Grunt is not designed for this, and if you use Grunt for what it is designed for, it works pretty well. However, it wasn't obvious to me the using Grunt to build an SDK was outside of its scope when I started the project, so I figured it may not be obvious to other people as well.

jessaustin 12y ago

...I would like the task system to provide an abstraction for pipelining tasks. (Perhaps gulp helps with this?) Hmmm, perhaps someone needs to look at any gulpfile ever written, anywhere. gulp.src('src') .pipe(this()) .pipe(that()); Maybe it's frustrating for anyone who spent "18 months" hacking grunt, but gulp really is better. Both tools are built on node for Pete's sake; async really shouldn't be that hard.

NTH 12y ago
I am excited to look into gulp!
nostrademons 12y ago
There is also the 'scripts' section of your package.json, 'npm run', and UNIX pipes: scripts: { "do_something": "this | that" } $ npm run do_something See also http://blog.keithcirkel.co.uk/why-we-should-stop-using-grunt/ http://blog.keithcirkel.co.uk/why-we-should-stop-using-grunt...
- jessaustin 12y ago
  Yes while gulp > grunt, in many cases npm by itself is what you want. gulp.watch(), in particular, seems more difficult to use than the watch facility built into many (but not all) node modules. Also something like Browserify must be somewhat shoehorned into gulp. It might be worth it to look at what gulp has that npm run is missing? Maybe it doesn't need to worry about streaming files, but the notion of some tasks depending on arbitrary sets of other tasks (a generalization of npm's "pre-" and "post-") seems pretty handy.
asadlionpk 12y ago
yes I found gulp to be more easy to implement than grunt. Maybe I haven't mastered grunt but I am happy with gulp.

greggman 12y ago

I probably just don't have enough experience. I like the idea that Grunt and Gulp are written in JS so using JS to build JS seems like a win. But...., aren't build systems supposed to do dependency checking and just built the minimal amount of stuff? Maybe few JS projects are big enough for it to matter or maybe I missed where the dependencies are checked but AFAICT the default is to build everything always.

skybrian 12y ago
First the build tool needs to load the build files that declare the dependencies, so it knows which dependencies to check. For big, distributed projects, this step can be slow.
- cdnsteve 12y ago
  Could this not be cached? If deps aren't changing, why do them all from scratch?
jdlshore 12y ago
Grunt doesn't do dirty-file dependency checking. Gulp does, sort of, but it's awkward. There are many, many build tools in JS-land. Grunt and Gulp happen to be the most popular. (This isn't really due to technical merit. They're popular mostly because they're popular.) My personal favorite is Jake, which is a classic Rake-like (or Make-like) tool.
- acdlite 12y ago
  Or instead of a Make-like tool you could just use Make itself :) Works really well.
  - jdlshore 12y ago
    You could, but I much prefer more expressive tools. Make's DSL is spectacularly crappy.
digitalzombie 12y ago

falcolas 12y ago

I keep seeing people mention "a mess of shell scripts", and then use replacements which are dozens if not hundreds of lines of custom-configuration files. Shell scripts, when you're not dealing with autotools generated script, aren't really that big, or even that complicated. Not to mention that the ability to chain disparate tools and subshells can be incredibly elegant and powerful.

tieTYT 12y ago
> Shell scripts, when you're not dealing with autotools generated script, aren't really that big, or even that complicated. They're not always portable. Some people develop on windows.
- falcolas 12y ago
  Windows offers shell scripts as well. Powershell, instead of Bash/Zsh/..., but they still offer many of the same opportunities. Considering that teams frequently homogenize on one type of machine - bet it windows or Mac or Linux, shell scripts are typically portable enough for these use cases.
  - onion2k 12y ago
    Msysgit on Windows comes with a bash emulator (for running hooks) that could be used in a cross-platform build process.
jeswin 12y ago
I agree mostly. But when you're in the node ecosystem, often times you need to run JS as part of the build. For example, you might need to create some JSON data as part of the build. I ended up writing my own build library (npm install crankshaft). Docs aren't up yet, but here's what build scripts look like - https://github.com/jeswin/hitchslap/blob/master/build.js https://github.com/jeswin/hitchslap/blob/master/build.js (simple example) and https://github.com/jeswin/fora/blob/master/www-client/build-config.js https://github.com/jeswin/fora/blob/master/www-client/build-... (more complex, with dependent tasks)
12y ago

pothibo 12y ago

I couldn't refrain myself for posting something I just wrote http://pothibo.com/2015/3/churn http://pothibo.com/2015/3/churn I think JavaScript people might need to read it.

MichaelGG 12y ago

How are all of you using any npm-based system combined with a continuous integration or clean build environment? It seems that a simple yeoman/grunt setup that just preprocesses some HTML and JS takes... a very long time. A minute or two just for the actual work. Plus 5-15 minutes for npm to make a thousand requests and write 10K+ files to the filesystem. I hack around it by keeping node_modules around as a special resource on a custom build server and symlinking it in, but that wouldn't work with a proper "start from zero" build system.

lucaspiller 12y ago
Why does it matter if your build system takes a while? You shouldn't be pushing it to there until you are happy it works locally, at which point you can move on to the next task and forget about it.
tieTYT 12y ago
> It seems that a simple yeoman/grunt setup that just preprocesses some HTML and JS takes... a very long time. I have not experienced this. I built a 15k+ line CoffeeScript app. Every time a file changed it would compile it to JS and run thousands of unit tests in ~1 second. As for the clean NPM... I never deleted that directory. I believe I read an article somewhere that you should actually commit your module dir. Maybe that's no longer in vogue though?
Kudos 12y ago
Both Travis CI and Circle CI cache your node_modules directory between builds. If you don't change your package.json, it just uses that cache instead.
doublerebel 12y ago
1) We have our own private npm that can optionally include a caching layer. This has been invaluable since the beginning of our company to manage internal dependencies. 2) We use StriderCD, the self-hosted CD/CI, and it too supports caching node_modules. 2 minutes is a 'long' time for us from git push to autodeploy. 3) We use gulp and npm always. Grunt and bower are a mess. EDIT: Also, we npm shinkwrap projects. It avoids numerous issues and makes module caching a natural part of the process.

mike_hearn 12y ago

I feel I must be missing something here, but ... The inability to specify config asynchronously turned into a disaster. Our primary need for async config was specifying temp directories for tasks to operate in. We used tmp, which does not have a synchronous api ...... I would like the task system to provide an abstraction for pipelining tasks Is this abstraction not normally just called a program? Where you can do something like: val tmp = makeTempDir() generateFoo(outputDir = tmp) generateBar(inputDir = tmp) etc? How much of this complexity is fundamental and how much is people attempting to work around Node's inherent limitations and fully asynchronous APIs? Perhaps the author should check out something like Gradle instead?

woah 12y ago
Gulp has no temporary directories.

andrewchambers 12y ago

Maybe I am missing something, but it seems like most of theses tools are working around the fact the windows doesn't come with make by default. They are essentially reinventing the wheel for web developers who have never worked from the command line before.

falcolas 12y ago
There's a lot of distaste for make as well, typically centered on the limitations of the dependency and .PHONY targets. I rarely find them to be limitations for my projects, but there's certainly a reason so many different build systems have been built, even for Linux.
brandonmenc 12y ago
I've written both makefiles and gulpfiles. Yeah, I guess you can get make to do the exact same things as gulp, but gulp already does what a web developer needs, in a language they already know, with a massive plugin ecosystem you don't have to re-implement in shell.
- andrewchambers 12y ago
  A plug in system is nice. I didn't say it was totally bad, I just think its interesting.

mcx 12y ago

I switched to webpack (the config is like ~100 loc) and it has replaced the need for a grunt or gulp file entirely.

gadr90 12y ago

I have had similar problems with my team, and I went with the "meta-project" approach. I created an external project with all default configs[1]. It works well-ish. Main complaint is speed, grunt really is dumb to not lazy load... [1]: http://github.com/vtex/grunt-vtex http://github.com/vtex/grunt-vtex

nmalaguti 12y ago

Check out Laravel Elixer[0]. It's built on top of gulp and exposes a very simple SDK for compiling the various parts of your app. [0]: http://laravel.com/docs/5.0/elixir http://laravel.com/docs/5.0/elixir

mikejholly 12y ago

grunt < gulp < webpack < make

jessaustin 12y ago
I'm willing to believe the first two comparisons even though I haven't used webpack, but I have tried to use make on multiple occasions over the course of decades. (That is, I've attempted to write new Makefiles and to modify existing ones to fit a purpose; I'm not talking about just using a Makefile someone else already wrote.) make is not nice. I admire anyone who can use it! To be specific, what advantages does webpack have over gulp that webpack doesn't also have over make? (I would also accept advantages that gulp has over grunt that gulp doesn't also have over make.)
pluma 12y ago
* < npm scripts

latchkey 12y ago

I started with grunt, then moved to gulp because I like the plugin support more and it felt less magical to me. Then, I found that working with many different projects, gulp wasn't DRY enough for me. I kept copying any pasting my tasks across projects and forgetting to go back and improve them as I went. I also disliked the chain of require statements and devDependencies in my package.json. I was spending too much time coding my build. So, I started a project [1] to move gulp tasks into re-usable functions and hide the devDependencies. Now I'm pretty happy with the result, my gulpfile [2] is now mostly configuration and very little code. [1] https://github.com/lookfirst/gulp-helpers/ https://github.com/lookfirst/gulp-helpers/ [2] https://github.com/lookfirst/systemjs-seed/blob/master/gulpfile.js https://github.com/lookfirst/systemjs-seed/blob/master/gulpf...