15 ms·

Why I Prefer Makefiles over Package.json Scripts

235 points by ducaale 5y ago

jgrahamc 5y ago
Reminds me of something I wrote years ago: https://blog.jgc.org/2010/11/things-make-got-right-and-how-to-make.html https://blog.jgc.org/2010/11/things-make-got-right-and-how-t...
- jitl 5y ago
  Has anyone Fixed Make in a way you find good and right since you wrote that? (My last few forays into the build systems domain have mostly turned up “Bazel but for X audience” so I can’t recall a satisfying solution)
gjvc 5y ago
I'm pleased to see this being espoused. One thing I always tell junior people is ""don't be tempted to think of make(1) as out-of-date [1], and to view packages available via npm etc as better replacements, as many tasks fit the model of make(1), if not the title "a program for directing recompilation" "" I am often viewed with suspicion until I show them a (much more compact) replacement for their custom python/js program expressed as a makefile without any of the process management requiring debugging. :-) An alternative approach for the "batches of files to process" situation is to generate the commands and feed them to GNU parallel for execution. [1] no pun intended
- nicoburns 5y ago
  Agreed if you only have to support linux/unix, but the one good thing about the Node ecosystem is that they tend to work on windows too.
  - gjvc 5y ago
    not the point, also "WSL"
  - verelo 5y ago
    While i agree, I’ve spent literal days fighting with node modules in recent months. Sure make isn’t perfect either but it’s a lot less, confusing? Ie json isn’t a language for humans, but often you really need to read it to correct some issue or bad merge from another contributor.
    - nicoburns 5y ago
      > Sure make isn’t perfect either but it’s a lot less, confusing? I think that's a matter of what you're used to. I don't use make all that often, so I find make files beyond a few lines quite a lot to get my head around, whereas I spend all day writing JavaScript so I can read/write JSON in my sleep.
      - verelo 5y ago
        Maybe. I actually use node much more often, i don't even recall the last time i used a make file in production, i just find parsing json as a human to be rather painful.
  - Steltek 5y ago
    Make has worked on Windows longer than Node.js has existed.
    - nicoburns 5y ago
      Make works fine, it's the commands you call from make that don't tend to as compatible. For example, try running rm -rf on windows. It might work if you have GNU tools installed, but it certainly won't work out of the box. On the other hand, the node.js `rimraf` package will do the same thing with no cross-platform compatibility issues.
      - Steltek 5y ago
        When using package.json, does npm intercept a script containing "rm -rf" and translate it to something different?
        geewee 5y ago
        No, that's why there's a bunch of packages such as rimraf[0] that implements that sort of functionality in a cross-platform way that most people use in their scripts [0]: https://www.npmjs.com/package/rimraf https://www.npmjs.com/package/rimraf
        nicoburns 5y ago
        No, but it does put binaries from any installed node packages in your PATH (locally when running scripts).
    - jchw 5y ago
      The history of Make on Windows would Make one think twice about relying on it. What shell do you get when running Make on Windows?
  - throw0101a 5y ago
    http://gnuwin32.sourceforge.net/packages/make.htm http://gnuwin32.sourceforge.net/packages/make.htm https://community.chocolatey.org/packages/make https://community.chocolatey.org/packages/make
- andrew_ 5y ago
  Might it not be a disservice to juniors to point them away from deeply understanding the platform they're working with?
  - gjvc 5y ago
    Showing someone an alternative approach does not prevent someone "deeply understanding" their current method; it's not zero-sum.
    - gjvc 5y ago
      Might it not be a disservice to juniors to point them away from deeply understanding the platform they're working with? I cannot overstate how annoyed I was by the patronising tone of this comment.
- jjice 5y ago
  Biggest pain of package.json to me is having to be crammed on one line. Slap in a conditional and now we have a messy thing to read, compared to its multiline counterpart.
- Wicher 5y ago
  Speaking of debugging, I found this handy: print-%: ; @echo '$(subst ','\'',$*=$($*))' include Makefile Save somewhere, eg as ~/Makefile.debug, then when you want to know the evaluated value of some variable or task in some Makefile, you prefix it with print- — for example, LOCALES: make -f ~/Makefile.debug print-LOCALES It'll print the evaluated LOCALES of the Makefile in the directory you run this in.
  - BeefWellington 5y ago
    This is amazing. Thank you for the tip!
  - gjvc 5y ago
    Very useful, will give it a shot. Thank you. :-)
Cthulhu_ 5y ago
I've only started using a Makefile ~two years ago when I started a big Go project at work, and I quite like it. There's a few caveats here and there - Makefile specific syntax, shell-script specific syntax, and whether I should put .PHONY in front of every command (I guess 'no unless you have a file with the exact same name'?), but it's quite compact and straightforward. The most complicated command I have is something that removes a folder or set of files, invokes the swagger generator with a list of options, copies it to a target folder, and passes it through a formatter/processor. But it's very straightforward, and it's "just" shell script, not shell script wrapped in a JSON document, or some Java tool invoked indirectly through an XML configuration file like back in the day with Maven.
- JamesSwift 5y ago
  If using Make as a task runner and not build cache, I just put `.PHONY: %` at the top which means everything is marked `.PHONY`
  - nablaone 5y ago
    Thanks!
silves89 5y ago
I have very similar use-cases, and I also I found makefiles a bit limiting. I wrote lk[1] to make this sort of thing easier, and you can just write plain bash functions instead of makefiles. [1] https://github.com/jamescoleuk/lk https://github.com/jamescoleuk/lk
jitl 5y ago
I like Make, and use it in personal JavaScript ecosystem projects that do actually need to build things with interdependency (eg https://github.com/justjake/quickjs-emscripten/blob/master/Makefile https://github.com/justjake/quickjs-emscripten/blob/master/M...). I’ve also seen Makefiles grown to be horrendous monstrosities masquerading as command-line tools; for about a year at Airbnb we used a 1000+ line Makefile as the main tool for fiddling with Kubernetes cuz one of our senior engineers didn’t like Ruby, and I’ve seen another one get close to that level of cravenness. What I learned from supporting Make and shell among a few different audiences is that most developers have no interest in how to write or maintain shell-like tooling. They forget or mess up quoting rules constantly, and eschew learning things and good design in these tools to a much greater degree than in their “normal” work in Java/Python/Ruby/Typescript/Golang. For every POSIXly Correct HN Commenter (of which I count myself a member), there’s 100 regular software engineers who won’t read a `man` page on what $@ or $< mean in Make. I know that if I start writing a Makefile for $JOB, it’s gonna be me and that one guy who uses tcsh who are gonna maintain it and answer questions. (Although for what it’s worth we don’t use package.json scripts either thank goodness. All our complicated build steps are typescript commands, and our glue is CI system YAML files.)
- unqueued 5y ago
  I would try to use JavaScript tooling in JavaScript projects, especially if I share it with other developers. I would love to see better handling of filename transformation, file watching, and parallelization. A few times over the years I had given up and just used Make instead of Gulp or Grunt. Most recently, I wanted to watch if files had been deleted (if directory entries had been modified). It ended up being a few lines of Makefile. And once you understand the syntax, it is expressive and simple, and way more portable. There was some good discussion awhile ago on this[1]: [1]: https://news.ycombinator.com/item?id=7622296 https://news.ycombinator.com/item?id=7622296
dncornholio 5y ago
But you don't write scripts in package.json and it also failed to show an example how the javascript dependencies could be loaded with make. I think make works well in combination with package.json, but not as a replacement. You can also do the same stuff in an NPM command (which just loads node code) that you could do in a makefile script.
- andrew_ 5y ago
  Not to mention that the top three package managers will load binary files from node_modules automagically by name only, without having to specify paths. That gets hairy in a monorepo, and package managers running package.json scripts manage them just fine.
  - silverwind 5y ago
    `npx` solves this elegantly as long as you `cd` to the project directory first.
BeefWellington 5y ago
Over here in crazytown I'm using Makefiles to automate system deployments. I thought it was insane at first but actually I've come around to it; Make hits that sweet spot of being ubiquitous, simple, and flexible. There's a little bit of a learning curve but once you're over that initial hurdle it's pretty straightforward, especially for one-way operations, e.g. install without uninstall, build without clean, etc.
boondaburrah 5y ago
For a lot of the same reasons in this article I like to use tup[0] when I can. It doesn't integrate into anything which is both good and bad. I wish my IDE could check with tup to see what dependencies get pulled in by what files. However, it's nice that it doesn't care what language or ecosystem I'm using. Also, it's very strict about declaring dependencies properly, and will actually fail the build if you've set things up in a way that depends on something not tracked (by watching filesystem access as the compiler runs). That gives me warm fuzzies that my builds are reproducible. Also I can get a neat dependency graph as a PNG if I want. [0] https://gittup.org/tup/ https://gittup.org/tup/
deleted 5y ago
[deleted]
devn0ll 5y ago
May I also direct your attention to self-documenting Makefiles: https://www.thapaliya.com/en/writings/well-documented-makefiles/ https://www.thapaliya.com/en/writings/well-documented-makefi...
- molszanski 5y ago
  Here is the one I use and recommend: https://github.com/awinecki/magicfile https://github.com/awinecki/magicfile
mro_name 5y ago
one of the few annoyances is spaces in paths. Just avoid them anyway, right?
- beej71 5y ago
  This is one of the few fixes I'd really like to see. I'm from a Unix background so I don't tend to have spaces anywhere, but it's still a splinter that needs removing.
nicoburns 5y ago
I would like to call attention to `just` (https://github.com/casey/just https://github.com/casey/just) which is a modern take on "make if we didn't have to keep it backwards compatible", and more focussed on just task running rather than tracking dependencies.
- marcuskaz 5y ago
  Yes! This is what a modern task runner should be. Includes ability to list recipes, and a consistent format between multiple platforms. Just two things Make doesn't have.
- michidk 5y ago
  I was just going to say that. Just is awesome and I use it it most of my projects.
- enriquto 5y ago
  There's no -j option nor mention of parallelism on the docs. Does it run independent targets in parallel? This is easily the main feature of make. I don't understand how it can work in parallel if targets are not somehow associated to files. What's the advantage to a shell script with separate sub-commands, then?
  - q3k 5y ago
    ‘just’ apparently isn’t a build system, just a ‘task runner’. So I assume it doesn’t need/want to implement parallelism.
    - dahfizz 5y ago
      It sounds like `just` is not a replacement for make, then. It seems like it could be a nice wrapper on top, i.e. `just build` could invoke the make build system.
    - toast0 5y ago
      Parallelism is pretty useful for just a 'task runner' too. I've (ab)used Make for server pushes, and sometimes you want those to run one at a time, and sometimes you want -j 100 (sometimes -j 100 is even effective).
WesolyKubeczek 5y ago
Why not both? Use make for dependency graph, but also use "npm run" or "yarn run" to make sure you're running with the correct Node and correct paths to all your tools. I know you can specify those in your Makefile too, but the plus of running via npm/yarn is that they know where your binaries live, where your mode_nodules are, and that sort of thing.
- andrew_ 5y ago
  replace that with `pnpm` and you've got a winner. npm and yarn are old hat (unless you're working with react native, which is another discussion)
- JamesSwift 5y ago
  Yes, I usually treat Make as the user-friendly facade to the underlying framework tooling (which might be e.g. Rails, Docker, or npm). `make bootstrap` gets the project setup. `make up` starts the project. For npm projects these usually just wrap package.json stuff. But I don't need to remember what tech a project is in, or where to look for the commands, if Make is the entry point.
karmicthreat 5y ago
Out in Grand Rapids, Mi where AO is located there seems to be a bit of a tradition of abusing make in a good way. It's where I picked up the habit as well. I think its just institutional knowledge/tradition that has creeped through the various firms here. I kind of wonder what other "traditions" get passed around on a regional level?
sigzero 5y ago
Task is nice: https://taskfile.dev/#/ https://taskfile.dev/#/
mark_l_watson 5y ago
I also use Makefiles, been using them for 40 years. I find that Makefile targets for misc. things like grabbing remote training data, running tests, building documentation, etc., etc. augments information in READ files, that is in addition to functionally saving time, serves as documentation when I haven’t looked at a project in 6 months.
deepsun 5y ago
What are the benefits of Makefiles compared to a bunch of shell scripts?
- enriquto 5y ago
  That you get parallelism and partial re-running for free.
  - andrew_ 5y ago
    parallelism is common in package managers these days https://pnpm.io/cli/run#--parallel https://pnpm.io/cli/run#--parallel
    - monocasa 5y ago
      No, make understands the directed acyclic graphs of dependencies, regardless of language, and will walk that graph in parallel. No forcing or ignoring sorting, and no enforced package level blocking on scripts.
      - immibis 5y ago
        It's a shame this doesn't work for Java in particular (not JS) as Java's coding conventions/requirements do not lend themselves at all to manually updating a list of file dependencies.
- JamesSwift 5y ago
  A single, system-and-language-agnostic entry point to the actions you want to run with very clear dependency rules built in.
  - deepsun 5y ago
    Besides Windows, shell scripts are system-and-language agnostic (not using bash/zsh-specific syntax of course). Also I'm not sure Windows can run Makefiles out-of-the-box.
andrew_ 5y ago
Stick around in any programming ecosystem long enough, and you live long enough to see strategy, taste, and "discovery" come full circle. We had make, then package.json "scripts," then grunt, then gulp, then the trend shifted back towards package.json "scripts." (Note: Using "scripts" to differentiate the package.json property versus the generic word use) I like that the author isn't being authoritative, but there could be some additional due diligence. I ran the make-is-faster loose benchmark via PNPM and runtime was 0m0.022s for make and 0m0.012s for PNPM. If I care about those 10ms, PNPM is my horse. Yarn is a glacier compared to PNPM. Another thing I would've liked to see comment on is the automatic pre and post paradigm that package.json "scripts" affords. The big three package managers all support pre and post, and it makes arranging "scripts" a breeze, it breaks down dependent steps into separate console output, and is generally easy to organize imho. All in all a nice write up for folks who might not really like package.json "scripts" to begin with, or for those who'd rather not gain more granular insight into how "scripts" works, but I don't see this being the nail in the coffin case against them.
- crate_barre 5y ago
  What in god’s name is pnpm?
  - andrew_ 5y ago
    https://pnpm.io/ https://pnpm.io/ it ships with Node.js along side npm and yarn these days, has for about a year I believe.
    - Guillaume86 5y ago
      What's your source on pnpm/yarn shipping with node? It doesn't in my experience.
      - p8123 5y ago
        `corepack enable` https://nodejs.org/api/corepack.html https://nodejs.org/api/corepack.html
Steltek 5y ago
This seems to be missing the obvious point: Make isn't anchored to any single language. If you will only ever use npm for your entire life, then you go ahead an live happily in package.json script land. But I've lost track of the number of languages and environments that I've worked in. Make ties it together by being the Good Enough anchor point that documents and launches all the other tools and compilers. - Compile this Rust app with Cargo then flash it to the esp32? Make. - Build this page with JS using Tool Of The Week and push it to a test container? Make. - Compile ancient C app and build a .deb? Make. And so on. Why not use shell scripts you may ask? Because shell scripts are way too free form and your tastes will change over time. Make forces just enough structure that you won't get carried away with yourself. Not for the basic task running stuff anyway.
- dongping 5y ago
  Bazel would be a better solution in terms of reproducibility and user-friendliness, though.
  - nuccy 5y ago
    I agree, but make is usually more abundant on much more systems (which is useful when there is no root privileges available). Also people are used to run make when they encounter a Makefile. I personally use Makefiles even to create Docker images, I find it simpler to run make images or make run, than remembering (or putting in a script and remember what parameters to use, as pointed in the comment above scripts are too free-style) how to do it manually. Though I also agree that Makefile is not a silver bullet and some more complex/niche methods may be required in particular cases.
  - physicsguy 5y ago
    It's just a nightmare to manage unless you're a Google-sized company
    - klodolph 5y ago
      I've been working with Bazel by myself. My experience is that it is not a nightmare... it's just the documentation is missing some pretty crucial "how-to" guides, and there are a couple features that changed a lot prior to 1.0 (so using them is a bit difficult). Some stuff that is very easy with makefiles is a bit harder with Bazel, so I can't give it an unqualified recommendation. The payoff is that some things that are very, very hard with makefiles are much easier with Bazel (using multiple languages, cross-compiling, downloading dependencies automatically, distributed builds, etc).
JohnHaugeland 5y ago
The second I see a node project built in make, I remove it That's a person who doesn't understand portability, tooling, or the need to stick to community norms. Everything about their library is going to be pure pain.
synergy20 5y ago
I use makefile for all my builds. cmake can help portability across OSes but it has a layer of its own complexity, I mostly work on linux alone, makefile seems more than enough. google etc produces new tools to build its huge code base but I don't have that large scale source code, makefile so far worked well for my scale. unless you have specific needs I feel makefile will do just fine. it's simple, readable, manageable, and get the job done fast.
aitchnyu 5y ago
I was amazed to discover Taskfile. Didn't realise it was a copy of Make. https://github.com/adriancooney/Taskfile https://github.com/adriancooney/Taskfile
SkyPuncher 5y ago
I don't have a problem with make files, but I do prefer to work directly with `package.json` when reasonable. Most notably, most of the IDE's work very well with `package.json`, but don't always handle make as cleanly.
btbuildem 5y ago
Make has been around for decades, and with good reason. I've enjoyed watching the grunts and gulps and whatevers appear as the New Shiny Thing, become complicated and collapse under their own weight as people struggle with overly complex configs, as I quietly do the thing that has always worked and has not changed in forever.
tejohnso 5y ago
Pretty convincing writeup. I've only used makefiles when compiling C / C++ but lately I'm finding even those are moving to CMAKE with makefiles becoming less and less common.
k__ 5y ago
A Makefile inside a JavaScript codebase is a good indicator that you will encounter other less idiomatic ways of doing things in that codebase.
- cxr 5y ago
  That's a bonus. Node and NPM parted ways with idiomatic JS a long time ago. Most packages in the Node ecosystem are pretty much, "What if a bunch of people who clearly hate JS nonetheless decided to spend most of their time and energy inside files ending in .js (or .ts)?" So to the extent that "idiomatic ways of doing things" really means "idiomatic wrt the Node ecosystem" and not "idiomatic to JS as it was meant to be written", that sounds like a win. <https://news.ycombinator.com/item?id=24500593 https://news.ycombinator.com/item?id=24500593>
rbongers 5y ago
The problem I have with Make is that it has a syntax that most web developers I have worked with would find archaic and unfamiliar. On some projects I've worked on, it's been used as basically a command alias system that only a few people can maintain. None of its caching or dependency chain capabilities were utilized. In these cases, shell scripts would have been a better option, and in some cases were later introduced with success.
davistreybig 5y ago
Earthly is an interesting open source project in this space that offers many of the benefits of a make file plus containerization plus some of the performance benefits of a Bazel/Pants. https://earthly.dev/ https://earthly.dev/
- benatkin 5y ago
  Believe it or not, it isn't an open source project. https://github.com/earthly/earthly/blob/main/LICENSE https://github.com/earthly/earthly/blob/main/LICENSE
  - adamgordonbell 5y ago
    It's BSL which is source available with the one restriction being you can't build a commercial competitor. And the source becomes MPL2 after 3 years. https://earthly.dev/bslfaq https://earthly.dev/bslfaq
    - gkbrk 5y ago
      What you said does not contradict what the parent comment said. You're both in agreement that it is not open source.
the-alchemist 5y ago
For the clojure crowd, there's bb tasks [0]. * parallelism * hooks (before/after each task, etc.) * command-line arg parsing * --help equivalent * bash/zsh/fish autocompletion * you can also just spawn a shell with bb/shell * yes, dependencies * regular programming language (Clojure), with commonly used shell functions like glob() * import code from locations (don't need to copy/paste between Makefiles) * built-in JSON and CSV support, so you can use your app's JSON files right in your build file [0]: https://book.babashka.org/#tasks https://book.babashka.org/#tasks
ravenstine 5y ago
I too have been using Makefiles in both my C++ and Deno projects and have been very happy with the unity it provides.
Mister_Snuggles 5y ago
I use make to run daily/hourly/etc batch processes in the ERP system at work. There's a Python shim to actually run processes in the ERP, plus a shell script to get things ready to run and send success/failure emails, and cron to kick batches off, but other than that it's pure make. In theory I should also be able to pass '-j 4' to make and have it run multiple processes at once, in the correct order with their dependencies properly satisfied, but I haven't actually tried that yet. It's clearly an abuse of a build tool, but it's also a testament to make's incredible flexibility. I've also heard of someone who replaced their Linux startup scripts (in the pre-systemd days) with make to speed up boot time. That's actually what inspired my batch schedule work.
ThePhysicist 5y ago
I'm also a fan of make but whenever another frontend dev starts working with one of my projects the first thing they do is rip out the perfectly good Makefile and replace it with npm scripts (often introducing errors). I guess it's a generational thing.
molszanski 5y ago
Here is Makefile "starter" I use: https://github.com/awinecki/magicfile https://github.com/awinecki/magicfile People call this "self-documenting makefile". It migrated with me from company to company, from project to projects. Through node, php, aws, docker, server management, cert updates, file processing and many many more.
kall 5y ago
One thing I appreciate about the javascript ecosystem and "scripts" is the focus on keeping everything contained in the project. The only assumption about the system is that it has node, from there it's npm i, npm start. Scripts are a way to make sure people always run the project-local version of various development tools and not whatever they happen to have installed. When I see a makefile, I expect they will be making more assumptions about my system and expect a little more work. I do appreciate the "has worked fine for decades" aspect of makefiles, and I guess docker solves some of this, but I still prefer a fully self contained javascript project.
- des429 5y ago
  Yeah this seems to miss some of the biggest features of npm scripts. Access to the project's local packages is probably the biggest feature not mentioned IMO.
  - throwawaycuriou 5y ago
    npx to the rescue
- yurishimo 5y ago
  To add one tiny thing to this, it’s absolutely imperative that your project be setup with safeguards for node version dependencies. I can’t tell you how many projects I’ve joined that don’t even have an nvmrc file, much less some guard to ensure a clean install and build.
- andreineculau 5y ago
  When you say everything contained in the project, you forget that there's more to life than just node/javascript. Just two silly examples: you cannot run/deploy your API backend without docker/kubernetes/aws-cli/etc, nor your mobile app even if it's in JavaScript (react-native). The only way to keep to node/javascript is if you only develop libraries (a website that doesn't get deployed -> still on library level).
  - 5y ago
jsz0 5y ago
Back when I was doing network engineering I used make to localize configs from boilerplate code templates. Not a common use for it or maybe even the best tool for the job but because I already knew Makefiles it was an easy solution for me to implement. Saved me endless hours of tedious error prone text editing. Just an example of why it's worthwhile to learn how to use these fundamental tools that have survived the test of time. Even if you don't have a specific use for it today it's a very useful tool to have in your toolbox.
limonkufu 5y ago
We are using make targets to automate ci/cd, local dev and a lot of other things by adding a set of easy to extend, customise template makefiles as submodules. The main advantage for us is it's same everywhere and it's agnostic to underlying technologies. (we are only supporting unix based environments and have guidelines to setup gnumake in macos). By using this way, we ensure that the general SDLC is same across different repositories and underlying tooling can change anytime without inducing a lot of refactoring We are also using make targets to document themselves. How to use, what are other targets, variables you need to define etc. This makes it a powerful CLI app for us that can run and support many things. For anyone interested: https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile and the similar parsing method for self documenting makefiles: https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blob/master/help.mk https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blo.... We didn't know about magicmake that's linked here that does the similar thing
nwsm 5y ago
package.json scripts are definitely a bit out of place. I think their prevalence stems from popular npm projects using the pattern for easy cross-platform developer experience. I've seen some shell scripts in opensource node projects, but not much makefile, and I actually had an interviewer once joke that makefiles were old and out of style. But as I have worked with less Windows developers over time, my projects have relied more on shell scripts and makefiles. I think the containerization movement has started to shift software development towards Linux-based tooling as well. As a cross-platform effort this is a bit ironic, but having overlapping developer experience and CI/CD tooling is pretty convenient when you're on a Unix kernel or WSL2. However, much of the npm ecosystem and community are focused on frontend work that often does not involve infrastructure-related tooling. Without the absolute need for any OS-specific tools, and with the many composable crossplatform scripts in vogue already, package.json scripts make sense (until they get out of hand).
efortis 5y ago
It’s helpful for mitigating an NPM arbitrary code execution vulnerability along with ‘ignore-scripts=true’ https://blog.uidrafter.com/getting-rid-of-npm-scripts https://blog.uidrafter.com/getting-rid-of-npm-scripts
spion 5y ago
Why I prefer package.json over makefiles: - turborepo (https://turborepo.org/ https://turborepo.org/) can describe dependency pipelines and provide automatic caching. Makefiles aren't designed for multi-input, multi-output scenarios - its really awkward: https://www.gnu.org/software/automake/manual/html_node/Multiple-Outputs.html https://www.gnu.org/software/automake/manual/html_node/Multi... and https://stackoverflow.com/questions/39237306/makefile-compiles-all-tsc-regardless-of-changes https://stackoverflow.com/questions/39237306/makefile-compil... - turborepo can also run never-ending targets in parallel (e.g. API server and static file server in development mode). Not sure how well make supports that - turborepo can depend on env var values. Makefiles can to, but like everything with makefiles, its an awkward workaround: https://stackoverflow.com/questions/14840474/make-target-which-depends-on-an-environment-variable https://stackoverflow.com/questions/14840474/make-target-whi... - makefiles do not work on Windows (They are portable, just not to platforms most node devs care about) - Its unclear whether `make` will ever add features to remove the awkwardness for supporting the various scenarios above. It doesn't seem likely to happen.
- spion 5y ago
  Also, it says something about the design of `scripts` that its so easy to build something like turborepo's dependency pipeline on top of it :)
andreineculau 5y ago
https://github.com/ysoftwareab/yplatform https://github.com/ysoftwareab/yplatform that this a notch higher (disclaimer: author here) As others have mentioned on this thread, the problem is not just "use Makefiles instead of package.json scripts", but quickly ending up with duplicate Makefile content and then supporting more than just one language. It's inevitable. PS for those that don’t know, npm’s “scripts” functionality as we know it today was not “designed”, but it was simply a byproduct. First introduced as an internal functionality for lifecycle support https://github.com/npm/npm/commit/c8e17cef720875c1f7cea1b49b7bc9f1325ccff5 https://github.com/npm/npm/commit/c8e17cef720875c1f7cea1b49b... on 1 Mar 2010 and later extend to run arbitrary targets https://github.com/npm/npm/commit/9b8c0cf87a9d4ef560e17e060f4ddc03b2ff1a1c https://github.com/npm/npm/commit/9b8c0cf87a9d4ef560e17e060f... on 13 Dec 2010 This is the entire design backstory https://github.com/npm/npm/issues/298 https://github.com/npm/npm/issues/298 . Needless to say that GNU Make’s equivalent (or any other build system) is not to be found in npm’s scripts.
mc4ndr3 5y ago
(GNU) Make's advantages include ease of use, a modicum of portability, and support for a variety of different software development command line stacks. Unfortunately, the shell commands that are typically setup in a Makefile, are rather unportable. They waste a touch of time and space. They can break in surprising ways. For this reason, I try to write my build tasks in terms of the same programming language as the application language. For example, Grunt for JavaScript projects, Shake for Haskell projects, Gradle for Groovy projects, Rez for C/C++ projects, Dale for D projects, TinyRick for Rust projects, and Mage for Go projects. Leiningen for Clojure projects, SBT for Scala projects. Vast for shell projects. Make, like Python, suffers from the appearance of simplicity, at the cost of long term maintainability. Ideally both the build system and the main application are compiled. That doesn't have to be a cumbersome process, but rather a guide to identify potential bugs sooner during development. Make is my first choice for random projects, but only when better build systems are unavailable.
yboris 5y ago
Just wanted to share nps as a nice way to manage scripts for your package.json https://www.npmjs.com/package/nps https://www.npmjs.com/package/nps You can create a dedicated JS file (with comments and more!) that will house your scripts, which you can run by using "nps" instead of "npm" as a command.
nsonha 5y ago
At one point I tried to do the same thing but then I realized there isn't any make feature I need to use. The key thing about make is to avoid building already built artifacts, if you don't need that you end up with things like .PHONY. There is'nt anything in make that helps with composing or showing a list of scripts etc either. At the end I ended up with plain javascript scripts for js project and plain bash scripts for everything else.
changhanlee 5y ago
The argument about variables in makefiles was convincing to me, but looking it up I learned for the first time you can actually use variables in package.json files as well. After all these years mucking about with package.json...

235 points by ducaale 5y ago

Reminds me of something I wrote years ago: https://blog.jgc.org/2010/11/things-make-got-right-and-how-to-make.html https://blog.jgc.org/2010/11/things-make-got-right-and-how-t...

jitl 5y ago
Has anyone Fixed Make in a way you find good and right since you wrote that? (My last few forays into the build systems domain have mostly turned up “Bazel but for X audience” so I can’t recall a satisfying solution)

gjvc 5y ago

I'm pleased to see this being espoused. One thing I always tell junior people is ""don't be tempted to think of make(1) as out-of-date [1], and to view packages available via npm etc as better replacements, as many tasks fit the model of make(1), if not the title "a program for directing recompilation" "" I am often viewed with suspicion until I show them a (much more compact) replacement for their custom python/js program expressed as a makefile without any of the process management requiring debugging. :-) An alternative approach for the "batches of files to process" situation is to generate the commands and feed them to GNU parallel for execution. [1] no pun intended

nicoburns 5y ago
Agreed if you only have to support linux/unix, but the one good thing about the Node ecosystem is that they tend to work on windows too.
- gjvc 5y ago
  not the point, also "WSL"
- verelo 5y ago
  While i agree, I’ve spent literal days fighting with node modules in recent months. Sure make isn’t perfect either but it’s a lot less, confusing? Ie json isn’t a language for humans, but often you really need to read it to correct some issue or bad merge from another contributor.
  - nicoburns 5y ago
    > Sure make isn’t perfect either but it’s a lot less, confusing? I think that's a matter of what you're used to. I don't use make all that often, so I find make files beyond a few lines quite a lot to get my head around, whereas I spend all day writing JavaScript so I can read/write JSON in my sleep.
    - verelo 5y ago
      Maybe. I actually use node much more often, i don't even recall the last time i used a make file in production, i just find parsing json as a human to be rather painful.
- Steltek 5y ago
  Make has worked on Windows longer than Node.js has existed.
  - nicoburns 5y ago
    Make works fine, it's the commands you call from make that don't tend to as compatible. For example, try running rm -rf on windows. It might work if you have GNU tools installed, but it certainly won't work out of the box. On the other hand, the node.js `rimraf` package will do the same thing with no cross-platform compatibility issues.
    - Steltek 5y ago
      When using package.json, does npm intercept a script containing "rm -rf" and translate it to something different?
      - geewee 5y ago
        No, that's why there's a bunch of packages such as rimraf[0] that implements that sort of functionality in a cross-platform way that most people use in their scripts [0]: https://www.npmjs.com/package/rimraf https://www.npmjs.com/package/rimraf
      - nicoburns 5y ago
        No, but it does put binaries from any installed node packages in your PATH (locally when running scripts).
  - jchw 5y ago
    The history of Make on Windows would Make one think twice about relying on it. What shell do you get when running Make on Windows?
- throw0101a 5y ago
  http://gnuwin32.sourceforge.net/packages/make.htm http://gnuwin32.sourceforge.net/packages/make.htm https://community.chocolatey.org/packages/make https://community.chocolatey.org/packages/make
andrew_ 5y ago
Might it not be a disservice to juniors to point them away from deeply understanding the platform they're working with?
- gjvc 5y ago
  Showing someone an alternative approach does not prevent someone "deeply understanding" their current method; it's not zero-sum.
  - gjvc 5y ago
    Might it not be a disservice to juniors to point them away from deeply understanding the platform they're working with? I cannot overstate how annoyed I was by the patronising tone of this comment.
jjice 5y ago
Biggest pain of package.json to me is having to be crammed on one line. Slap in a conditional and now we have a messy thing to read, compared to its multiline counterpart.
Wicher 5y ago
Speaking of debugging, I found this handy: print-%: ; @echo '$(subst ','\'',$*=$($*))' include Makefile Save somewhere, eg as ~/Makefile.debug, then when you want to know the evaluated value of some variable or task in some Makefile, you prefix it with print- — for example, LOCALES: make -f ~/Makefile.debug print-LOCALES It'll print the evaluated LOCALES of the Makefile in the directory you run this in.
- BeefWellington 5y ago
  This is amazing. Thank you for the tip!
- gjvc 5y ago
  Very useful, will give it a shot. Thank you. :-)

Cthulhu_ 5y ago

I've only started using a Makefile ~two years ago when I started a big Go project at work, and I quite like it. There's a few caveats here and there - Makefile specific syntax, shell-script specific syntax, and whether I should put .PHONY in front of every command (I guess 'no unless you have a file with the exact same name'?), but it's quite compact and straightforward. The most complicated command I have is something that removes a folder or set of files, invokes the swagger generator with a list of options, copies it to a target folder, and passes it through a formatter/processor. But it's very straightforward, and it's "just" shell script, not shell script wrapped in a JSON document, or some Java tool invoked indirectly through an XML configuration file like back in the day with Maven.

JamesSwift 5y ago
If using Make as a task runner and not build cache, I just put `.PHONY: %` at the top which means everything is marked `.PHONY`
- nablaone 5y ago
  Thanks!

silves89 5y ago

I have very similar use-cases, and I also I found makefiles a bit limiting. I wrote lk[1] to make this sort of thing easier, and you can just write plain bash functions instead of makefiles. [1] https://github.com/jamescoleuk/lk https://github.com/jamescoleuk/lk

jitl 5y ago

I like Make, and use it in personal JavaScript ecosystem projects that do actually need to build things with interdependency (eg https://github.com/justjake/quickjs-emscripten/blob/master/Makefile https://github.com/justjake/quickjs-emscripten/blob/master/M...). I’ve also seen Makefiles grown to be horrendous monstrosities masquerading as command-line tools; for about a year at Airbnb we used a 1000+ line Makefile as the main tool for fiddling with Kubernetes cuz one of our senior engineers didn’t like Ruby, and I’ve seen another one get close to that level of cravenness. What I learned from supporting Make and shell among a few different audiences is that most developers have no interest in how to write or maintain shell-like tooling. They forget or mess up quoting rules constantly, and eschew learning things and good design in these tools to a much greater degree than in their “normal” work in Java/Python/Ruby/Typescript/Golang. For every POSIXly Correct HN Commenter (of which I count myself a member), there’s 100 regular software engineers who won’t read a `man` page on what $@ or $< mean in Make. I know that if I start writing a Makefile for $JOB, it’s gonna be me and that one guy who uses tcsh who are gonna maintain it and answer questions. (Although for what it’s worth we don’t use package.json scripts either thank goodness. All our complicated build steps are typescript commands, and our glue is CI system YAML files.)

unqueued 5y ago
I would try to use JavaScript tooling in JavaScript projects, especially if I share it with other developers. I would love to see better handling of filename transformation, file watching, and parallelization. A few times over the years I had given up and just used Make instead of Gulp or Grunt. Most recently, I wanted to watch if files had been deleted (if directory entries had been modified). It ended up being a few lines of Makefile. And once you understand the syntax, it is expressive and simple, and way more portable. There was some good discussion awhile ago on this[1]: [1]: https://news.ycombinator.com/item?id=7622296 https://news.ycombinator.com/item?id=7622296

dncornholio 5y ago

But you don't write scripts in package.json and it also failed to show an example how the javascript dependencies could be loaded with make. I think make works well in combination with package.json, but not as a replacement. You can also do the same stuff in an NPM command (which just loads node code) that you could do in a makefile script.

andrew_ 5y ago
Not to mention that the top three package managers will load binary files from node_modules automagically by name only, without having to specify paths. That gets hairy in a monorepo, and package managers running package.json scripts manage them just fine.
- silverwind 5y ago
  `npx` solves this elegantly as long as you `cd` to the project directory first.

BeefWellington 5y ago

Over here in crazytown I'm using Makefiles to automate system deployments. I thought it was insane at first but actually I've come around to it; Make hits that sweet spot of being ubiquitous, simple, and flexible. There's a little bit of a learning curve but once you're over that initial hurdle it's pretty straightforward, especially for one-way operations, e.g. install without uninstall, build without clean, etc.

boondaburrah 5y ago

For a lot of the same reasons in this article I like to use tup[0] when I can. It doesn't integrate into anything which is both good and bad. I wish my IDE could check with tup to see what dependencies get pulled in by what files. However, it's nice that it doesn't care what language or ecosystem I'm using. Also, it's very strict about declaring dependencies properly, and will actually fail the build if you've set things up in a way that depends on something not tracked (by watching filesystem access as the compiler runs). That gives me warm fuzzies that my builds are reproducible. Also I can get a neat dependency graph as a PNG if I want. [0] https://gittup.org/tup/ https://gittup.org/tup/

deleted 5y ago

[deleted]

devn0ll 5y ago

May I also direct your attention to self-documenting Makefiles: https://www.thapaliya.com/en/writings/well-documented-makefiles/ https://www.thapaliya.com/en/writings/well-documented-makefi...

molszanski 5y ago
Here is the one I use and recommend: https://github.com/awinecki/magicfile https://github.com/awinecki/magicfile

mro_name 5y ago

one of the few annoyances is spaces in paths. Just avoid them anyway, right?

beej71 5y ago
This is one of the few fixes I'd really like to see. I'm from a Unix background so I don't tend to have spaces anywhere, but it's still a splinter that needs removing.

nicoburns 5y ago

I would like to call attention to `just` (https://github.com/casey/just https://github.com/casey/just) which is a modern take on "make if we didn't have to keep it backwards compatible", and more focussed on just task running rather than tracking dependencies.

marcuskaz 5y ago
Yes! This is what a modern task runner should be. Includes ability to list recipes, and a consistent format between multiple platforms. Just two things Make doesn't have.
michidk 5y ago
I was just going to say that. Just is awesome and I use it it most of my projects.
enriquto 5y ago
There's no -j option nor mention of parallelism on the docs. Does it run independent targets in parallel? This is easily the main feature of make. I don't understand how it can work in parallel if targets are not somehow associated to files. What's the advantage to a shell script with separate sub-commands, then?
- q3k 5y ago
  ‘just’ apparently isn’t a build system, just a ‘task runner’. So I assume it doesn’t need/want to implement parallelism.
  - dahfizz 5y ago
    It sounds like `just` is not a replacement for make, then. It seems like it could be a nice wrapper on top, i.e. `just build` could invoke the make build system.
  - toast0 5y ago
    Parallelism is pretty useful for just a 'task runner' too. I've (ab)used Make for server pushes, and sometimes you want those to run one at a time, and sometimes you want -j 100 (sometimes -j 100 is even effective).

WesolyKubeczek 5y ago

Why not both? Use make for dependency graph, but also use "npm run" or "yarn run" to make sure you're running with the correct Node and correct paths to all your tools. I know you can specify those in your Makefile too, but the plus of running via npm/yarn is that they know where your binaries live, where your mode_nodules are, and that sort of thing.

andrew_ 5y ago
replace that with `pnpm` and you've got a winner. npm and yarn are old hat (unless you're working with react native, which is another discussion)
JamesSwift 5y ago
Yes, I usually treat Make as the user-friendly facade to the underlying framework tooling (which might be e.g. Rails, Docker, or npm). `make bootstrap` gets the project setup. `make up` starts the project. For npm projects these usually just wrap package.json stuff. But I don't need to remember what tech a project is in, or where to look for the commands, if Make is the entry point.

karmicthreat 5y ago

Out in Grand Rapids, Mi where AO is located there seems to be a bit of a tradition of abusing make in a good way. It's where I picked up the habit as well. I think its just institutional knowledge/tradition that has creeped through the various firms here. I kind of wonder what other "traditions" get passed around on a regional level?

sigzero 5y ago

Task is nice: https://taskfile.dev/#/ https://taskfile.dev/#/

mark_l_watson 5y ago

I also use Makefiles, been using them for 40 years. I find that Makefile targets for misc. things like grabbing remote training data, running tests, building documentation, etc., etc. augments information in READ files, that is in addition to functionally saving time, serves as documentation when I haven’t looked at a project in 6 months.

deepsun 5y ago

What are the benefits of Makefiles compared to a bunch of shell scripts?

enriquto 5y ago
That you get parallelism and partial re-running for free.
- andrew_ 5y ago
  parallelism is common in package managers these days https://pnpm.io/cli/run#--parallel https://pnpm.io/cli/run#--parallel
  - monocasa 5y ago
    No, make understands the directed acyclic graphs of dependencies, regardless of language, and will walk that graph in parallel. No forcing or ignoring sorting, and no enforced package level blocking on scripts.
    - immibis 5y ago
      It's a shame this doesn't work for Java in particular (not JS) as Java's coding conventions/requirements do not lend themselves at all to manually updating a list of file dependencies.
JamesSwift 5y ago
A single, system-and-language-agnostic entry point to the actions you want to run with very clear dependency rules built in.
- deepsun 5y ago
  Besides Windows, shell scripts are system-and-language agnostic (not using bash/zsh-specific syntax of course). Also I'm not sure Windows can run Makefiles out-of-the-box.

andrew_ 5y ago

Stick around in any programming ecosystem long enough, and you live long enough to see strategy, taste, and "discovery" come full circle. We had make, then package.json "scripts," then grunt, then gulp, then the trend shifted back towards package.json "scripts." (Note: Using "scripts" to differentiate the package.json property versus the generic word use) I like that the author isn't being authoritative, but there could be some additional due diligence. I ran the make-is-faster loose benchmark via PNPM and runtime was 0m0.022s for make and 0m0.012s for PNPM. If I care about those 10ms, PNPM is my horse. Yarn is a glacier compared to PNPM. Another thing I would've liked to see comment on is the automatic pre and post paradigm that package.json "scripts" affords. The big three package managers all support pre and post, and it makes arranging "scripts" a breeze, it breaks down dependent steps into separate console output, and is generally easy to organize imho. All in all a nice write up for folks who might not really like package.json "scripts" to begin with, or for those who'd rather not gain more granular insight into how "scripts" works, but I don't see this being the nail in the coffin case against them.

crate_barre 5y ago
What in god’s name is pnpm?
- andrew_ 5y ago
  https://pnpm.io/ https://pnpm.io/ it ships with Node.js along side npm and yarn these days, has for about a year I believe.
  - Guillaume86 5y ago
    What's your source on pnpm/yarn shipping with node? It doesn't in my experience.
    - p8123 5y ago
      `corepack enable` https://nodejs.org/api/corepack.html https://nodejs.org/api/corepack.html

Steltek 5y ago

This seems to be missing the obvious point: Make isn't anchored to any single language. If you will only ever use npm for your entire life, then you go ahead an live happily in package.json script land. But I've lost track of the number of languages and environments that I've worked in. Make ties it together by being the Good Enough anchor point that documents and launches all the other tools and compilers. - Compile this Rust app with Cargo then flash it to the esp32? Make. - Build this page with JS using Tool Of The Week and push it to a test container? Make. - Compile ancient C app and build a .deb? Make. And so on. Why not use shell scripts you may ask? Because shell scripts are way too free form and your tastes will change over time. Make forces just enough structure that you won't get carried away with yourself. Not for the basic task running stuff anyway.

dongping 5y ago
Bazel would be a better solution in terms of reproducibility and user-friendliness, though.
- nuccy 5y ago
  I agree, but make is usually more abundant on much more systems (which is useful when there is no root privileges available). Also people are used to run make when they encounter a Makefile. I personally use Makefiles even to create Docker images, I find it simpler to run make images or make run, than remembering (or putting in a script and remember what parameters to use, as pointed in the comment above scripts are too free-style) how to do it manually. Though I also agree that Makefile is not a silver bullet and some more complex/niche methods may be required in particular cases.
- physicsguy 5y ago
  It's just a nightmare to manage unless you're a Google-sized company
  - klodolph 5y ago
    I've been working with Bazel by myself. My experience is that it is not a nightmare... it's just the documentation is missing some pretty crucial "how-to" guides, and there are a couple features that changed a lot prior to 1.0 (so using them is a bit difficult). Some stuff that is very easy with makefiles is a bit harder with Bazel, so I can't give it an unqualified recommendation. The payoff is that some things that are very, very hard with makefiles are much easier with Bazel (using multiple languages, cross-compiling, downloading dependencies automatically, distributed builds, etc).

JohnHaugeland 5y ago

The second I see a node project built in make, I remove it That's a person who doesn't understand portability, tooling, or the need to stick to community norms. Everything about their library is going to be pure pain.

synergy20 5y ago

I use makefile for all my builds. cmake can help portability across OSes but it has a layer of its own complexity, I mostly work on linux alone, makefile seems more than enough. google etc produces new tools to build its huge code base but I don't have that large scale source code, makefile so far worked well for my scale. unless you have specific needs I feel makefile will do just fine. it's simple, readable, manageable, and get the job done fast.

aitchnyu 5y ago

I was amazed to discover Taskfile. Didn't realise it was a copy of Make. https://github.com/adriancooney/Taskfile https://github.com/adriancooney/Taskfile

SkyPuncher 5y ago

I don't have a problem with make files, but I do prefer to work directly with `package.json` when reasonable. Most notably, most of the IDE's work very well with `package.json`, but don't always handle make as cleanly.

btbuildem 5y ago

Make has been around for decades, and with good reason. I've enjoyed watching the grunts and gulps and whatevers appear as the New Shiny Thing, become complicated and collapse under their own weight as people struggle with overly complex configs, as I quietly do the thing that has always worked and has not changed in forever.

tejohnso 5y ago

Pretty convincing writeup. I've only used makefiles when compiling C / C++ but lately I'm finding even those are moving to CMAKE with makefiles becoming less and less common.

k__ 5y ago

A Makefile inside a JavaScript codebase is a good indicator that you will encounter other less idiomatic ways of doing things in that codebase.

cxr 5y ago
That's a bonus. Node and NPM parted ways with idiomatic JS a long time ago. Most packages in the Node ecosystem are pretty much, "What if a bunch of people who clearly hate JS nonetheless decided to spend most of their time and energy inside files ending in .js (or .ts)?" So to the extent that "idiomatic ways of doing things" really means "idiomatic wrt the Node ecosystem" and not "idiomatic to JS as it was meant to be written", that sounds like a win. <https://news.ycombinator.com/item?id=24500593 https://news.ycombinator.com/item?id=24500593>

rbongers 5y ago

The problem I have with Make is that it has a syntax that most web developers I have worked with would find archaic and unfamiliar. On some projects I've worked on, it's been used as basically a command alias system that only a few people can maintain. None of its caching or dependency chain capabilities were utilized. In these cases, shell scripts would have been a better option, and in some cases were later introduced with success.

davistreybig 5y ago

Earthly is an interesting open source project in this space that offers many of the benefits of a make file plus containerization plus some of the performance benefits of a Bazel/Pants. https://earthly.dev/ https://earthly.dev/

benatkin 5y ago
Believe it or not, it isn't an open source project. https://github.com/earthly/earthly/blob/main/LICENSE https://github.com/earthly/earthly/blob/main/LICENSE
- adamgordonbell 5y ago
  It's BSL which is source available with the one restriction being you can't build a commercial competitor. And the source becomes MPL2 after 3 years. https://earthly.dev/bslfaq https://earthly.dev/bslfaq
  - gkbrk 5y ago
    What you said does not contradict what the parent comment said. You're both in agreement that it is not open source.

the-alchemist 5y ago

For the clojure crowd, there's bb tasks [0]. * parallelism * hooks (before/after each task, etc.) * command-line arg parsing * --help equivalent * bash/zsh/fish autocompletion * you can also just spawn a shell with bb/shell * yes, dependencies * regular programming language (Clojure), with commonly used shell functions like glob() * import code from locations (don't need to copy/paste between Makefiles) * built-in JSON and CSV support, so you can use your app's JSON files right in your build file [0]: https://book.babashka.org/#tasks https://book.babashka.org/#tasks

ravenstine 5y ago

I too have been using Makefiles in both my C++ and Deno projects and have been very happy with the unity it provides.

Mister_Snuggles 5y ago

I use make to run daily/hourly/etc batch processes in the ERP system at work. There's a Python shim to actually run processes in the ERP, plus a shell script to get things ready to run and send success/failure emails, and cron to kick batches off, but other than that it's pure make. In theory I should also be able to pass '-j 4' to make and have it run multiple processes at once, in the correct order with their dependencies properly satisfied, but I haven't actually tried that yet. It's clearly an abuse of a build tool, but it's also a testament to make's incredible flexibility. I've also heard of someone who replaced their Linux startup scripts (in the pre-systemd days) with make to speed up boot time. That's actually what inspired my batch schedule work.

ThePhysicist 5y ago

I'm also a fan of make but whenever another frontend dev starts working with one of my projects the first thing they do is rip out the perfectly good Makefile and replace it with npm scripts (often introducing errors). I guess it's a generational thing.

molszanski 5y ago

Here is Makefile "starter" I use: https://github.com/awinecki/magicfile https://github.com/awinecki/magicfile People call this "self-documenting makefile". It migrated with me from company to company, from project to projects. Through node, php, aws, docker, server management, cert updates, file processing and many many more.

kall 5y ago

One thing I appreciate about the javascript ecosystem and "scripts" is the focus on keeping everything contained in the project. The only assumption about the system is that it has node, from there it's npm i, npm start. Scripts are a way to make sure people always run the project-local version of various development tools and not whatever they happen to have installed. When I see a makefile, I expect they will be making more assumptions about my system and expect a little more work. I do appreciate the "has worked fine for decades" aspect of makefiles, and I guess docker solves some of this, but I still prefer a fully self contained javascript project.

des429 5y ago
Yeah this seems to miss some of the biggest features of npm scripts. Access to the project's local packages is probably the biggest feature not mentioned IMO.
- throwawaycuriou 5y ago
  npx to the rescue
yurishimo 5y ago
To add one tiny thing to this, it’s absolutely imperative that your project be setup with safeguards for node version dependencies. I can’t tell you how many projects I’ve joined that don’t even have an nvmrc file, much less some guard to ensure a clean install and build.
andreineculau 5y ago
When you say everything contained in the project, you forget that there's more to life than just node/javascript. Just two silly examples: you cannot run/deploy your API backend without docker/kubernetes/aws-cli/etc, nor your mobile app even if it's in JavaScript (react-native). The only way to keep to node/javascript is if you only develop libraries (a website that doesn't get deployed -> still on library level).
- 5y ago

jsz0 5y ago

Back when I was doing network engineering I used make to localize configs from boilerplate code templates. Not a common use for it or maybe even the best tool for the job but because I already knew Makefiles it was an easy solution for me to implement. Saved me endless hours of tedious error prone text editing. Just an example of why it's worthwhile to learn how to use these fundamental tools that have survived the test of time. Even if you don't have a specific use for it today it's a very useful tool to have in your toolbox.

limonkufu 5y ago

We are using make targets to automate ci/cd, local dev and a lot of other things by adding a set of easy to extend, customise template makefiles as submodules. The main advantage for us is it's same everywhere and it's agnostic to underlying technologies. (we are only supporting unix based environments and have guidelines to setup gnumake in macos). By using this way, we ensure that the general SDLC is same across different repositories and underlying tooling can change anytime without inducing a lot of refactoring We are also using make targets to document themselves. How to use, what are other targets, variables you need to define etc. This makes it a powerful CLI app for us that can run and support many things. For anyone interested: https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile and the similar parsing method for self documenting makefiles: https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blob/master/help.mk https://gitlab.com/ska-telescope/sdi/ska-cicd-makefile/-/blo.... We didn't know about magicmake that's linked here that does the similar thing

nwsm 5y ago

package.json scripts are definitely a bit out of place. I think their prevalence stems from popular npm projects using the pattern for easy cross-platform developer experience. I've seen some shell scripts in opensource node projects, but not much makefile, and I actually had an interviewer once joke that makefiles were old and out of style. But as I have worked with less Windows developers over time, my projects have relied more on shell scripts and makefiles. I think the containerization movement has started to shift software development towards Linux-based tooling as well. As a cross-platform effort this is a bit ironic, but having overlapping developer experience and CI/CD tooling is pretty convenient when you're on a Unix kernel or WSL2. However, much of the npm ecosystem and community are focused on frontend work that often does not involve infrastructure-related tooling. Without the absolute need for any OS-specific tools, and with the many composable crossplatform scripts in vogue already, package.json scripts make sense (until they get out of hand).

efortis 5y ago

It’s helpful for mitigating an NPM arbitrary code execution vulnerability along with ‘ignore-scripts=true’ https://blog.uidrafter.com/getting-rid-of-npm-scripts https://blog.uidrafter.com/getting-rid-of-npm-scripts

spion 5y ago

Why I prefer package.json over makefiles: - turborepo (https://turborepo.org/ https://turborepo.org/) can describe dependency pipelines and provide automatic caching. Makefiles aren't designed for multi-input, multi-output scenarios - its really awkward: https://www.gnu.org/software/automake/manual/html_node/Multiple-Outputs.html https://www.gnu.org/software/automake/manual/html_node/Multi... and https://stackoverflow.com/questions/39237306/makefile-compiles-all-tsc-regardless-of-changes https://stackoverflow.com/questions/39237306/makefile-compil... - turborepo can also run never-ending targets in parallel (e.g. API server and static file server in development mode). Not sure how well make supports that - turborepo can depend on env var values. Makefiles can to, but like everything with makefiles, its an awkward workaround: https://stackoverflow.com/questions/14840474/make-target-which-depends-on-an-environment-variable https://stackoverflow.com/questions/14840474/make-target-whi... - makefiles do not work on Windows (They are portable, just not to platforms most node devs care about) - Its unclear whether `make` will ever add features to remove the awkwardness for supporting the various scenarios above. It doesn't seem likely to happen.

spion 5y ago
Also, it says something about the design of `scripts` that its so easy to build something like turborepo's dependency pipeline on top of it :)

andreineculau 5y ago

https://github.com/ysoftwareab/yplatform https://github.com/ysoftwareab/yplatform that this a notch higher (disclaimer: author here) As others have mentioned on this thread, the problem is not just "use Makefiles instead of package.json scripts", but quickly ending up with duplicate Makefile content and then supporting more than just one language. It's inevitable. PS for those that don’t know, npm’s “scripts” functionality as we know it today was not “designed”, but it was simply a byproduct. First introduced as an internal functionality for lifecycle support https://github.com/npm/npm/commit/c8e17cef720875c1f7cea1b49b7bc9f1325ccff5 https://github.com/npm/npm/commit/c8e17cef720875c1f7cea1b49b... on 1 Mar 2010 and later extend to run arbitrary targets https://github.com/npm/npm/commit/9b8c0cf87a9d4ef560e17e060f4ddc03b2ff1a1c https://github.com/npm/npm/commit/9b8c0cf87a9d4ef560e17e060f... on 13 Dec 2010 This is the entire design backstory https://github.com/npm/npm/issues/298 https://github.com/npm/npm/issues/298 . Needless to say that GNU Make’s equivalent (or any other build system) is not to be found in npm’s scripts.

mc4ndr3 5y ago

(GNU) Make's advantages include ease of use, a modicum of portability, and support for a variety of different software development command line stacks. Unfortunately, the shell commands that are typically setup in a Makefile, are rather unportable. They waste a touch of time and space. They can break in surprising ways. For this reason, I try to write my build tasks in terms of the same programming language as the application language. For example, Grunt for JavaScript projects, Shake for Haskell projects, Gradle for Groovy projects, Rez for C/C++ projects, Dale for D projects, TinyRick for Rust projects, and Mage for Go projects. Leiningen for Clojure projects, SBT for Scala projects. Vast for shell projects. Make, like Python, suffers from the appearance of simplicity, at the cost of long term maintainability. Ideally both the build system and the main application are compiled. That doesn't have to be a cumbersome process, but rather a guide to identify potential bugs sooner during development. Make is my first choice for random projects, but only when better build systems are unavailable.

yboris 5y ago

Just wanted to share nps as a nice way to manage scripts for your package.json https://www.npmjs.com/package/nps https://www.npmjs.com/package/nps You can create a dedicated JS file (with comments and more!) that will house your scripts, which you can run by using "nps" instead of "npm" as a command.

nsonha 5y ago

At one point I tried to do the same thing but then I realized there isn't any make feature I need to use. The key thing about make is to avoid building already built artifacts, if you don't need that you end up with things like .PHONY. There is'nt anything in make that helps with composing or showing a list of scripts etc either. At the end I ended up with plain javascript scripts for js project and plain bash scripts for everything else.

changhanlee 5y ago

The argument about variables in makefiles was convincing to me, but looking it up I learned for the first time you can actually use variables in package.json files as well. After all these years mucking about with package.json...