12 ms·
Time for Makefiles to Make a Comeback
- jolson88 9y agoAuthor here. Thanks for the great comments everybody. I'm honored this post even got attention here (_long_ time reader and lurker here). I'm glad to see a bunch of constructive feedback here that helps me clear up my ignorance around make since I'm lacking in experience compared to other folks (okay, really, I'm a Make noob).
- jph 9y agoFor a better "make" try "do" (also known as "DJB redo"). It's designed by Dan Bernstein of crypto fame and implemented by Avery Pennarun now at Google. http://apenwarr.ca/log/?m=201012#14 http://apenwarr.ca/log/?m=201012#14 I will contribute $100 to any Rust leader who wants to start coding "do" in Rust.
- mitch9 9y agoThis seems like a prime moment to ask for a review of https://github.com/sagebind/rote https://github.com/sagebind/rote
- jph 9y agoRote is a great accomplishment. IMHO it's superior to Make, Rake, Cake, etc. Thank you for writing this! If you're curious, take a look at Do. It's like all of the above, yet flipped over on their heads. The value of Do is in the design, which is essentially the opposite of Make. Do leverages the composability of items, files, scripts, artifacts, and typical Unix pipes. I would be very interested in your opinions about Do, and if you want the $100 to jumpstart working on Do in Rust, let me know how to donate to you.
- zimpenfish 9y ago+1 for do/redo. Although it can be a bugger to get right for go (because there's no intermediate files) (although I think I've solved that now.)
- sounds 9y agoI looked at your profile, but didn't see any clues. Please share? How do you use redo for go projects?
- zimpenfish 9y ago[looks at pending blog posts guiltily] https://bitbucket.org/rjp/can-x-win/src https://bitbucket.org/rjp/can-x-win/src is a good example - basically, apart from `all.do` containing the targets, I have a `default.do` which looks for a matching `*.od` which has the DEPS and then passes those to `go build`.
- JdeBP 9y agoThat's somewhat misleading. * http://jdebp.eu./FGA/introduction-to-redo.html http://jdebp.eu./FGA/introduction-to-redo.html * https://news.ycombinator.com/item?id=15060193 https://news.ycombinator.com/item?id=15060193
- AndrewCHM 9y agoI'm confused So its agreeable that the building pipeline of NPM calling some packer does replace any-other-language calling any-other-packer, but what is the point in replacing the role of `npm build`, with a makefile that just calls what npm would of called anyways? replacing make with npm with make-calling-npm?
- vikiomega9 9y agoI haven't done something similar with npm but I did write make files to call specific commands. For example a particularly long activate function very specific to internal tooling. Doing this makes sense for such commands and not for primitives.
- ealexhudson 9y agoI wrote a bit about an approach I'm using, A Touch Of Make (ATOM), which helps provide better developer UX across teams - particularly ones working on microservices. https://www.alexhudson.com/2017/04/26/articulating-atom-approach/ https://www.alexhudson.com/2017/04/26/articulating-atom-appr... You're right, replacing "npm build" with "make build" doesn't win you anything. But that's only true on the small scale. In a service world, there are lots of projects, each with different requirements. Some will have front-end, some won't. Others will require a totally different build process. There will probably be different languages involved. Using make, you can standardise a lot of this. If you set up a coding standard that after you clone a repo, "make dep" should grab anything the project requires, then developers don't initially need to know whether that's calling out to npm or composer or pip or whatever - it's just "working". This is a much bigger win when you have a number of projects. The process is standardised, so developers know it's only a two or three step build (or whatever you've setup). They know how to do it, and when they need to look under the covers, they can see how it works, and this knowledge is transferable from one project to another because Make is universal. I don't advocate doing big Makefiles - that's why I call this approach ATOM; only use a touch of Make. But used judiciously, it smooths out projects very nicely. Not everyone needs that, though.
- everheardofc 9y ago
- Yaggo 9y agoI used to be big fan of using Makefile for web development, but have since changed my mind, because make is not very good fit for watch-mode incremental building.
- vortico 9y agoInterestingly enough, I tried using Makefiles for a web project yesterday, and it was a success. SOURCES = index.pug layout.pug style.styl main.coffee privacy.md OBJECTS = index.html style.css main.js privacy.html all: $(OBJECTS) watch: http-server & while true; do \ $(MAKE); \ inotifywait -qq $(SOURCES); \ done %.html: %.pug pug -P $^ %.css: %.styl stylus $^ %.js: %.coffee coffee -c $^ %.html: %.md markdown $^ > $@ clean: rm -rfv $(OBJECTS) This could be extended to produce an out-of-source build by using `-o "$(PUBLIC)"` on the command line generators, but I didn't need it for this project.
- jgrahamc 9y agoDo you know about the ; style for rules that have only one line of commands? It helps reduce the number of tabs. %.html: %.pug ; pug -P $^ I find it super useful for compact Makefiles: %.html: %.pug ; pug -P $^ %.css: %.styl ; stylus $^ %.js: %.coffee ; coffee -c $^ %.html: %.md ; markdown $^ > $@ clean: ; rm -rfv $(OBJECTS) Also, worth making clean .PHONY since it's not a file.
- Yaggo 9y agoSure, that's the traditional way of doing things, and I like its simplicity and the fact that the 40 years old core concept is still so useful. But most of us want to require() modules also in the browser, so you'll anyway need browserify or other bundler, which already have cross-platform watch mode & plugin ecosystem, and are expected to be run continously, and suddenly the Makefile feels rather redundant, especially with npm scripting.
- GlennS 9y agoI'd recommend adding: .PHONY: watch .PHONY: clean In the appropriate places.
- verytrivial 9y agoFriends don't let friends write recursive makefiles.
- marcv81 9y agoFriends don't let friends write Makefiles at all. The author does not appear to be familiar with them, except for the trivial examples in this post.
- Pica_soO 9y agoEvery folder gets its own little makefile. Calling those recursive shouldnt result in trouble?
- zwp 9y agoYou might like "Recursive Make Considered Harmful", Miller. http://aegis.sourceforge.net/auug97.pdf http://aegis.sourceforge.net/auug97.pdf Although I suspect the main complaint (slow, due to checking directories where nothing has changed) has mostly been swept under the rug by increased CPU speeds some of the other items (managing dependencies, ordering, ...) are still worth considering today.
- jhasse 9y agoTup solves this: http://gittup.org/tup/ http://gittup.org/tup/
- dangerbird2 9y agoThe types of trivial examples in the post are exactly what you should use makefiles for. make excels at managing simple file transformations: you get into trouble when you try to do complex cross-platform scripting with it.
- dangerbird2 9y agoIt seems like each of the folders with makes files having Makefiles also have a package.json, suggesting they are self-contained packages under the same repository. Still, it seems like a questionable move
- arca_vorago 9y agoI just got done watching a video called "make for reproducibility in science", which made a lot of sense. The impressio I get is people do weird things to make and then hit edge cases and complain.
- jswrenn 9y ago> The impression I get is people do weird things to make and then hit edge cases and complain. Like not handling file paths that have spaces? [0] That's one hell of an edge case. [0] http://savannah.gnu.org/bugs/?712 http://savannah.gnu.org/bugs/?712
- arca_vorago 9y agoI'm speechless at that link. I don't understand why something so basic isn't fixed yet in such an important package. I guess I can see why qmake/cmake have been eating the market share now. For what it's worth, I am pretty sure one of my first BSD systems forced me into using snakecase so that would explain why I never hit that limitation. That said, make, being designed originally for C, and C being also known for snakecase, and early linux also known for discouraging spaces in names, I can see how this happened in the first place. What I don't understand is the lack of an implimented fix.
- dmytroi 9y agoMake provides awesome UX: in most cases you just type "make" and stuff just works, but there is a tradeoff: it's nearly impossible to write a crossplatform make file. Why you ask? Try copying a file (or directory) from one location to another, sure "cp", but it's -R is different on Linux and macOS! And then comes Windows - there is no good built-in alternative to "cp" as it's hard to make "xcopy" to ignore failures if target file/directory already exist (it tends to ask if you want to overwrite it or not), WSL might fix that in the future, but currently it's quite slow on IO side of things. So, sure given enough time and kung-fu we can write crossplatform make files ... but even if copying one silly file is such PITA .. it's hard to advocate for make in 2017. Probably the wisest solution is just using make to run something more comfortable, for C/C++ it can be cmake, premake, tup, etc. But then again: if I need to add something, like another build step, do I add it to the tool of choice or make? It's a very hard balance to sustain. PS. Copying files specifically is such huge PITA that cmake implements crossplatform copy for you, just run "cmake -E copy $in $out" (can be added as add_custom_command in CMakeLists.txt)
- jlebrech 9y agomaybe projects should have their own ./bin folder for make to use and have those command refer to the right native command. mixing make with a modern scripting language would make this possible.
- lederhosen 9y agoGNU Make has built in support for scheme
- gkya 9y agoIf you want to write a portable makefile, you use a CP variable which is conditionally set depending on what uname says, e.g. cp -r on GNU Linux and whatever it is on Mac OS. But also, copying files around is a bit out of make's scope, as it is a tool for transforming files. Most standard tools that are used in makefiles (sed, awk, m4, c toolchain) have posix interfaces, and other compilers usually don't have incompatible competing implementations. Even if you want to use a certain implementations features, say GNU features of awk, gawk+GNU coreutils are available for all popular systems, as well as bsdutils. Requiring them is better than when every program require its own build system because they are portable and reusable packages.
- bsenftner 9y agoI've been using make files constantly since '76 when I first started programming in (US) 5th grade. It has been one of my career amusements watching the build systems come and go. I gave up talking about Make years ago. There is a huge population of us that smile and get things done, while others screw around with new, complicated, never-learned-the-past tools. I use one of the earliest Make versions that barely does anything beyond recognize rules, rule actions, and simplistic macros. I've seen this version of Make used to build a feature animated film. Witnessing the versatility of that feature film's multitude of software and renders and composites all built through Make taught me Make is the only tool needed for any building of anything. I remember when MS introduced a Make with extensions, and many developers ran over and fucked up their build environments, starting them on the path to the complex manual required build mess most people have today. I write my Make films by hand, still. It is that easy. And you really should have a build environment that is that easy. When you need it easy, as in some major deadline and crap is broken, you will thank yourself.
- kruhft 9y agoWhich version of make do you use? The one packaged in the 'heirloom' setup, or something different that's still available?
- johnhenry 9y agoYou make a few statements regarding films: First, "I've seen this version of Make used to build a feature animated film" and later you state "I write my Make films by hand". I suspect the second statement might be a type-o -- accidentally using "films" in place of "files" -- but in the case that it's not, I wonder if you could detail the practice of "Make films" -- seems like it might be interesting.
- bsenftner 9y agoI meant "make feature animated films". During the late 80's, before more 'sophisticated'; build tools polluted developer's young brains, early visual effects were "nothing more" than complex Makefiles triggering renders and composites. All one needed was a facility that looked at the date stamp of two files, and if one was older that meant these series of command lines is executed. Beyond very basic #define style macro substitution, that was all that was used to create many feature film effects, as well as nearly ever single early 3D animated renderer produced animation before 1992.
- Blackthorn 9y agoHaving put together a build system together for building a modular os using make: no, it's really not time to make a comeback. Make is simultaneously powerful enough to seem pretty good for use, but not powerful enough to do anything outside its very narrow problem domain without really ugly hacks.
- abritinthebay 9y agoAgree. God forbid you have to maintain cross-platform compatible makefiles either.
- deleted 9y ago[deleted]
- Iv 9y agoIf you compare Makefiles to the JavaScript ecosystem it will do so favorably, but many things will. Make is a good build system but it is a shitty deployment system. Yes, you can do everything you want if you put enough effort in it as it is a complete scripting system, something many alternatives are not. It does not mean that it is a good idea to do so. Recently a client made me begrudgingly try Maven. 'Yet another make system' I thought. Yup, pretty standard one: 'mvn install' ,I'll compile everything and will also do something make has never done for me: install dependencies. Sure, it is doable to do this with a makefile. With a lot of efforts multiplied by the number of platforms you target. In Maven you just add repos and dependencies to your pom file. So yes, there recent build systems that are worse than Make but it does not mean that make is the perfect system with all the features we need. It is minimalistic. It works. But it should not be your only option.
- mikegerwitz 9y agoMake isn't intended to be used for dependency management. With traditional `./configure && make && make install`, dependencies are detected by the configure script, or manually provided by the user if they have custom library paths. Dependencies are handled by your package manager or something else (be it submodules in git, custom scripts, etc). There's a separation of concerns. For generating those configure files, Autoconf is extremely powerful (and really not that complicated once you have a basic understanding). If you use Automake, you get all the standard targets (including `make dist`, which will build your distribution archive, as is being done manually in this article) generated for you.
- orclev 9y agoThat's a rather dated view of what a build system should be. Most developers these days don't want to have to manually install all of a projects dependencies in order to build it. More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects, which is go read the docs to find the list of mandatory and optional dependencies, then go look through the distribution package manager to try to match up package names with library names (including a few head scratchers where something is included inside of a differently named package), and finally install the whole mess and hope that the projects config script properly finds them all.
- sime2009 9y agoI really like the make approach but I want a modern updated implementation of the idea. With the domain of web development in mind, my wishlist of requirements looks something like this: * Written in JS and installable via npm (i.e. runs on Windows, Linux and the other brand) * 'makefiles' are just JS code. * Supports using Shelljs for the command scripting parts and being cross platform. * Supports calling tools in "node_modules/.bin/" * Supports parallel builds for big modules projects. * No plugins, promises, streams, async or any other nonsense like most other JS/web related build tools or "task runners".
- senatorobama 9y agoWhat do people think of 'tup'?
- jhasse 9y agoI really like it, but I don't know if it's worth the hassle. For example compared to ninja it isn't that much faster and it's missing a good configuration system anyway. Definitely better than Make though.
- leafo 9y agoIt's my favorite build system. A big advantage is automatic dependency tracking. It picks up unspecified inputs and automatically rebuilds anything further down the dependency tree. It also knows how to clean up files that it no longer generates since it understands you removing and adding build steps. I can do huge refactors and it cleans up files without any additional work from me. The filesystem watch automatically builds things so I never have to run it directly. I use it on my production server when deploying since it's 100% accurate at incrementally moving between different project states. There are definitely some disadvantages, you have to work around its syntax and limitations. For complicated build functions I've written the config in Lua and it works fine.
- GlennS 9y agoI tried out Tup, and thought it was really impressive and probably great for people who need to write C, but not suited for what I was trying to do. The project I tried to use it on was a before/after comparison of the results of some development work to an energy model. So, a couple of my make targets cloned old and new versions of the repo. Tup doesn't like you to create directories, and wants to track every file that comes into existence and check that it is either a) a declared output or b) deleted by the end of the job. It really didn't enjoy trying to process a whole Git clone operation.
- user5994461 9y ago>>> Time for Makefiles to Make a Comeback Until your build chain is f----- by the tab VS space issue of makefiles. Then abandon makefiles again, for good.
- bb88 9y agoThis is really just an editor issue. Editors can parse a make file and do the right thing these days.
- overgard 9y agoNonononono. There's a reason makefiles have been reinvented 400 times: they're a nightmare. I don't love the alternatives, but they are pretty much all universally better.
- pippy 9y agoDependency hell is one of the main reasons why developers are flocking to bloated solutions such as election: a single monolithic dependency with guaranteed cross functionalist behavior. Most build tools are stuck in the 70-80's where a single library weighing hundreds of kilobytes was a big thing. In comparison to saving a few kilobytes vs spending hours of a developers time hunting down old versions of libraries is a no-brainier. Especially when you take security into account. I've seen .Net developers just give up in frustration and download dll files from those dodgy download sites and put them into production distributions. I personally loathe make. I've never had a good experience when using it.
- Nanshan 9y agoYeah, I'm writing a C build system just use shell+make, and it's work very nice for learning C programming and try some new ideas. https://github.com/junjiemars/nore https://github.com/junjiemars/nore
- Aardwolf 9y agoI wish there was something as standard, ubiquitous, fast&light, dependency-less, and adopted by the whole Unix ecosystem and beyond like make is now, yet better (= less archaic, more "write once works everywhere", uniform single way of depending on other projects, focused on the actual code/bins/libs and their dependencies instead of specific packaging for specific distros, ...). Not some "hype of the day" build system of course as that by definition makes it a short lived maintenance burden rather than universal. While make is universal, with makefiles now it's so bad, that even if you write pure C90 code with zero dependencies (which should work everywhere), users will still find reasons why you should add all kinds of platform dependent stuff or things for a particular distro to your makefile...
- jgrahamc 9y agoI have a theory that the people who love make are the same people who love macros and the people who hate make don't love them. I wrote a book on make and I fully recognize that there are people who look at what I've written there and think I'm completely mad: https://www.nostarch.com/gnumake https://www.nostarch.com/gnumake Make's greatest irony is that it's a system for building files based on dependencies but has no way of actually discovering dependencies. I wrote about this for Dr Dobbs a while back: http://www.drdobbs.com/architecture-and-design/dependency-management-the-irony-of-make/228701768 http://www.drdobbs.com/architecture-and-design/dependency-ma... And make has deficiencies that are a nightmare to work around and get right (e.g. spaces in filenames, recursion, ...) If anyone else out there is a make hacker like myself they might enjoy the GNU Make Standard Library project: http://gmsl.sourceforge.net/ http://gmsl.sourceforge.net/
- llimllib 9y ago(disclaimer: I like make but not macros) > And make has deficiencies that are a nightmare to work around and get right (e.g. spaces in filenames, recursion, ...) Yes! we seem to be stuck in a local maximum with make. I wrote my own make replacement, as have many before me, and learned the folly of my own ways. Nowadays I use (and love) make despite its terrible handicaps, but I really wish there were something fundamentally similar but a) less C-focused and b) containing many fewer footguns.
- lederhosen 9y agoShake is similar to Make with some improvements (it is a Haskell DSL), I have not used it, but it looks good. http://shakebuild.com/ http://shakebuild.com/
- hashmal 9y agoTIL Make is not used widely anymore. I may live in a bubble, but my last use of it was… maybe 2 hours ago (to build someone else's project) and yesterday (to build one of my own projects).
- abritinthebay 9y agoIt’s used for basically most C/C++ projects and somewhat outside of that. It’s not been adopted (with good reason) in many newer language communities, so depending on your tooling and platform you may see it less.
- bryanlarsen 9y agoYup, if Make was used extensively for Javascript, I wouldn't have had to file this feature request: https://github.com/webpack/webpack-cli/issues/152 https://github.com/webpack/webpack-cli/issues/152
- whipoodle 9y agoDiscover make, realize what a bunch of dumb hipsters everyone is who created build systems after it, try to use it, see how deficient it is for the problems you're trying to solve, realize why those dumb hipsters made all the stuff they made, repeat. If Chesterton's Gate were a build system.
- Walkman 9y agoIf you would see our Makefile which can do literally_everything_ with the project, you would not write this article. :) Makefiles don't scale, hard to learn because of awkward syntax.
- apple4ever 9y agoThat's the think. The concept of Makefiles are great. The syntax is pretty terrible. GNUstep's Makefiles are actually really nice.
- welly 9y agoRecent experience of working with various task runners such as Phing and Robo (plus others), eventually my previous employer returned back to simple Makefiles and I in turn have done the same. For my needs of generating small website builds, Make is perfect.
- jancsika 9y ago> Makefiles are simply a declarative way to transform one file (or series of files) into another. Tab echo it certainly is not greater than side_effect.txt.
- ruffrey 9y agoWe have been using make for build and test scripts for docker+Node apps. I have no complaints after a couple years.