68 ms·
Afraid of Makefiles? Don't be
- rileytg 9y agowow i've been feeling like not knowing make has been a major weakness of mine, this article has finally tied all my learning together. i feel totally capable of using make now. thank you.
- sevensor 9y agoMake is great! Just remember to use tabs for indentation. Make is very picky about that.
- rrmm 9y agoMakefiles are easy for small to medium sized projects with few configurations. After that it seems like people throw up their hands and use autotools to deal with all the recursive make file business. Most attempts to improve build tools completely replace make rather than adding features. I like the basic simplicity and the syntax, (the tab thing is a bit annoying but easy enough to adapt to). It'd be interesting to hear everyone's go to build tools.
- wwwigham 9y ago> It'd be interesting to hear everyone's go to build tools. A script written in whatever the primary language of the project is (usually with some library support), ideally; to reduce the minimum required knowledge to be a first time contributor to the project. Some kind of js build tool for js projects, fake for f#, and so on. I don't want people to need to learn "the one build tool DSL to rule them all" (be that makefile syntax, bazel rules, or whatever) to contribute to a project's infrastructure, on top of the project's primary language.
- sevensor 9y agoSo for C? Fortran? You'd have users of those languages use those languages to write programs to do the build? But then, how would you keep them from generalizing their good ideas about builds into some kind of... system?
- adrianN 9y agoThat is a good way to have your build system expand until it can read email.
- MereInterest 9y agoCurrently, I use scons for C/C++ projects, though with a decent amount of stuff built on top of it. When I realized that I had a rudimentary package manager in it, I started thinking that maybe I should go searching for something that already does what I need. I've used make quite a bit, and it is doable for individual projects. Where I start having trouble is managing libraries that may be used across multiple projects. When a library needs to supply its own configuration, and be subject to configuration specified of the project using that library, things get rather complicated, which is why I turned to scons.
- tpush 9y agoFor my current project in C++ I just use cmake. It works fine on Linux, Windows, FreeBSD & OpenBSD, spits out stuff that integrates with the native/most commonly used build environment on the respective platform, provides facilities for config, build & installing, and if you avoid all legacy cruft it's even somewhat decent.
- syncsynchalt 9y agoToday's simple makefiles are the end result of lessons hard learned. You'd be horrified to see what the output of imake looked like. From memory here's a Makefile that serves most of my needs (use tabs): SOURCE=$(wildcard *.c) OBJS=$(patsubst %.c,%.o, $(SOURCE)) CFLAGS=-Wall # define CFLAGS and LDFLAGS as necessary all: name_of_bin name_of_bin: $(OBJS) $(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) %.o: %.c $(CC) $(CFLAGS) -o $@ $^ clean: rm -f *.o name_of_bin .PHONY: clean all
- wyldfire 9y agobetter still, IMO -- capitalize on those sweet, sweet implicit rules. CFLAGS=-Wall # define CFLAGS, CXXFLAGS, LDLIBS and LDFLAGS as necessary all: name_of_bin name_of_bin: a.o b.o c.o clean: $(RM) -f *.o name_of_bin .PHONY: clean all
- MereInterest 9y agoI don't really like those, because then it makes it harder to separate out the source and the build directory. VPATH is a partial solution, but I find it much easier to have a single explicit pattern rule, rather than using both implicit pattern rules and modification of those implicit pattern rules.
- syncsynchalt 9y agoThat's fair. In actual projects of more than a few source files I do copy/paste some boilerplate to give me a build dir, dependency files, and so on. I just thought this (actual real-world example that I use for smallish projects) would help someone.
- syncsynchalt 9y agoThanks... I suspected I was just redefining a built-in pattern rule, but I put it there anyway just so I can remember whether it's LDFLAGS (linker) vs LFLAGS (lexer). Plus I was thinking maybe less magic for someone trying to make sense of it. Is there no command needed for name_of_bin to be linked? Does it automatically figure it out based on *.o dependencies?
- wyldfire 9y ago> You've learned 90% of what you need to know about make. That's probably in the ballpark, anyways. The good (and horrible) stuff: - implicit rules - target specific variables - functions - includes I find that with implicit rules and includes I can make really sane, 20-25 line makefiles that are not a nightmare to comprehend. For a serious project of any scope, it's rare to use bare makefiles, though. recursive make, autotools/m4, cmake, etc all rear their beautiful/ugly heads soon enough. But make is my go-to for a simple example/reproducible/portable test case.
- gumby 9y ago> For a serious project of any scope, it's rare to use bare makefiles, though. recursive make, autotools/m4, cmake, etc all rear their beautiful/ugly heads soon enough. If I didn't have to provide the option to build under Xcode or VS, I wouldn't have to live in the hell that is Cmake. I'd just use make.
- tomjakubowski 9y agocmake has improved so much since 3.0 that it's almost unrecognizable. Though much of the cruft remains, thanks to backwards compatibility, and anything involving list or string operations is still absolute hell, I dare say it's actually pleasant to use, at least compared to 2.8.whatever-shipped-with-ubuntu-precise. Personally, I'm glad we use cmake at work simply because we have a diversity of preferred build tools: some people like to work in XCode or Eclipse, others like to use a text editor and make, others like to use a text editor and ninja. While we all suffer a bit with cmake, we all benefit from its position as a "metabuild" tool.
- nwmcsween 9y agoI hate implicit anything, trying to reason about anything complex with implicits just annoys the reader.
- rcthompson 9y agoFor people who are more comfortable in Python, I highly recommend Snakemake[1]. I use it for both big stuff like automating data analysis workflows and small stuff like building my Resume PDF from LyX source. [1]: https://snakemake.readthedocs.io/en/stable/ https://snakemake.readthedocs.io/en/stable/
- pkkim 9y agoOne important tip is that the commands under a target each run sequentially, but in separate shells. So if you went to set env vars, cd, activate a Python virtualenv, etc to affect the next command, you need to make them a single command, like: target: cd ./dir; ./script.sh
- bluejekyll 9y agoMake is awesome. I have always loved make, and got really good with some of its magic. After switching to Java years ago, we collectively decided, "platform independent tools are better", and then we used ant. Man was ant bad, but hey! It was platform independent. Then we started using maven, and man, maven is ridiculously complex, especially adding custom tasks, but at least it was declarative. After getting into Rust, I have to say, Cargo got the declarative build just right. But then, for some basic scripts I decided to pick Make back up. And I wondered, why did we move away from this? It's so simple and straightforward. My suggestion, like others are saying, is keep it simple. Try and make declarative files, without needing to customize to projects. I do wish Make had a platform independent strict mode, because this is still an issue if you want to support different Unixes and Windows. p.s. I just thought of an interesting project. Something like oh-my-zsh for common configs.
- cperciva 9y agomake does have a platform independent strict mode. Unfortunately the ".POSIX:" mode is sufficiently limited that you won't want to use it for anything you're writing by hand.
- bluejekyll 9y agoWow. I didn't realize this. Does it support Windows, without Linux etc installed?
- mauvehaus 9y agoDon't know if there's a Win32 build of GNU make, but Microsoft has nmake. I haven't written a Makefile for it in years, but I have a hard time imagining there being much portability between the two. Not in the Makefile stucture, but rather in the commands it invokes to actually build targets.
- cperciva 9y agoI'd guess that nmake is POSIX-compliant (or close to it), simply because the POSIX requirements are so minimal that it would take quite a bit of effort to avoid being POSIX compliant.
- danso 9y agoWorth pointing to Mike Bostock's essay, Why Use Make: https://bost.ocks.org/mike/make/ https://bost.ocks.org/mike/make/
- raimue 9y agoBy using pseudo targets only in the example and not real files, the article misses the main point of targets and dependencies: target rules will only be executed if the dependencies changed. make will compare the time of last modification (mtime) on the filesystem to avoid unnecessary compilation. To me, this is the most important advantage of a proper Makefile over a simple shell script always executing lots of commands.
- Touche 9y agoDon't you have to list every file though? Many of my projects have hundreds of source files.
- ycombobreaker 9y agoNo, you can use `wildcard`
- deleted 9y ago[deleted]
- syncsynchalt 9y agoSo my example (below in another comment), there's a solution using $(wildcard).
- deleted 9y ago[deleted]
- flukus 9y agoPersonal blog spam, I learned make recently too and discovered it was good for high level languages as well, here is an example of building a c# project: http://flukus.github.io/rediscovering-make.html http://flukus.github.io/rediscovering-make.html . Now the blog itself is built with make: http://flukus.github.io/building-a-blog-engine.html http://flukus.github.io/building-a-blog-engine.html
- quickthrower2 9y agoLove the simple blog design. That effectively nullifies this as being blogspam.
- flukus 9y agoThanks. I took the "making this pretty is the web clients job" approach, it looks nice in firefox reader mode. The two biggest issues are: 1. Syntax higlighting doesn't exist in reader. 2. The index page (http://flukus.github.io/ http://flukus.github.io/) can't render in reader mode, it didn't hit the "must have a paragraph with at least 68 characters" or whatever the arbitrary limitation is. For the later I'm hoping the add some sort of meta tag that allows me to enable it in future.
- nu11p01n73R 9y agoLove the simplicity of the design. Is there any reason why you didn't include the date of publishing in the pages. Or is it just me who looks for the dates on all the blogs that I read.
- flukus 9y agoI look for the date too, especially on technical blogs, it's meant to be there but there's a bug somewhere in the pipeline.
- majewsky 9y agoI wouldn't call is "simplicity", but rather "absence" of a design. See the arguments laid out in http://bettermotherfuckingwebsite.com/ http://bettermotherfuckingwebsite.com/
- bauerd 9y agoI remember trying to wrap my head around the monstrosity that is Webpack. Gave up and used make, never looked back since
- fiatjaf 9y agoWebpack is an horrible monster that makes the machine-generated Makefiles look pretty.
- mauvehaus 9y agoI feel like any discussion of make is incomplete without a link to Recursive Make Considered Harmful[0]. Whether you agree with the premise or not, it does a nice job of introducing some advanced constructs that make supports and provides a non-contrived context in which you might use them. [0] http://aegis.sourceforge.net/auug97.pdf http://aegis.sourceforge.net/auug97.pdf
- gergoerdi 9y agoNon-Recursive Make is also Considered Harmful: https://news.ycombinator.com/item?id=11441719 https://news.ycombinator.com/item?id=11441719 In seriousness, the linked paper describes Shake, a Make replacement with two party tricks: one, it is implemented as a DSL embedded in Haskell, thus giving a nice way of programmatic rule generation; and two, it supports monadic dependencies, unlike Make's applicative-only ones.
- deleted 9y ago[deleted]
- chungy 9y agoI think the primary thing that makes people fear Makefiles is that they try learning it by inspecting the output of automake/autoconf, cmake, or other such systems. These machine-generated Makefiles are almost always awful to look at, primarily because they have several dozen workarounds and least-common-denominators for make implementations dating back to the 1980s. A properly hand-tailored Makefile is a thing of beauty, and it is not difficult.
- zeta0134 9y agoI'm really impressed with the makefiles provided by devKitPro, which is a toolchain for making homebrew on a handful of video game platforms: https://github.com/devkitPro/nds-examples https://github.com/devkitPro/nds-examples They're very clearly hand made, do just enough to be immensely useful, and are well commented enough to be easy to modify for a Make beginner, just like I was back in the day. I'm sure there are other great examples floating around the 'net, but I cut my teeth on GBA and NDS development, and taught myself make by following in their footsteps.
- boondaburrah 9y agoI keep trying to get into GBA development but this is actually what keeps blocking me. Everything they've written is set up just for their environment and is not really explained how it works. Sure, they explain what to change for your game, but my arm compiler is over here and could you just tell me where this gba.spec file comes from or what it does?! (etc. etc.)
- digi_owl 9y agoAnd yet we keep producing such tool over and over. Meson seems to be the latest fashion in Linux circles, it seems...
- valarauca1 9y agoAll build systems suck. You are trying to solve a Turing complete problem in a simplest low impact way as possible. There is no good solution. Everyone will fall short in one way or another. Hence a new build system. The wheel never stops.
- mauvehaus 9y agoHas anybody successfully used make to build java code? I realize there are any number of other options (ant, maven, and gradle arguably being the most popular). In fact, I realize that the whole idea of using make is probably outright foolish owing to the intertwined nature of the classpath (which expresses runtime dependencies) and compile-time dependencies (which may not be available in compiled form on the classpath) in Java. I'm merely curious if it can be done.
- bluejekyll 9y agoA long time ago I had a build system for C++ and Java. It worked fine. The issue is generally that you want it to work on Windows and Unix, and then life is hell...
- agibsonccc 9y agoThe problem with using make here is transitive dependency resolution. Many of the new build systems have it "built in" now a days. Maybe make could wrap something that does that? Either way...I wouldn't recommend it unless it wrapped something that understood maven central.
- ajross 9y agoSure it can be done. Long, long ago, before Ant existed, I worked on a big Java project built entirely with javac and GNU make. In fact javac's built-in dependency tracking makes this significantly easier than with C.
- Animats 9y agoThe trouble with "make" is that it's supposed to be driven by dependencies, but in practice it's used as a scripting language. If the dependency stuff worked, you would never need make clean; make or touch
- ycombobreaker 9y agoIn my real world at work, the "dependency stuff" works just fine. I never clean for my iterative development, and `-B` is what you want for a pristine/paranoia build.
- chubot 9y agoYeah this is my pet peeve about how people use Makefiles. Make is supposed to be a dependency-driven "dataflow" language, but over time it's evolved into something like shell. And people tend to use it like shell. The most glaring example is .PHONY targets, which are a hack and should just be shell functions. In 'make <foo>', <foo> should be data, not code. 'make test' doesn't make sense, but 'make bin/myprog' does. I posted this link in another comment showing how Make, Shell, and Awk overlap: http://www.oilshell.org/blog/2016/11/14.html http://www.oilshell.org/blog/2016/11/14.html Here are some more comments on Make's confused execution model. It's sort of functional/dataflow and sort of not. In practice you end up "stepping through" an imperative program rather than reasoning about inputs and outputs like in a functional program. https://news.ycombinator.com/item?id=14840696 https://news.ycombinator.com/item?id=14840696 And at the end of this post, I talk a bit more about the functional model: http://www.oilshell.org/blog/2017/05/31.html http://www.oilshell.org/blog/2017/05/31.html
- amirbehzad 9y agoWhat simple tool would you suggest to address those .PHONY-targets instead of a Makefile?
- chubot 9y agoI recommend using plain shell scripts. Each .PHONY target is a simple shell function, and then the last line of the script is "$@", which means run function $1 with parameters $2 to $n. So instead of 'make test', I just use './test.sh all', or './test.sh foo'. The test script can invoke make. The idea is that 'dataflow' parts are done in Make, and the imperative parts are done in shell. This works out fairly well if you're disciplined about it. The only point of Makefiles is to support quick incremental builds. If there's no incrementality, then you might as well use shell. (Note: whenever you use Make, you're using shell by definition. There's no possibility of only using Make. So I take Make out of the picture where it's not necessary.) For example, all the instructions here are of the form 'foo.sh ACTION': https://github.com/oilshell/oil/wiki/Contributing https://github.com/oilshell/oil/wiki/Contributing Pretty much every shell script in the repo uses "the argv dispatch pattern"... I've been meaning to write a blog post about that pattern, i.e. using functions with the last line as "$@". https://github.com/oilshell/oil https://github.com/oilshell/oil
- bitwize 9y agoOr just use cmake and save yourself time, effort, and pain.
- humanrebar 9y agoYes. Among other things, a modern build system really needs a "help" command.
- fiatjaf 9y agoMakefiles are simple, but 99% of the existing Makefiles are computer-generated incomprehensible blobs. I don't want that.
- AceJohnny2 9y ago"Build systems are the bastard stepchild of every software project" -- me a years ago I've work in embedded software for over a decade, and all projects have used Make. I have a love-hate relationship with Make. It's powerful and effective at what it does, but its syntax is bad and it lacks good datastructures and some basic functions that are useful when your project reaches several hundred files and multiple outputs. In other words, it does not scale well. Worth noting that JGC's Gnu Make Standard Library (GMSL) [1] appears to be a solution for some of that, though I haven't applied it to our current project yet. Everyone ends up adding their own half-broken hacks to work around some of Make's limitations. Most commonly, extracting header file dependency from C files and integrating that into Make's dependency tree. I've looked at alternative build systems. For blank-slate candidates, tup [2] seemed like the most interesting for doing native dependency extraction and leveraging Lua for its datastructures and functions (though I initially rejected it due the the silliness of its front page.) djb's redo [3] (implemented by apenwarr [4]) looked like another interesting concept, until you realize that punting on Make's macro syntax to the shell means the tool is only doing half the job: having a good language to specify your targets and dependency is actually most of the problem. Oh, and while I'm around I'll reiterate my biggest gripe with Make: it has two mechanisms to keep "intermediate" files, .INTERMEDIATE and .PRECIOUS. The first does not take wildcard arguments, the second does but it also keeps any half-generated broken artifact if the build is interrupted, which is a great way to break your build. Please can someone better than me add wildcard support to .INTERMEDIATE. [1] http://gmsl.sourceforge.net http://gmsl.sourceforge.net [2] http://gittup.org/tup/ http://gittup.org/tup/ Also its creator, Mike Shal, now works at Mozilla on their build system [3] http://cr.yp.to/redo.html http://cr.yp.to/redo.html [4] https://github.com/apenwarr/redo https://github.com/apenwarr/redo
- joeyo 9y agoTup is really great. I wish it were more widely used. Tup can effortlessly handle a lot of situations that Make chokes on or requires deep magic to get right. A good example is the clean handling of automatically generated header files. This is all it takes to integrate protobufs into a Tupfile: : foreach *.proto |> !protoc |> %g.pb.cc | %g.pb.h : foreach *.pb.cc | *.pb.h |> !cpp |> %g.pb.o The "|>" is the pipe operator and I've elided the definitions of the "!protoc" and "!cpp" macros (but they're about what you'd expect). Tup detects whenever a .proto file is changed and does the right thing. Getting this to work with Make requires advanced tricks like .PHONY and .PRECIOUS.
- vacri 9y agoOne very important thing missing from this primer is that Make targets are not 'mini-scripts', even though they look like it. Every line is 'its own script' in its own subshell - state is not passed between lines. Make is scary because it's arcane and contains a lot of gotcha rules. I avoided learning Make for a long time. I'm glad I did learn it in the end, though I wouldn't call myself properly fluent in it yet. But there are a ton of gotchas and historical artifacts in Make.
- makapuf 9y ago> Every line is 'its own script' in its own subshell - state is not passed between lines. Unless you use the ONESHELL option.
- rdtsc 9y agoSneaky pro-tip - use Makefiles to parallelize jobs that have nothing to do with building software. Then throw a -j16 or something at it and watch the magic happen. I was stuck on an old DoD redhat box and it didn't have gnu parallel or other such things and co-worker suggested make. It was available and it did the job nicely.
- mturmon 9y agoSeconded. Really good for preprocessing of data files, or repetitive data pipelines. Just set up a naming convention for the intermediate files and define transformation rules. Then you can drop one new file in somewhere, type "make", and the minimal set of processing will just run. If you need more power, use the wildcard expansion and "patsubst" type rules to define rules at runtime.
- deleted 9y ago[deleted]
- coredog64 9y agoGnu xargs since circa 2009 has had a '-P' option that will let you execute simple tasks in parallel.
- rdtsc 9y agoI forgot why I couldn't use xargs -P there. If I had to guess ancient DoD version of RHEL, like say RHEL 5. Note it was released in 2007 and xargs as you mentioned had that since 2009. Sometimes Redhat backport things but I just remember not finding anything there and then being very happy to have discovered make. I think even with xargs -P I would have gone with make anyway as it involved a few dependencies and checking of file times.
- JdeBP 9y agoFor more, see https://news.ycombinator.com/item?id=14836340 https://news.ycombinator.com/item?id=14836340 (Ask HN: In what creative ways are you using Makefiles?). I use a build system, redo rather than make, to rebuild my GOPHER site and Debian/FreeBSD package repositories. (https://news.ycombinator.com/item?id=14837740 https://news.ycombinator.com/item?id=14837740 https://news.ycombinator.com/item?id=14928216 https://news.ycombinator.com/item?id=14928216)
- epx 9y agoThose who don't understand Make are condemned to reimplement it, poorly.
- stormbeta 9y agoThe closest thing I've seen that seem likes a real replacement is Bazel, but I haven't tried using it for anything but a couple toy projects yet. The concepts seem solid though, as expected from Google. My main complaint about make is that relying on timestamps, while fast, is error-prone and doesn't work with idempotent commands that might refresh the timestamp without altering content.
- devdoomari 9y agoso I guess makefile is somewhat like gulp JS... but can I split a makefile into multiple files?
- cbcoutinho 9y agoYes, you can split it up even into separate directories
- Joky 9y agoMake is fine for simple cases, but I'm working on a project that is based on buildroot right now, and it is kind of a nightmare: make just does not provide any good way at this scale to keep track of what's going on and inspect / understand what goes wrong. Especially in the context of a highly parallel build with some dependencies are gonna get missing. In general also all the implicit it has makes it hard to predict what can happen. Again when you scale to support a project that would be 1) large and 2) wouldn't have a regular structure. On another smaller scale: doing an incremental build of LLVM is a lot faster with Ninja compared to Make (crake-generated). Make is great: just don't use it where it is not the best fit.
- deleted 9y ago[deleted]
- brian-armstrong 9y agoUsing Cmake is so much nicer than make, and it's deeply cross-platform. Cmake makes cross-compiling really easy, while with make you have to be careful and preserve flags correctly. Much nicer to just include a cmake module that sets up everything for you. Plus it can generate xcode and visual studio configs for you. Doing make by hand just seems unncessary.
- kyberias 9y agoI always felt that for small setups, makefiles are much more concise and easy to understand. And more generic. Cmakefile.txt seems always really messy and introduce concepts such as project and exes and dlls. But of course Cmake is super-portable I know.
- brian-armstrong 9y agoEvery time I think this, I end up going back later and wishing I had just done Cmake. If you just want to compile all the .c files it's easy enough to GLOB and make a couple targets, then come back and make it more sophisticated later
- everheardofc 9y agoExcept with make most small setups are at least 20-40 lines of nontrivial code becaue you have to redo everything from scratch
- slrz 9y agoinclude std.mk TARGET=foo HFILES=bar.h OFILES=main.o foo.o baz.o include cmd.mk This is what Makefiles tend to look like in some places.
- jcelerier 9y agoIs project(my_software CXX) add_library(my_lib foo.cpp bar.cpp) target_include_directories(my_lib PUBLIC include/lib) add_executable(my_app bluh.cpp) target_link_libraries(my_app my_lib) really more complicated than the equivalent makefile ? Especially given that: * this can generate ninja files instead of make * it adds the common targets such as make help, make clean, make install * dependency graphs can be generated with cmake --graphviz * solutions for various IDEs * etc...
- gtramont 9y agoHere's some tips I like to follow whenever writing Makefiles (I find them joyful to write): http://clarkgrubb.com/makefile-style-guide http://clarkgrubb.com/makefile-style-guide
- leastangle 9y agoI did not know people are afraid of Makefiles. Maybe a naïve question, but what is so scary about make?
- analog31 9y agoI wouldn't say I'm afraid of one, but I can't read one and tell you what it does, or write one that does anything useful. Unless I'm way off the mark, it's just a programming language, one that I will eventually learn when I need it badly enough. I even snuck a manual off the freebie table at work.
- syncsynchalt 9y agoI think chungy is right: most people get their first experience with Makefiles by trying to debug some automake monstrosity. When they write their own, they assume they have to be just as complicated (or even worse, they'll copy/paste from an autogenerated Makefile or use it as a base). If I thought every Makefile had to be like that I'd write ./build.sh too.
- dmitriid 9y agoIt's all fun and games while your commands basically look like target: invoke-some-tool And then, in a bigger project, you start dealing with dependencies. Exclusions. Code generation. Config-dependent builds. Dev vs stage vs prod. Sub-projects. Invocation of tools depending on other tools. Pipelines. Toolchains. Then Makefiles quickly devolve into an incomprehensible undebuggable nightmare with arcane syntax rules and behaviours. It doesn't help that there are next to none good resources on Makefiles.
- systemz 9y agoInstead of makefile I can recommend Taskfile https://hackernoon.com/introducing-the-taskfile-5ddfe7ed83bd https://hackernoon.com/introducing-the-taskfile-5ddfe7ed83bd Simple to use without any magic.
- ejholmes 9y agoMake's underlying design is great (it builds a DAG of dependencies, which allows for parallel walking of the graph), but there's a number of practical problems that make it a royal pain to use as a generic build system: 1. Using make in a CI system doesn't really work, because of the way it handles conditional building based on mtime. Sometimes you just don't want the condition to be based on mtime, but rather a deterministic hash, or something else entirely. 2. Make is _really_ hard to use to try to compose a large build system from small re-usable steps. If you try to break it up into multiple Makefiles, you lose all of the benefits of a single connected graph. Read the article about why recursive make is harmful: http://aegis.sourceforge.net/auug97.pdf http://aegis.sourceforge.net/auug97.pdf 3. Let's be honest, nobody really wants to learn Makefile syntax. As a shameless plug, I built a tool similar to Make and redo, but just allows you to describe everything as a set of executables. It still builds a DAG of the dependencies, and allows you to compose massive build systems from smaller components: https://github.com/ejholmes/walk https://github.com/ejholmes/walk. You can use this to build anything your heart desires, as long as you can describe it as a graph of dependencies.
- Groxx 9y agoVery much agreed :| It's great when it's kept fairly simple (for both automating random stuff and building things! and it's installed everywhere! and has some cross-platform tools!), but it quickly turns into a nest of vicious footguns unless you're extraordinarily careful.
- ejholmes 9y agoAlso, I'm always surprised how little mention there is of graphs when we start talking about build systems. If you're making a build system (for literally anything) the DAG is your best friend. This is how all major build system tools work (make, terraform, systemd (yes, it's a build system when you think about it)) and it's how we're able to parallelize execution so easily. It's important to be conscious of the fact that this is what your doing when your making a build system; connecting a graph. Highly recommend reading https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-042j-mathematics-for-computer-science-spring-2015/readings/MIT6_042JS15_Session17.pdf https://ocw.mit.edu/courses/electrical-engineering-and-compu... for some theory on parallel execution for graphs, if your interested in things like this.
- whipoodle 9y agoI just always hated that you're supposed to write ".PHONY". The first time I forgot to do that and it built a random file named after a build step I'd scrap the whole thing.
- anilakar 9y agoWho needs makefiles when you have a build system, you might ask. The truth: go see the configuration of a Jenkins project, and there's a high chance that one of the lines there still says "make".
- misnome 9y agoAlmost every build system (where I think it isn't controversial to say make is most often used) looks nice and simple with short, single-output examples to demonstrate the basis of a system. It's when you start having hundreds of sources, targets, external dependencies, flags and special cases that it becomes hard to write sane, understandable Makefiles, which it presumably why people tend to use other systems to generate makefiles. So sure, understanding what make is, and how it works is probably important, since it'll be around forever. But there are usually N better ways of expressing a build system, nowadays.
- callumlocke 9y agoI'm not scared of Makefiles, I just find them painful to work with.
- mschuster91 9y agoPlease, don't ship your own Makefiles. Yes, autotools sucks - but there is one thing that sucks more: no "make uninstall" target. Good people do not ship software without a way to get rid of it, if needed.
- zwischenzug 9y agoThis is great, and needs saying. Recently I wrote a similar blog about an alternative app pattern that uses makefiles: https://zwischenzugs.wordpress.com/2017/08/07/a-non-cloud-serverless-application-pattern-using-git-and-docker/ https://zwischenzugs.wordpress.com/2017/08/07/a-non-cloud-se...
- majewsky 9y agoI guess that you're the owner of that blog, so I want to file a bug report: There are ligatures in your code snippets, e.g. in "git diff", the "ff" is a single character that's as wide as the "i" before it, which is really weird, and also not what the terminal would do. So you might want to add this to your CSS: pre { font-variant-ligatures: none }
- rcarmo 9y agoThese days, most of my projects have a Makefile with four or five simple commands that _just work_ regardless of the language, runtime or operating system in use: - make deps to setup/update dependencies - make serve to start a local server - make test to run automated tests - make deploy to package/push to production - make clean to remove previously built containers/binaries/whatever There are usually a bunch of other more specific commands or targets (like dynamically defined targets to, say, scale-frontends-5 and other trickery), but this way I can switch to any project and get it running without bothering to lookup the npm/lein/Python incantation du jour. Having sane, overridable (?=) defaults for environment variables is also great, and makes it very easy to do stuff like FOOBAR=/opt/scratch make serve for one-offs. Dependency management is a much deeper and broader topic, but the usefulness of Makefiles to act as a living document of how to actually run your stuff (including documenting environment settings and build steps) shouldn't be ignored. (Edit: mention defaults)
- MyPseudonym 9y agoThere are other projects than just web apps you know.
- rcarmo 9y agoI use the above for database imports, ETL and Azure template deployments too. And who says "serve" launches an HTTP server? :)
- ojosilva 9y agoOpinion poll. I'm writing a little automation language in YAML and I was wondering if people prefer a dependency graph concept where tasks run parallel by default, unless stated as dependency, or a sequential set of instructions where tasks only run in parallel if explicitly "forked". I'd say people would lean towards the former, but time and real world experience has shown that sequential dominates everything else.
- mycat 9y agoSo what is npm install of C/C++ ?
- Crontab 9y agoI haven't ever had to mess with Makefiles, as I don't program anything other the basic shell scripts, but I do remember reading articles in the past that indicated Makefiles can be used for more than just programming. For example: using Makefiles to automate static webpage creation and image file conversion.
- knowsuchagency 9y agoMake is fine, but I think we have better tools nowadays to do the same things. Even though it may not have been originally intended as such, I've found Fabric http://docs.fabfile.org/en/1.13/tutorial.html http://docs.fabfile.org/en/1.13/tutorial.html to be far far more powerful and intuitive as a means of creating CLI's (that you can easily parametrize and test) around common tasks such as building software.
- elnygren 9y agoI almost always roll a basic Makefile for even simple web projects. PHONY commands like "make run" and "make test" in every project make context switching a bit more easier. While things like "npm start" are nice, not all projects are Node.js. In my current startup we're gonna have standardised Makefiles in each project so its easy to build, test, run, install any microservice locally :)
- quantos 9y agoI had written Non Recursive Makefile Boilerplate (nrmb) for C, which should work in large projects with recursive directory structure. There is no need to manually add source file names in makefile, it automaically do this. One makefile compiles it all. Of course, it isn't perfect but it does the job and you can modify it for your project. Here is the link https://github.com/quantos-unios/nrmb https://github.com/quantos-unios/nrmb Have a look :)
- nstart 9y agoSo I saw this and thought why not give it a try. How hard could it be right? My goal? Take my bash file that does just this (I started go just yesterday so I might be doing cross compiling wrong :D) : ``` export GOPATH=$(pwd) export PATH=$PATH:$GOPATH/bin go install target/to/build export GOOS=darwin export GOARCH=amd64 go install target/to/build ``` which should be simple. Right? Set environment variables, run a command. Set another environment variable, run a command. 45 minutes in and I haven't been able to quite figure it out just yet. I definitely figured out how to write my build.sh files in less than 15 minutes for sure when I started out.
- JacksCracked 9y agoEach logical line of a makefile is executed in its own shell. This means that variables cannot be set across lines and directory changes cannot be changed across lines.
- ComputerGuru 9y agoJust use the export command to set variables across lines.
- sethammons 9y agoAside: for Go, just set your GOPATH and such once. Like ~/go. And never touch it again. You now can 'go get' any packages you need. When you set your package up for source control, its root will be (say, for a github package) $GOPATH/src/github.com/your-account/your-package. When you want to work on that package, you cd to that directory, and manage it there. This is why many who use Go don't get all the fuss with makefiles that some folks insist on. You just go to your project directory and 'go test' or 'go build' or whatnot. Simple. No need to 'make test' or 'make build' unless you have some strange, complicated set up.
- wink 9y agoHere's what I use: https://github.com/wywy/check-bucket-size/blob/master/Makefile https://github.com/wywy/check-bucket-size/blob/master/Makefi...
- 9y ago
- DangerousPie 9y agoIf you want all the greatness of Makefiles without the painful syntax I can highly recommend Snakemake: https://snakemake.readthedocs.io/en/stable/ https://snakemake.readthedocs.io/en/stable/ It has completely replaced Makefiles for me. It can be used to run shell commands just like make, but the fact that it is written in Python allows you to also run arbitrary Python code straight from the Makefile (Snakefile). So now instead of writing a command-line interface for each of my Python scripts, I can simply import the script in the Snakefile and call a function directly. Eg. rule make_plot: input: data = "{name}.txt" output: plot = "{name}.png" run: import my_package my_package.plot(input['data'], output['plot'], name = wildcards['name']) Another great feature is its integration with cluster engines like SGD/LSF, which means it can automatically submit jobs to the cluster instead of running them locally.
- deleted 9y ago[deleted]
- rurban 9y agoLooks like much more syntax to me, hence more painful. Make, bsdmake and gmake syntax is minimal and precise enough.
- crdoconnor 9y agoWow. This project has probably the worst example of the telescoping constructor antipattern I've ever seen: https://snakemake.readthedocs.io/en/stable/api_reference/snakemake.html https://snakemake.readthedocs.io/en/stable/api_reference/sna...
- todd8 9y agoIt's Python, not Java, so it can't have the telescoping constructor anti-pattern. Admittedly, there are a huge number of arguments to the snakefile constructor, but they are optional named arguments making the constructor safer and easier to use than Java's telescoping constructor and it alternatives (Java Bean pattern or builder patterns). This application apparently has many options or settings.
- baby 9y agoOther resources: https://gist.github.com/isaacs/62a2d1825d04437c6f08 https://gist.github.com/isaacs/62a2d1825d04437c6f08 https://learnxinyminutes.com/docs/make/ https://learnxinyminutes.com/docs/make/
- JelteF 9y ago> Add an @ sign to suppress output of the command that is executed. This is the exact opposite. It supresses echoing the command that is being executed. It's output is still shown like normal.
- user5994461 9y ago>>> Congratulations! You've learned 90% of what you need to know about The next 90% will be to learn that Make breaks when having tabs and spaces in the same file, and your developers all use slightly different editors that will mix them up all the time.
- vortico 9y agoMost editors that I've used switch to tabs mode when opening Makefile* files. But yes, it's a weird restriction. I suppose it's a result of being one of the first languages with syntactical indentation.
- qznc 9y agoI love Make for my small projects. It still could be better. Here is my list: * Colorize errors * Hide output unless the command fails * Automatic help command which shows (non-file) targets * Automatic clean command which deletes all intermediate files * Hash-based update detection instead of mtime * Changes in "Makefile" trigger rebuilds * Parallel builds by default * Handling multi-file outputs * Continuous mode which watches the file system for changes and rebuilds automatically I know of no build system which provides these features and is still simple and generic. Tup is close, but it fails with LaTeX, because of the circular dependencies (generates and reads aux file).
- martin_ky 9y agoDue to its versatility, Makefiles can be creatively used beyond building software projects. Case in point: I used a very simple hand-crafted Makefile [1] to drive massive Ansible deployment jobs (thousands of independently deployed hosts) and work around several Ansible design deficiencies (inability to run whole playbooks in parallel - not just individual tasks, hangs when deploying to hosts over unstable connection, etc.) The principle was to create a make target and rule for every host. The rule runs ansible-playbook for this single host only. Running the playbook for e.g. 4 hosts in parallel was as simple as running 'make -j4'. At the end of the make rule, an empty file with the name of the host was created in the current directory - this file was the target of the rule - it prevented running Ansible for the same host again - kind of like Ansible retry file, only better. I realize that Ansible probably is not the best tool for this kind of job, but this Makefile approach worked very well and was hacked together very quickly. [1] https://gist.github.com/martinky/819ca4a9678dad554807b68705bedbcb https://gist.github.com/martinky/819ca4a9678dad554807b68705b...
- athenot 9y agoAfter using the various javascript build processes, I went back to good old makefiles and the result is way simpler. I have a target to build the final project with optimizations and a target to build a live-reload version of the project, that watches for changes on disk and rebuilds the parts as needed (thanks to watchify). This works in my cases because I have browserify doing all the heavy lifting with respect to dependency management.
- deleted 9y ago[deleted]
- leksak 9y agoI like Tup
- analognoise 9y agoDelphi/Pascal haven't used make files for...I don't know how long. Shout out to FreePascal/Lazarus yet again!
- erAck 9y agoTake a look at the LibreOffice gbuild system, completely written in GNU make "language". And then come back saying you're not afraid of make ;-) Still, it probably would be much harder, if possible at all (doubted for most), to achieve the same with any other tool mentioned here.
- eighthnate 9y agoIs that a rhetorical question? Is there anyone who is afraid of makefiles? Makefiles exist to make your life easier.