8 ms·
A Love Letter to Make
- andrewstuart 4y agoI picked up a project with make file and it’s insides were so horrible I switched the project to cmake. I asked ChatGPT to write the CmakeLists.txt for me and after some debugging it worked.
- dTal 4y agoWell that's an interesting new level on the teetering tower of build systems. Perhaps, in the future, projects will simply include ChatGPT prompts instead of CmakeLists.txt directly. Or a script to generate the correct ChatGPT prompt...
- WesolyKubeczek 4y ago(A prompt (to generate a script to generate a prompt)^N) to generate CMakeLists.txt A prayer that the result is correct is not included.
- moffkalast 4y agoIt's kind of infuriating that CMakeLists even has to be manually made, because the compiler can clearly tell you what's wrong about almost every line in it if you mess it up. It should be completely doable to generate at least like 95% of it at compile time. Just so much repetitive boilerplate, like it was made by Oracle or something.
- andrewstuart 4y agoI don’t understand what you’re unhappy about.
- moffkalast 4y agoAbout having to specify all deps manually, all executable files, in which directories they are, which libs to link, etc. I mainly use it with ROS, where there's even more custom garbage to specify, like message building, package definitions and more. Don't forget to get it all in the correct order too and list every dep in the correct group depending on how it's used, or it won't compile. Fml. Things every modern language with a deps manager infers completely automatically without having to ever think about it. Sure it's understandable that C++ is still so far behind the times for legacy compatibility reasons, but I don't see why we haven't yet gradually moved towards something less tedious.
- oftenwrong 4y agoI always see others complain about Makefile syntax, especially the whitespace, but that is, to me, one of the smallest problems with make. For myself, the top problems are: - No tracking of changes to build rules. The build rule of a target should be one of its dependencies. - No protection from dirty builds. Your sources root is also your output root, and make does not help you keep it clean. - No options for extending the build to use alternate up-to-date checks (e.g. not mtimes), caches, execution mechanisms, etc. These all have workarounds, but they require the developer to manually include the workarounds in every build rule.
- pif 4y ago> Your sources root is also your output root Only if you learnt to program by reading Microsoft examples. Unix developers do not mix compiled files with source files.
- WesolyKubeczek 4y agoLaughs in openssl Seriously, it’s very project-dependent, but you’re lucky if your thing of choice that you need to build often supports a separate build directory, let alone enforces it.
- oftenwrong 4y agoYes, it is one of those things that many experienced makefile writers know to avoid, but the tool does not guide new users to this wiser approach. Contrast with Bazel, which does not pollute the workspace with outputs.
- nerdponx 4y agoBut Make isn't really a build tool, even if it was/is intended as such. This kind of thing is exactly what more sophisticated build tools are good for.
- duped 4y agoOne of the foundational concepts in an autotools builds is mixing generated files with source files.
- walthamstow 4y agoWe use Make a lot at $dayjob, mostly for running tasks on AWS CodeBuild machines and the like. I used to curse having to trawl through docs older than myself just to get something working but now I just send the recipe I need to write/edit to ChatGPT and it does it for me.
- kazinator 4y ago> with your insistence on using tabs instead of spaces, You can tell GNU Make to use a leading character other than tab for recipe lines, like the > symbol or whatever: .RECIPEPREFIX = > This is a variable with a leading dot its name, not a directive like .SUFFIXES: or .PHONY:.
- kuon 4y agoI don't really like make, but I use it in all my projects as a bootstrap. You can enter any of my project and do make help and get some help. Also make server will launch a server (static server, phoenix, rails, webpack, esbuild, whatever) and print the port. I just use it to wrap language specific tools. This way I can type the 3 same commands to get started in any of my projects as I change tooling and language 3 times a day :P
- CJefferson 4y agoMy biggest disappointment with make, and almost all it’s clones, is that I have to manually write out pre-reqs and targets. My computer knows what files I read and write. Use that to calculate what you should do. Some systems head in this direction — there is “tup”, and a Python program I love called fabricate ( https://github.com/brushtechnology/fabricate https://github.com/brushtechnology/fabricate ), that records what a program reads, then doesn’t rerun it if no input has changed.
- tpoacher 4y agofor gcc there is the -M flag to generate dependencies. similarly, there's nothing stopping you from autogenerating them in some way that you prefer.
- dwheeler 4y agoI used GNU make to manage a data pipeline with a large number of (mostly Python) scripts written by a team. It had to process lots of data using many steps. Make worked very well for this use case. It easily auto-generated dependencies, ran tests, and supported parallel execution. In a tiny fraction of a second it determined what needed to be done, and only did that. We created a small collection that we released as open source software. You can get it here: https://github.com/david-a-wheeler/make-booster https://github.com/david-a-wheeler/make-booster I think a lot of hate on make is due to poor use. If your makefile is complex, refactor it. Auto-generate dependencies (it only takes a few lines in GNU make). And don't use recursive make, that way lies madness. I also think GNU make is the wiser tool; POSIX make lacks too much in many cases.
- WesolyKubeczek 4y ago> Auto-generate dependencies (it only takes a few lines in GNU make) Care to give an example? The manual is quite frustrating to piece this together, when it comes to dynamic dependencies, and god forbid, dynamic targets.
- dwheeler 4y agoCheck out: https://spin.atomicobject.com/2016/08/26/makefile-c-projects/ https://spin.atomicobject.com/2016/08/26/makefile-c-projects... https://make.mad-scientist.net/papers/advanced-auto-dependency-generation/ https://make.mad-scientist.net/papers/advanced-auto-dependen...
- kazinator 4y agoI've contributed patches to Make, such as the support for grouped targets in a regular direct rule (not pattern). I receive the mailing list and sometimes answer questions. Many years ago, I transplanted the process image saving code from GNU Emacs into GNU Make. I keyed that to a new option in make, --dump, which would save the executable image and quit after reading all the makefile material, instead of executing them. The resulting executable would come up with all the Makefiles already in memory, and so dispatch the build very quickly. This was on a project where reading the tree of Makefiles took 30 seconds. If you touched some C++ file in the tree, it would take half a minute to begin to dispatch the command to rebuild it and then link. With the dumped image, it was nearly instant. --- OMG, I just found the code for this. I thought it had been lost for 13 years. Turns out, I have the whole damn SVN database from that folded startup, of the Linux distro and toolchains stuff.
- cmrdporcupine 4y agoThe original mental model behind Make is pretty. At its best it feels like prolog for software construction. Like Horne clauses for compiler invocations on source units. The problem is that the dirty real world of software construction doesn't match this ideal because the tooling is full of OS and compiler and library oddities that require explicit non-declarative interventions. And that's how we ended up with source trees littered with shell scripts and M4 and autoconf and automake and it was a disaster. One of the joys of operating in a net-new ecosystem like Rust or Go is having modern build tools built explicitly to avoid all this. Make filled the "batteries not included" niche for C compilers in the 80s. But the world was too heterogeneous for it to scale nicely for larger, more complicated projects.
- mjd 4y agoYou may love make, but make doesn't love you back.
- coding123 4y agoIn 2029, make will love you back. MakeGPT!
- liampulles 4y agoThe Makefile structure is the real king - the simple idea of targets, dependencies and using the plain old shell is very powerful and quite easy to understand (until you get into some of the recursive, iterative bits at least). Very useful if you have custom CLIs and multiple languages in your repo too.
- xdennis 4y ago> the simple idea of targets That works very well for compiling C. But make is often abused for things that don't match that pattern at all, like a Python project when there are no intermediate targets and they are all .PHONY.
- Izkata 4y agoPython is actually one of the things I've found it to be great with, you can for example have all your "command" recipes depend on the virtualenv directory, and the virtualenv directory depend on your requirements / lock file. That way you don't have to worry about keeping the virtualenv up-to-date, it'll automatically do so any time the requirements file changes next time you try to do something, and it'll skip that step entirely if nothing changed. The only downside is that the last part requires a small addition, the install doesn't change the virtualenv dir's last-modified, so that recipe has to end with "touch $@".
- deleted 4y ago[deleted]
- duped 4y ago> And what really sets you apart, is your reliability. You never let me down or give me any trouble. You always do exactly what I tell you to do, without any fuss or drama (...) I've had so many issues with builds failing when the number of jobs increases, random shit breaking between versions of make, issues with timestamps causing false positives for reconfiguration, and so on - it's really not reliable at all when doing big builds on modern systems where you care about caching, distribution, and parallelism.
- blueflow 4y agoHaving your other comment in mind, is your grievance with Make in general or some specific Makefiles you have to work with?
- duped 4y agoIt's with make - a modern build system should be content addressed and not rely on mtimes to determine if anything is out of date. Add on that many make builds are nondeterministic and you have a recipe for sadness.
- JohnFen 4y agoI love make. I've used dozens of different build systems over the decades, and while each has their own strengths and weaknesses, make remains my favorite.
- tpoacher 4y agoSame. I honestly don't get the hate. I'm inclined to think that people who shiver at its name, either haven't bothered to read the manual and go beyond basic features, or have encountered some pretty shoddy makefiles (made possible by make's power and flexibility in the first place). Or, more likely, both.
- tpoacher 4y agoRandom: did you know that you can write your recipes in python? Or any language for that matter? This is as simple as SHELL = python (technically one also has to set shellflags, but python happens to share the same "execute command" flags as bash)
- Izkata 4y agoIf you're actually going to do this, you'll probably also want: .ONESHELL: This puts the entire recipe into one script, instead of running each line separately, so you can use (for example) variables across lines. It does mean you lose the "abort the recipe immediately if any individual line fails" functionality in make, but for python that's just how it works anyway when an exception is raised.
- tpoacher 4y agoOneshell is indeed a useful thing to keep in mind, but it's mostly when you want to retain state between commands; why would it be more relevant to python than other languages?
- Izkata 4y ago...you used python as the example.
- tpoacher 4y agoYes, and your comment seemed to imply that state between commands in make recipes (and thus use of oneshell) is somehow more important with a python shell than with the default bash shell. And I'm curious why you say this, as I wouldn't have thought it to necessarily be the case, so I thought perhaps you had some nice insight about the use of python as a make shell that I hadn't considered.
- Izkata 4y agoJust because with default (ba)sh recipes it's usually unnecessary, to the point where people are surprised when learn make does this. You're working directly on files so you don't need to carry any state in memory across lines. With python, this isn't really possible - even the most basic "open a file and write a value" spans two lines with state, so you need to write it in an unusual way, use backslashes to escape the newline, or use ONESHELL.
- yawpitch 4y agoNo rule to make target `love'.
- trashtester 4y agoReason: explicit content
- tannhaeuser 4y agomake love not .war
- WesolyKubeczek 4y agoMake is very versatile but unfortunately underappreciated today. But just look at buildroot, it’s a ginormous system made with Makefiles. I only wish for something which works like Make but has a more ergonomic syntax and an ability to define your own functions to determine if a target is up-to-date. And for the love of everything that’s good, stop pigeonholing make into being a tool for compiling C-shaped programs. It’s useful for more than that.
- cout 4y agoI think the closest we have today to "make with better syntax" is ninja. It lacks pattern rules and macros and other fancy features because it is expected that ninja files are built with a code generator, but for a tool that just handles specifying targets, rules, and dependencies, it works great (and as a bonus it is faster than make).
- WesolyKubeczek 4y agoWhich makes ninja not that good enough a replacement, as make has the same problem: it’s so layered in arcane that in most cases you rely on tools to generate Makefiles for you anyway. There are “alternatives” like some Python frameworks, but they are more “command runners” rather than graph resolvers, they lack inbuilt or simple ways to determine that a target is up to date, or they promise paralellism* (with fine print that maybe someday in a future version). One thing make is lacking (arcane stuff aside) is dynamic dependency graph, which could have made it viable for things like javascript transpiling and bundling. Even for C projects which are sorta kinda match made in heaven, you need a “makedepend” tool to traverse your sources.
- gjvc 4y agoAnd for the love of everything that’s good, stop pigeonholing make into being a tool for compiling C-shaped programs. It’s useful for more than that. Absolutely. When you need to transform one set of files into another and have the parallelism taken care of and the cleanup able to be expressed as a target, it's a great tool. One of many one uses on the jobsite every day.
- 4y ago
- colonwqbang 4y agoMake can be a great tool, but it requires extreme amounts of diligence and good taste in order to scale well.
- chasinglogic 4y agoMan I love Make. But recently started a new job and we decided to use Just* and it's been fantastic. I doubt I would use Make again unless I was planning to use it as a real build system (which has been the minority use case of my use cases in the last 5 or so years). * https://github.com/casey/just https://github.com/casey/just
- tpoacher 4y agoI dont get it. Isn't "just" simply a "friendlier make" but only for linear pipelines that could have been simple bash scripts in the first place? (in which case, why not just use a bash script to begin with?)
- cassepipe 4y agoSame, except I do not love make. Love did not grow in the three years I have had to use it for C and C++ building and as a command file. I don't understand the love. There are so many footguns. Nothing you want from it is easy. `make` experts always tell you that your makefile is wrong even when it does what it is intended to and when you ask to see a good makefile, you're given a document you can barely understand even though you thought you had good `make` knowledge. Correct handling of header files needs files generated by the compiler the alternative being recompiling everything when you change one header file. Run I now use `xmake` for building (found love at last) and basic scripts or `just` for command files
- JohnFen 4y agoI love make, personally, because no other system gives me the level of flexibility and power. I use it for all sorts of things that aren't even building code. > There are so many footguns. That's true. That sorta comes with flexibility and power. > `make` experts always tell you that your makefile is wrong even when it does what it is intended to I wouldn't do this. "If it's stupid and works, it's not stupid." > Correct handling of header files needs files generated by the compiler the alternative being recompiling everything when you change one header file. I'm not sure I understand what you mean here.
- 4y ago
- billpg 4y agoI miss makefiles. I had a side-project that included a program that converted a text file into a C# file and I wanted that integrated into the build process so when I hit F5, the system would - if needed - invoke the code-gen program and then recompile the .cs file that got generated. Alas, I couldn't find a good way to do it so I ended up adding pre-build and post-build actions. They sorta work but I have to manually invoke builds occasionally. I asked on stack-overflow if there was a way to do this. https://stackoverflow.com/q/73816110/3568 https://stackoverflow.com/q/73816110/3568 (Short version of answers: No.)
- deleted 4y ago[deleted]
- pjc50 4y agoYou certainly can have tool-generated C# files, because the GRPC protobuf compiler does that and integrates it nicely into VS projects. It's not exactly what I'd call simple, though: https://github.com/grpc/grpc/blob/master/src/csharp/Grpc.Tools/build/_protobuf/Google.Protobuf.Tools.targets https://github.com/grpc/grpc/blob/master/src/csharp/Grpc.Too... so I'm not surprised that people don't try to spend days learning it and making it work. One could probably write a "how to speak MSBuild for people who understand Make" page based on https://learn.microsoft.com/en-us/visualstudio/msbuild/msbuild-concepts?view=vs-2022 https://learn.microsoft.com/en-us/visualstudio/msbuild/msbui... ; basically Make -> MSBuild Item : File (e.g. foo.c) Target : Target (e.g. %.o:%.c) (environment) variable : Property tab-indented shell : Task (usually a C# assembly, although there's a generic "run command" one as well) Free tip for anyone trying to debug MSBuild: https://msbuildlog.com/ https://msbuildlog.com/ . AFAIK there isn't anything quite as good for Make.
- sph 4y agoReads a lot like Stockholm Syndrome. Make is terrible. It's just widespread, not very hard to start with and better than all the alternatives (which often are better, but not as easily found or preinstalled) But it is obtuse for no good reason other than being a piece of software born in a long-gone era of computing. Whoever loves Makefile has never had to debug the compilation process of any non-trivial build system, with three dozen ways to define variables, rules and targets, spread over multiple files. When in doubt, I'll create a Makefile (actually, I use `just` for my personal projects because 99% of the time in 2023 one just needs a task runner). There is much better software to write love letters about.
- blueflow 4y agoI don't understand what you are unhappy about. When maintaining recipes for building packages, i'd always prefer to debug a Makefile where i can insert a `sh -x` than some magic tools that attempts to do everything for me and then fails without observable reason.
- duped 4y ago> than some magic tools that attempts to do everything for me and then fails without observable reason. Ok, but then what about autoconf/autoreconf, aclocal, and automake used to generate makefiles for what seems like the entire GNU ecosystem? At least the "magic" tools that are more modern have documentation. You can print debug a cmake build pretty dang easily, and it at least doesn't pull in the entire user environment implicitly.
- blueflow 4y agoPlease back up the implicit claim of missing documentation.
- JohnFen 4y ago> Ok, but then what about autoconf/autoreconf, aclocal, and automake used to generate makefiles for what seems like the entire GNU ecosystem? I thought we were talking about make, though. Autoconf and the ilk is an entirely different conversation.
- nickelpro 4y agoIf all I want to do is run a compiler over some C files, sure, make is fine. You can love it for that. That's almost never exclusively what I want to do. I want to chainload a dependency manager, I want to download, compile, and link those dependencies. I want to run a code generator and integrate that generated code into the IDE's intellisense without ever letting that generated code out of the build tree. I want to generate a compile_commands.json to communicate to my IDE the relationships between the in-source libraries, executables, dependencies, and generated code. For most build systems it is sufficient to describe the relationships between these components and the rest you get for free. make would require manually specifying every single command, every single tool, every argument. Miserable.
- tipiirai 4y agoOur team uses it to execute node programs. Starts so much faster than npx.
- yamtaddle 4y agoI am not following how Make is a replacement for npx. Npx is just for us lazy command-line users who can't be bothered to make sure a Node package is installed before trying to run a command, right? Make would either have to run npx itself, or you'd have to make sure the package you want to run was installed before running its command, in which case you can skip npx... but that's the same with or without make.
- Izkata 4y agoI don't know about npx specifically, but you can have the command recipe depend on the installed package, then have a first recipe to do the install only if it's missing. I do something like this with python and virtualenvs.
- deepakprakash 4y agoI could never get my head around Make except for very simple commands. Was very recently looking for an alternative for automating frequently used commands for our Elixir app – especially for local dev (without using docker) and came across Taskfile (https://taskfile.dev/ https://taskfile.dev/) and have been liking it quite a bit.
- duckqlz 4y agoI am guessing this was written by ChatGPT. As far as I know, No one likes make. Imagine having a choice between make and cargo and still picking make. ridiculous.
- cout 4y agoI like make. So now you can say that as far as you know, no one likes make except one oddball who replied to you on HN.
- lelanthran 4y agoTwo. Make works better than anything else for what it does. Once your project grows, then sure, other tools is less work, but they still require arcane and obscure knowledge to make thos big things work.
- billpg 4y agoThere's a parallel universe where the designer of make decided that lines starting with any whitespace indicated the commands to run, instead of requiring actual tab characters.
- robinsonb5 4y agoAnd in that universe Python uses braces.