5 ms·

If you compare Makefiles to the JavaScript ecosystem it will do so favorably, but many things will. Make is a good build system but it is a shitty deployment s

by Iv 9y ago

If you compare Makefiles to the JavaScript ecosystem it will do so favorably, but many things will. Make is a good build system but it is a shitty deployment system. Yes, you can do everything you want if you put enough effort in it as it is a complete scripting system, something many alternatives are not. It does not mean that it is a good idea to do so. Recently a client made me begrudgingly try Maven. 'Yet another make system' I thought. Yup, pretty standard one: 'mvn install' ,I'll compile everything and will also do something make has never done for me: install dependencies. Sure, it is doable to do this with a makefile. With a lot of efforts multiplied by the number of platforms you target. In Maven you just add repos and dependencies to your pom file. So yes, there recent build systems that are worse than Make but it does not mean that make is the perfect system with all the features we need. It is minimalistic. It works. But it should not be your only option.

mikegerwitz 9y ago
Make isn't intended to be used for dependency management. With traditional `./configure && make && make install`, dependencies are detected by the configure script, or manually provided by the user if they have custom library paths. Dependencies are handled by your package manager or something else (be it submodules in git, custom scripts, etc). There's a separation of concerns. For generating those configure files, Autoconf is extremely powerful (and really not that complicated once you have a basic understanding). If you use Automake, you get all the standard targets (including `make dist`, which will build your distribution archive, as is being done manually in this article) generated for you.
- orclev 9y ago
  That's a rather dated view of what a build system should be. Most developers these days don't want to have to manually install all of a projects dependencies in order to build it. More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects, which is go read the docs to find the list of mandatory and optional dependencies, then go look through the distribution package manager to try to match up package names with library names (including a few head scratchers where something is included inside of a differently named package), and finally install the whole mess and hope that the projects config script properly finds them all.
  - mikegerwitz 9y ago
    > That's a rather dated view of what a build system should be The author mentions JavaScript, which has npm as the package manager, which isn't a build system. It can _invoke_ a build system, but the build output produced is separately published to npm, or the build is triggered by e.g. a post-install hook. I use GNU Make with npm. Some use e.g. Grunt. Etc. > More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. Make has been used this way for quite some time. Debian has its reproducible builds project. GNU Guix builds everything in an isolated environment, and even handles multiple versions of multiple packages---if you have five different packages that each depend on five different versions of the same library, then it'll install five different versions of the library and work just fine. > I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects Make is just _part_ of a build system---it isn't necessarily one in itself. Some projects might use it exclusively in their build system, but others use a suite of tools. I use Autoconf (which generates configure) and Automake, which in turn generates a Makefile with all standard targets. If you check out a git repository for one of my JS projects, you run `npm install` to get the dependencies before building. Usually they're runtime dependencies, though, so they're not needed for the build. Other dependencies are detected via the configure script (e.g. I use graphviz for certain project, and the script makes sure it's installed and supports the feature I need).
    - orclev 9y ago
      For the most part, JavaScript doesn't need a build system, so from a practical standpoint npm is its build system. Most JS projects shouldn't require anything more complicated than npm install to put them in the proper location. Now you can argue for something like a minifier, and things like Typescript add a wrinkle (as it actually does need a build system), but all that should still be able to be handled by npm. I'm curious how you would use Make to install dependencies. I'm aware of Guix, but that's probably an extreme example, most people probably aren't willing to sandbox literally their entire OS install, plus it's entirely unreasonable to require a particular OS in order to handle dependencies of your project. It shouldn't matter what OS I'm using (within reason), the project should have an automated way to install any dependencies it requires. Here's the thing with Make, even if used only as part of a build system, it's both too complicated, and not complicated enough at the same time. The makefile format when used very sparingly isn't too bad, but once you pass needing more than 2 or 3 rules it gets unwieldy, and on any significantly sized project it will need more than 2 or 3 rules. Using autoconf and automake just proves the point, now you're using a build system to build your build system. Autoconf in particular is way too complicated, and an absolute nightmare to try to extend or customize. Instead of using a tiny little sliver of make as essentially build system glue to connect the actual build tools together, why not just use slightly more powerful build tools and get rid of make all together?
- athenot 9y ago
  Yes Make does one thing and one thing well. Having said that, if you really want to manage dependencies with Make, it's still trivial to do a curl/yum/rpm install when some component is missing. I've done that in the form of a `make deps` pseudo-target, to prevent doing anything that the user doesn't expect.
  - mikegerwitz 9y ago
    Yes having a separate target that isn't automatically invoked unexpectedly is important---the user doesn't expect network requests during build traditionally. Environments that build in isolation with no network access would fail as well.
  - Iv 9y ago
    > it's still trivial to do a curl/yum/rpm install when some component is missing. Oh yeah? Which one? curl, yum, rpm, apt-get? All of them? Which revisions are you targeting? Which package names? And that only covers windows. A lot of apps need to be deployable under windows and OSX too. If you start being even slightly cross-platform, you soon have your own build system.
- deleted 9y ago
  [deleted]
- Iv 9y ago
  Well ./configure does not install dependencies either. Even worse: it fails at the first missing dependency instead of listing them. Making an automake project to compile is actually a long and manual process. Just today I had to compile an older version of gimp, just a 2 years old one. Configure. Fails. Apt-get this. Configure. Fails. Apt-get that. Oh, no, configure doesn't want obscure-package4, it wants obscure-package2, that has been removed from your distribution's repos. Arrrrrr! Truth be told, I just rebooted on windows and dowloaded a binary from an archive. With maven, I can just tell it which release I want and it just fetches it and all its dependencies, compiles them and installs them. Full auto. If I add a lib in my project, it adds it automatically and the intellisense works, the source navigation works. Make is years behind.
  - mikegerwitz 9y ago
    > Even worse: it fails at the first missing dependency instead of listing them. Yes, this is a fair criticism. Fortunately, many projects state their dependencies. But not all do. If I'm already on a Debian system that has another version of the package, `apt-get build-dep' is very helpful to get most of the way there. Of course this is completely outside Autoconf. Whether it fails immediately or not depends on how the author writes `configure.ac'. Usually `AC_MSG_ERROR' is used right away, because it's convenient. Instead, some authors choose to set a flag and fail at the end.
  - JdeBP 9y ago
    Why wasn't apt-get build-dep gimp or aptitude build-depends gimp the easy answer?
orclev 9y ago
That last bit is really key. Nearly every modern build system is also a dependency management system, something that really isn't possible using Make. There have been tools introduced to address that particular need for C/C++, but they integrate poorly if at all with the build system, so it turns all your builds into (at least initially) three step processes of run dependency installer, run config tool to figure out exactly which flavor of C compiler/linker/etc. to invoke, run make to actually build the thing. It's also possible to do some truly awful things with make. I once worked on an embedded device that the SDK provided by the OEM was written using a truly horrific makefile system. There were no less than 80 makefiles all tied together to build an entire system image, including the building of the linux kernel, all the bintools, and a bunch of other custom stuff the OEM provided. Trying to understand, much less modify any part of that build system was an absolute nightmare, and is one of the few times I've actually wished they had used shell scripts instead of make for some of that stuff. As for Maven, it was quite a pioneer in its day, but it hasn't aged well, in particular being written in XML which just adds needless verbosity. For a more modern take, look at Gradle, which provides everything that Maven does (and uses mavens dependency system), but is a lot more flexible because it's a DSL for project building instead of a declarative system like Maven.
- snuxoll 9y ago
  I think the declarative model is why I prefer Maven over any other build system I've worked with, yeah XML is a little noisy but if it's really that big of a deal there's the polyglot plugin. Maven POM's are pretty easy to figure out once you've worked with them for a while, there's a learning curve when you start adding plugins and have to get at least a basic understanding of build phases - but a large chunk of projects need nothing more than the default plugin bindings, they just add their dependencies and go off to the races.
- rleigh 9y ago
  > Nearly every modern build system is also a dependency management system, something that really isn't possible using Make The BSD Ports tree provides exactly this functionality. All package dependencies provided via standard make variables and rules, enabling recursive resolution of all package dependencies.
abritinthebay 9y ago
Eh, different strokes. Every time I have to use Makefiles I yearn for the comparative flexibility and simplicity of the JS ecosystem. Make files just seem good to users of the due to experience. They work, but let’s not pretend they’re easy to work with.
- stephenr 9y ago
  > simplicity of the JS ecosystem [citation needed]
  - abritinthebay 9y ago
    Citation: I work with it every day. You can make it complex but that’s true of everything. It’s not hard or complex to get working initially, or maintain, which is not true of makefiles
    - stephenr 9y ago
      > It’s not hard or complex to get working initially, or maintain, which is not true of makefiles Again, [citation needed]. I use Makefiles in a variety of projects: - building Vagrant base boxes - building environment-specific configuration for LAMP-ish CRUD apps - testing and building a range of shell-script based packages Does Make have some warts? Sure, everything does. I'm not claiming it's perfect. But it's a far sight clearer - to me at least - what's going on with a Makefile. Makefiles also don't need to be replaced by something new in 6 months because the entire community has decided that its time for a new tool to do basically the same thing, solve 1 problem with the old tool and introduce 5 new ones.
dozzie 9y ago
> Make is a good build system but it is a shitty deployment system. Because you really should not deploy things with your build system. It's the today's tooling that got this backward. For deployment you have much saner tools, like package systems.
- mst 9y ago
  But only 'git pull' is web scale!!!! (sigh)

by Iv 9y ago

mikegerwitz 9y ago

Make isn't intended to be used for dependency management. With traditional `./configure && make && make install`, dependencies are detected by the configure script, or manually provided by the user if they have custom library paths. Dependencies are handled by your package manager or something else (be it submodules in git, custom scripts, etc). There's a separation of concerns. For generating those configure files, Autoconf is extremely powerful (and really not that complicated once you have a basic understanding). If you use Automake, you get all the standard targets (including `make dist`, which will build your distribution archive, as is being done manually in this article) generated for you.

orclev 9y ago
That's a rather dated view of what a build system should be. Most developers these days don't want to have to manually install all of a projects dependencies in order to build it. More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects, which is go read the docs to find the list of mandatory and optional dependencies, then go look through the distribution package manager to try to match up package names with library names (including a few head scratchers where something is included inside of a differently named package), and finally install the whole mess and hope that the projects config script properly finds them all.
- mikegerwitz 9y ago
  > That's a rather dated view of what a build system should be The author mentions JavaScript, which has npm as the package manager, which isn't a build system. It can _invoke_ a build system, but the build output produced is separately published to npm, or the build is triggered by e.g. a post-install hook. I use GNU Make with npm. Some use e.g. Grunt. Etc. > More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. Make has been used this way for quite some time. Debian has its reproducible builds project. GNU Guix builds everything in an isolated environment, and even handles multiple versions of multiple packages---if you have five different packages that each depend on five different versions of the same library, then it'll install five different versions of the library and work just fine. > I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects Make is just _part_ of a build system---it isn't necessarily one in itself. Some projects might use it exclusively in their build system, but others use a suite of tools. I use Autoconf (which generates configure) and Automake, which in turn generates a Makefile with all standard targets. If you check out a git repository for one of my JS projects, you run `npm install` to get the dependencies before building. Usually they're runtime dependencies, though, so they're not needed for the build. Other dependencies are detected via the configure script (e.g. I use graphviz for certain project, and the script makes sure it's installed and supports the feature I need).
  - orclev 9y ago
    For the most part, JavaScript doesn't need a build system, so from a practical standpoint npm is its build system. Most JS projects shouldn't require anything more complicated than npm install to put them in the proper location. Now you can argue for something like a minifier, and things like Typescript add a wrinkle (as it actually does need a build system), but all that should still be able to be handled by npm. I'm curious how you would use Make to install dependencies. I'm aware of Guix, but that's probably an extreme example, most people probably aren't willing to sandbox literally their entire OS install, plus it's entirely unreasonable to require a particular OS in order to handle dependencies of your project. It shouldn't matter what OS I'm using (within reason), the project should have an automated way to install any dependencies it requires. Here's the thing with Make, even if used only as part of a build system, it's both too complicated, and not complicated enough at the same time. The makefile format when used very sparingly isn't too bad, but once you pass needing more than 2 or 3 rules it gets unwieldy, and on any significantly sized project it will need more than 2 or 3 rules. Using autoconf and automake just proves the point, now you're using a build system to build your build system. Autoconf in particular is way too complicated, and an absolute nightmare to try to extend or customize. Instead of using a tiny little sliver of make as essentially build system glue to connect the actual build tools together, why not just use slightly more powerful build tools and get rid of make all together?
athenot 9y ago
Yes Make does one thing and one thing well. Having said that, if you really want to manage dependencies with Make, it's still trivial to do a curl/yum/rpm install when some component is missing. I've done that in the form of a `make deps` pseudo-target, to prevent doing anything that the user doesn't expect.
- mikegerwitz 9y ago
  Yes having a separate target that isn't automatically invoked unexpectedly is important---the user doesn't expect network requests during build traditionally. Environments that build in isolation with no network access would fail as well.
- Iv 9y ago
  > it's still trivial to do a curl/yum/rpm install when some component is missing. Oh yeah? Which one? curl, yum, rpm, apt-get? All of them? Which revisions are you targeting? Which package names? And that only covers windows. A lot of apps need to be deployable under windows and OSX too. If you start being even slightly cross-platform, you soon have your own build system.
deleted 9y ago
[deleted]
Iv 9y ago
Well ./configure does not install dependencies either. Even worse: it fails at the first missing dependency instead of listing them. Making an automake project to compile is actually a long and manual process. Just today I had to compile an older version of gimp, just a 2 years old one. Configure. Fails. Apt-get this. Configure. Fails. Apt-get that. Oh, no, configure doesn't want obscure-package4, it wants obscure-package2, that has been removed from your distribution's repos. Arrrrrr! Truth be told, I just rebooted on windows and dowloaded a binary from an archive. With maven, I can just tell it which release I want and it just fetches it and all its dependencies, compiles them and installs them. Full auto. If I add a lib in my project, it adds it automatically and the intellisense works, the source navigation works. Make is years behind.
- mikegerwitz 9y ago
  > Even worse: it fails at the first missing dependency instead of listing them. Yes, this is a fair criticism. Fortunately, many projects state their dependencies. But not all do. If I'm already on a Debian system that has another version of the package, `apt-get build-dep' is very helpful to get most of the way there. Of course this is completely outside Autoconf. Whether it fails immediately or not depends on how the author writes `configure.ac'. Usually `AC_MSG_ERROR' is used right away, because it's convenient. Instead, some authors choose to set a flag and fail at the end.
- JdeBP 9y ago
  Why wasn't apt-get build-dep gimp or aptitude build-depends gimp the easy answer?

orclev 9y ago

That last bit is really key. Nearly every modern build system is also a dependency management system, something that really isn't possible using Make. There have been tools introduced to address that particular need for C/C++, but they integrate poorly if at all with the build system, so it turns all your builds into (at least initially) three step processes of run dependency installer, run config tool to figure out exactly which flavor of C compiler/linker/etc. to invoke, run make to actually build the thing. It's also possible to do some truly awful things with make. I once worked on an embedded device that the SDK provided by the OEM was written using a truly horrific makefile system. There were no less than 80 makefiles all tied together to build an entire system image, including the building of the linux kernel, all the bintools, and a bunch of other custom stuff the OEM provided. Trying to understand, much less modify any part of that build system was an absolute nightmare, and is one of the few times I've actually wished they had used shell scripts instead of make for some of that stuff. As for Maven, it was quite a pioneer in its day, but it hasn't aged well, in particular being written in XML which just adds needless verbosity. For a more modern take, look at Gradle, which provides everything that Maven does (and uses mavens dependency system), but is a lot more flexible because it's a DSL for project building instead of a declarative system like Maven.

snuxoll 9y ago
I think the declarative model is why I prefer Maven over any other build system I've worked with, yeah XML is a little noisy but if it's really that big of a deal there's the polyglot plugin. Maven POM's are pretty easy to figure out once you've worked with them for a while, there's a learning curve when you start adding plugins and have to get at least a basic understanding of build phases - but a large chunk of projects need nothing more than the default plugin bindings, they just add their dependencies and go off to the races.
rleigh 9y ago
> Nearly every modern build system is also a dependency management system, something that really isn't possible using Make The BSD Ports tree provides exactly this functionality. All package dependencies provided via standard make variables and rules, enabling recursive resolution of all package dependencies.

abritinthebay 9y ago

Eh, different strokes. Every time I have to use Makefiles I yearn for the comparative flexibility and simplicity of the JS ecosystem. Make files just seem good to users of the due to experience. They work, but let’s not pretend they’re easy to work with.

stephenr 9y ago
> simplicity of the JS ecosystem [citation needed]
- abritinthebay 9y ago
  Citation: I work with it every day. You can make it complex but that’s true of everything. It’s not hard or complex to get working initially, or maintain, which is not true of makefiles
  - stephenr 9y ago
    > It’s not hard or complex to get working initially, or maintain, which is not true of makefiles Again, [citation needed]. I use Makefiles in a variety of projects: - building Vagrant base boxes - building environment-specific configuration for LAMP-ish CRUD apps - testing and building a range of shell-script based packages Does Make have some warts? Sure, everything does. I'm not claiming it's perfect. But it's a far sight clearer - to me at least - what's going on with a Makefile. Makefiles also don't need to be replaced by something new in 6 months because the entire community has decided that its time for a new tool to do basically the same thing, solve 1 problem with the old tool and introduce 5 new ones.

dozzie 9y ago

> Make is a good build system but it is a shitty deployment system. Because you really should not deploy things with your build system. It's the today's tooling that got this backward. For deployment you have much saner tools, like package systems.

mst 9y ago
But only 'git pull' is web scale!!!! (sigh)