3 ms·

That's a rather dated view of what a build system should be. Most developers these days don't want to have to manually install all of a projects dependencies in

by orclev 9y ago

That's a rather dated view of what a build system should be. Most developers these days don't want to have to manually install all of a projects dependencies in order to build it. More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects, which is go read the docs to find the list of mandatory and optional dependencies, then go look through the distribution package manager to try to match up package names with library names (including a few head scratchers where something is included inside of a differently named package), and finally install the whole mess and hope that the projects config script properly finds them all.

mikegerwitz 9y ago
> That's a rather dated view of what a build system should be The author mentions JavaScript, which has npm as the package manager, which isn't a build system. It can _invoke_ a build system, but the build output produced is separately published to npm, or the build is triggered by e.g. a post-install hook. I use GNU Make with npm. Some use e.g. Grunt. Etc. > More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. Make has been used this way for quite some time. Debian has its reproducible builds project. GNU Guix builds everything in an isolated environment, and even handles multiple versions of multiple packages---if you have five different packages that each depend on five different versions of the same library, then it'll install five different versions of the library and work just fine. > I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects Make is just _part_ of a build system---it isn't necessarily one in itself. Some projects might use it exclusively in their build system, but others use a suite of tools. I use Autoconf (which generates configure) and Automake, which in turn generates a Makefile with all standard targets. If you check out a git repository for one of my JS projects, you run `npm install` to get the dependencies before building. Usually they're runtime dependencies, though, so they're not needed for the build. Other dependencies are detected via the configure script (e.g. I use graphviz for certain project, and the script makes sure it's installed and supports the feature I need).
- orclev 9y ago
  For the most part, JavaScript doesn't need a build system, so from a practical standpoint npm is its build system. Most JS projects shouldn't require anything more complicated than npm install to put them in the proper location. Now you can argue for something like a minifier, and things like Typescript add a wrinkle (as it actually does need a build system), but all that should still be able to be handled by npm. I'm curious how you would use Make to install dependencies. I'm aware of Guix, but that's probably an extreme example, most people probably aren't willing to sandbox literally their entire OS install, plus it's entirely unreasonable to require a particular OS in order to handle dependencies of your project. It shouldn't matter what OS I'm using (within reason), the project should have an automated way to install any dependencies it requires. Here's the thing with Make, even if used only as part of a build system, it's both too complicated, and not complicated enough at the same time. The makefile format when used very sparingly isn't too bad, but once you pass needing more than 2 or 3 rules it gets unwieldy, and on any significantly sized project it will need more than 2 or 3 rules. Using autoconf and automake just proves the point, now you're using a build system to build your build system. Autoconf in particular is way too complicated, and an absolute nightmare to try to extend or customize. Instead of using a tiny little sliver of make as essentially build system glue to connect the actual build tools together, why not just use slightly more powerful build tools and get rid of make all together?
  - rekado 9y ago
    You wrote: > I'm aware of Guix, but that's probably an extreme example, most people probably aren't willing to sandbox literally their entire OS install, plus it's entirely unreasonable to require a particular OS in order to handle dependencies of your project. It shouldn't matter what OS I'm using (within reason), the project should have an automated way to install any dependencies it requires. Guix is just a package manager, not an operating system. The Guix System Distribution, on the other hand, is a GNU system. You can use Guix on any GNU+Linux system; you do not need GuixSD for that.
  - jolson88 9y ago
    Author here. Great feedback! Thanks for taking the time to share it :). If I were creating a JavaScript library package that I was just going to push up to npm, I don't think I'd use Make either. yes, Typescript adds a wrinkle, but like you mention, that can all be handled by npm scripts as well. If it was just a JavaScript library, I'd definitely just use npm scripts and call out to eslint, and other tools just fine and not be any less happy. However, big front-end applications I think are starting to become more and more complex and can benefit from this. Part of the reason is perhaps more because of the lack-of-functionality around incremental builds in the compilers themselves (thinking less/sass/minify/uglify/etc., this is where I've seen people usually fall back to using Gulp). As the application becomes larger, I think the lack of incremental build becomes more and more of a tax on the developer because the builds become longer and longer (I've worked on several projects like that). I think perhaps the biggest impact on this project where Make is used had more to do with the modular project architecture than anything else. If it weren't for that, I'm not sure I'd ever even think of Make. And frankly, I think a modular project structure in a single github repository like this project uses is most likely not the right solution (or even a good idea) for a vast number of JavaScript/TypeScript projects out there. When the article talks about dependencies, it is more talking about dependencies within the project itself (due to the modular architecture). Before I switched it over to Make, it was using a combination of preinstall, postinstall, and custom "prestart" scripts to do everything in the right order because it wasn't as simple as using `npm link` or doing a single npm install in the root of the project. These various scripts become a total mishmash of different approaches and it became increasingly difficult to understand and visualize what was being built when and in what order. It wasn't uncommon to run into issues as well where one project would be `npm install`'d before it's dependencies were actually ready to be used (since they hadn't been compiled and packaged yet). With that said, Make in this project doesn't install dependencies in the sense that I think you are discussing. This project is most definitely a Node.js project and uses npm for all that stuff. It's just within the project itself, there are sub-projects that are NPM packages themselves that can be built and deployed separately. That's where it began to fall down. But the project still most definitely uses "npm install", "npm start", and such. It's just the build component of issuing "npm run build" just shells out to "make -r" to build everything in the right order, in an incremental fashion, and to get it where it needs to be. There are many reasons behind this modular architecture that I never went into in the post for good reasons. And like I said, I would be sad if lots of projects thought all-the-sudden it was a good idea to do it :P. All that said, over the last 10+ years, I've grown increasingly tired of new frameworks coming out every 18 months (or less) that simply re-invent the wheel, but do it in a different way. I think there are plenty of situations where Make would perfectly suffice yet people immediately pull in a code-based build tool with tons of dependencies because it's what "everybody is using now."

by orclev 9y ago

mikegerwitz 9y ago

> That's a rather dated view of what a build system should be The author mentions JavaScript, which has npm as the package manager, which isn't a build system. It can _invoke_ a build system, but the build output produced is separately published to npm, or the build is triggered by e.g. a post-install hook. I use GNU Make with npm. Some use e.g. Grunt. Etc. > More importantly, there's also been a push towards wherever possible installing dependencies into sandboxes to try to prevent things like DLL hell, which isn't possible if you install dependencies at the OS level. Make has been used this way for quite some time. Debian has its reproducible builds project. GNU Guix builds everything in an isolated environment, and even handles multiple versions of multiple packages---if you have five different packages that each depend on five different versions of the same library, then it'll install five different versions of the library and work just fine. > I much rather just check out a project and run its build system to fetch all the dependencies instead of what we used to have to do with C projects Make is just _part_ of a build system---it isn't necessarily one in itself. Some projects might use it exclusively in their build system, but others use a suite of tools. I use Autoconf (which generates configure) and Automake, which in turn generates a Makefile with all standard targets. If you check out a git repository for one of my JS projects, you run `npm install` to get the dependencies before building. Usually they're runtime dependencies, though, so they're not needed for the build. Other dependencies are detected via the configure script (e.g. I use graphviz for certain project, and the script makes sure it's installed and supports the feature I need).

orclev 9y ago
For the most part, JavaScript doesn't need a build system, so from a practical standpoint npm is its build system. Most JS projects shouldn't require anything more complicated than npm install to put them in the proper location. Now you can argue for something like a minifier, and things like Typescript add a wrinkle (as it actually does need a build system), but all that should still be able to be handled by npm. I'm curious how you would use Make to install dependencies. I'm aware of Guix, but that's probably an extreme example, most people probably aren't willing to sandbox literally their entire OS install, plus it's entirely unreasonable to require a particular OS in order to handle dependencies of your project. It shouldn't matter what OS I'm using (within reason), the project should have an automated way to install any dependencies it requires. Here's the thing with Make, even if used only as part of a build system, it's both too complicated, and not complicated enough at the same time. The makefile format when used very sparingly isn't too bad, but once you pass needing more than 2 or 3 rules it gets unwieldy, and on any significantly sized project it will need more than 2 or 3 rules. Using autoconf and automake just proves the point, now you're using a build system to build your build system. Autoconf in particular is way too complicated, and an absolute nightmare to try to extend or customize. Instead of using a tiny little sliver of make as essentially build system glue to connect the actual build tools together, why not just use slightly more powerful build tools and get rid of make all together?
- rekado 9y ago
  You wrote: > I'm aware of Guix, but that's probably an extreme example, most people probably aren't willing to sandbox literally their entire OS install, plus it's entirely unreasonable to require a particular OS in order to handle dependencies of your project. It shouldn't matter what OS I'm using (within reason), the project should have an automated way to install any dependencies it requires. Guix is just a package manager, not an operating system. The Guix System Distribution, on the other hand, is a GNU system. You can use Guix on any GNU+Linux system; you do not need GuixSD for that.
- jolson88 9y ago
  Author here. Great feedback! Thanks for taking the time to share it :). If I were creating a JavaScript library package that I was just going to push up to npm, I don't think I'd use Make either. yes, Typescript adds a wrinkle, but like you mention, that can all be handled by npm scripts as well. If it was just a JavaScript library, I'd definitely just use npm scripts and call out to eslint, and other tools just fine and not be any less happy. However, big front-end applications I think are starting to become more and more complex and can benefit from this. Part of the reason is perhaps more because of the lack-of-functionality around incremental builds in the compilers themselves (thinking less/sass/minify/uglify/etc., this is where I've seen people usually fall back to using Gulp). As the application becomes larger, I think the lack of incremental build becomes more and more of a tax on the developer because the builds become longer and longer (I've worked on several projects like that). I think perhaps the biggest impact on this project where Make is used had more to do with the modular project architecture than anything else. If it weren't for that, I'm not sure I'd ever even think of Make. And frankly, I think a modular project structure in a single github repository like this project uses is most likely not the right solution (or even a good idea) for a vast number of JavaScript/TypeScript projects out there. When the article talks about dependencies, it is more talking about dependencies within the project itself (due to the modular architecture). Before I switched it over to Make, it was using a combination of preinstall, postinstall, and custom "prestart" scripts to do everything in the right order because it wasn't as simple as using `npm link` or doing a single npm install in the root of the project. These various scripts become a total mishmash of different approaches and it became increasingly difficult to understand and visualize what was being built when and in what order. It wasn't uncommon to run into issues as well where one project would be `npm install`'d before it's dependencies were actually ready to be used (since they hadn't been compiled and packaged yet). With that said, Make in this project doesn't install dependencies in the sense that I think you are discussing. This project is most definitely a Node.js project and uses npm for all that stuff. It's just within the project itself, there are sub-projects that are NPM packages themselves that can be built and deployed separately. That's where it began to fall down. But the project still most definitely uses "npm install", "npm start", and such. It's just the build component of issuing "npm run build" just shells out to "make -r" to build everything in the right order, in an incremental fashion, and to get it where it needs to be. There are many reasons behind this modular architecture that I never went into in the post for good reasons. And like I said, I would be sad if lots of projects thought all-the-sudden it was a good idea to do it :P. All that said, over the last 10+ years, I've grown increasingly tired of new frameworks coming out every 18 months (or less) that simply re-invent the wheel, but do it in a different way. I think there are plenty of situations where Make would perfectly suffice yet people immediately pull in a code-based build tool with tons of dependencies because it's what "everybody is using now."