5 ms·

How else would you add/manage external dependencies for your project? Do you manually install them on your local host? That's not reproducible and doesn't supp

How else would you add/manage external dependencies for your project? Do you manually install them on your local host? That's not reproducible and doesn't support multiple versions for different projects. Do you use an uber build system like Yocto to wrap your CMake build, and let it pull the packages? Do you manually copy all the external deps into your own project repo, and then manually keep them updated with the latest changes?

krapht 5y ago
I think parent is saying to use a package manager.
- dagmx 5y ago
  That would be the same issue though, just moved to a different spot
  - nerdponx 5y ago
    Not necessarily. Package managers can cache things, especially if you're fetching the same version over and over. Also, separation of concerns is a good thing IMO.
    - howinteresting 5y ago
      Cargo etc also cache dependencies in a central location. "Separation of concerns" is ill-defined — you can use cargo as just a package manager, just a build system, or both. Most people use it as both but there are definitely places (e.g. bigcos) that use just one of the aspects of cargo.
  - gspr 5y ago
    Separation of tasks. The build system builds. Another system (like a package manager, if you are so inclined, or not if not) manages packages.
    - howinteresting 5y ago
      Systems that combine the two generally let you use just one or the other. For example, cargo fetch just acts as the package manager. cargo build can be configured (through vendoring or just not using any external dependencies) to not perform any package management. What is missing is a modern cross-language, cross-platform package manager, I agree. But there are really good reasons for that, chief among them being that I don't think it's possible to define the scope of such a thing in a coherent fashion.
      - gspr 5y ago
        > Systems that combine the two generally let you use just one or the other. For example, cargo fetch just acts as the package manager. cargo build can be configured (through vendoring or just not using any external dependencies) to not perform any package management. Indeed, and I'm still highly skeptical. Learning Rust has been an amazingly rare sequence of "wow that's perfect" uttered from my mouth – except for Cargo. I couldn't disable the networking functionality of Cargo fast enough!
- gspr 5y ago
  Yes. Or not, if it's not suited for the task. But it seems insane to me to shoehorn this into the build system. I don't even understand why the latter should contain networking code at all.
  - jcelerier 5y ago
    Let me guess, you also don't like that systemd is a system manager
enriquto 5y ago
> Do you manually copy all the external deps into your own project repo, and then manually keep them updated with the latest changes? That is the saner option among the ones that you mention. The sanest, of course, is to not update your dependencies unless really necessary, to avoid introducing new bugs.
- jasonpeacock 5y ago
  > The sanest, of course, is to not update your dependencies unless really necessary, to avoid introducing new bugs. So instead you live with all the existing (and unknown) bugs and security issues? And when you finally do update those frozen dependencies it becomes a nightmare because of all the drift, not just in that package but also its dependencies. You should be updating regularly; if you're concerned about stability then stay a release or two behind latest but that's not guaranteed to more stable than latest. Too often I see people take the approach you're espousing to only suffer more later when they are forced to take an update, and then they become so scared of future updates that they never update again, manually backporting patches, and creating a nightmare legacy application.
- jcelerier 5y ago
  > The sanest, of course, is to not update your dependencies unless really necessary, to avoid introducing new bugs My experience over the 40 or so packages that I regularly use is that updates always have less bugs that old versions though. Even for my system, I build most of the software I use from git head and it just works better than whatever's in Ubuntu or Debian on average
bluGill 5y ago
One of the above, depending on my current situation each of the above (and a few others you didn't mention) make the most sense. That is why the cmake for a project shouldn't pull in dependencies, it should just build with whatever happens to be there or throw an error if something is missing. Now there do exist "uber build systems" that are written in cmake. I'm fine with those existing (though I would probably not chose cmake to write it in, that is the author's choice). However when building a project you should not force a method to get dependencies on your users. The package management is a system level concern and you should not mess with it. Note that I just put a lot of hate on systems like Cargo, pip, or hackage. Or google's pull everything into the one repository. Dependencies belong to tools like apt, rpm, yocto, ,snap, conan (when not building packages to install) - Microsoft/Apple are deficient for not providing them. Your program should just attempt to build with what it is given, not try to subvert my system - whatever it is. Yes I know it makes your life easier to not have to deal with more than one [possibly buggy] version of a dependency, but it makes my life worse.
- joshuamorton 5y ago
  > The package management is a system level concern and you should not mess with it. This makes anything approaching a hermetic/deterministic/reproducible build basically impossible. That's not good. How do I get consistent builds across systems if I have to hope that some (set of, if I'm building on multiple OSes) external dependencies give me the correct version of the package I'm looking for, especially since many of the tools you listed don't, for example, support multiple simultaneous versions of a particular package. I'm also confused by precisely what you mean by package management. Tools like pip cargo and bazel etc. Don't manage compiled binaries, but uncompiled sources. They deal with code dependencies, not dynamic library dependencies. Installing a source file via apt doesn't make a whole lot of sense.
  - daemin 5y ago
    Have a script which downloads, installs, and builds dependencies. Have a script which generates build project/solution/script files for the actual build. These scripts can use whatever tools your organisation is most comfortable with, up to and including a mono repo.
gspr 5y ago
> How else would you add/manage external dependencies for your project? Either using a package manager, or doing it "manually" (like you suggest) – a process which can of course also be automated outside of the build system. > Do you manually install them on your local host? That's not reproducible and doesn't support multiple versions for different projects. Of course that's reproducible and supports multiple versions. Any build system worth its salt can be directed to look for dependencies at specific locations. Then multiple versions of things can be installed at multiple locations. This is literally how things have been done for ages. > Do you manually copy all the external deps into your own project repo, and then manually keep them updated with the latest changes? I abhor bundling dependencies, so I'm probably not the right person to ask, but supposing that you do want to bundle dependencies: yes (a process that can be scripted just as easily as it can be handed off to the build system!)
- Joker_vD 5y ago
  Why do I want the package manager to fetch and install dependencies for a project? I don't want to put all kinds of random junk into my /usr/lib before running "make" in the project's directory stops failing. And I've seen makefiles with "sudo apt install libqt5core libxml2-dev ..." in them and I didn't like that.
  - gspr 5y ago
    > Why do I want the package manager to fetch and install dependencies for a project? You don't have to do that. If you separate the task of building from the task of managing software, you can mix and match solutions for each separate task as you wish. > And I've seen makefiles with "sudo apt install libqt5core libxml2-dev ..." in them and I didn't like that. That's obviously terrible too.
- howinteresting 5y ago
  Build systems that manage dependencies are that automation you're talking about, except they're easy-to-reuse across many projects. That's incredibly valuable. Being able to ship software, especially across many platforms is much more important than maintaining some sort of purist separation between build system and package manager. Your view of affairs, after being dominant for decades, is finally receding into the background, and the world is a better place for it.

krapht 5y ago

I think parent is saying to use a package manager.

dagmx 5y ago
That would be the same issue though, just moved to a different spot
- nerdponx 5y ago
  Not necessarily. Package managers can cache things, especially if you're fetching the same version over and over. Also, separation of concerns is a good thing IMO.
  - howinteresting 5y ago
    Cargo etc also cache dependencies in a central location. "Separation of concerns" is ill-defined — you can use cargo as just a package manager, just a build system, or both. Most people use it as both but there are definitely places (e.g. bigcos) that use just one of the aspects of cargo.
- gspr 5y ago
  Separation of tasks. The build system builds. Another system (like a package manager, if you are so inclined, or not if not) manages packages.
  - howinteresting 5y ago
    Systems that combine the two generally let you use just one or the other. For example, cargo fetch just acts as the package manager. cargo build can be configured (through vendoring or just not using any external dependencies) to not perform any package management. What is missing is a modern cross-language, cross-platform package manager, I agree. But there are really good reasons for that, chief among them being that I don't think it's possible to define the scope of such a thing in a coherent fashion.
    - gspr 5y ago
      > Systems that combine the two generally let you use just one or the other. For example, cargo fetch just acts as the package manager. cargo build can be configured (through vendoring or just not using any external dependencies) to not perform any package management. Indeed, and I'm still highly skeptical. Learning Rust has been an amazingly rare sequence of "wow that's perfect" uttered from my mouth – except for Cargo. I couldn't disable the networking functionality of Cargo fast enough!
gspr 5y ago
Yes. Or not, if it's not suited for the task. But it seems insane to me to shoehorn this into the build system. I don't even understand why the latter should contain networking code at all.
- jcelerier 5y ago
  Let me guess, you also don't like that systemd is a system manager

enriquto 5y ago

> Do you manually copy all the external deps into your own project repo, and then manually keep them updated with the latest changes? That is the saner option among the ones that you mention. The sanest, of course, is to not update your dependencies unless really necessary, to avoid introducing new bugs.

jasonpeacock 5y ago
> The sanest, of course, is to not update your dependencies unless really necessary, to avoid introducing new bugs. So instead you live with all the existing (and unknown) bugs and security issues? And when you finally do update those frozen dependencies it becomes a nightmare because of all the drift, not just in that package but also its dependencies. You should be updating regularly; if you're concerned about stability then stay a release or two behind latest but that's not guaranteed to more stable than latest. Too often I see people take the approach you're espousing to only suffer more later when they are forced to take an update, and then they become so scared of future updates that they never update again, manually backporting patches, and creating a nightmare legacy application.
jcelerier 5y ago
> The sanest, of course, is to not update your dependencies unless really necessary, to avoid introducing new bugs My experience over the 40 or so packages that I regularly use is that updates always have less bugs that old versions though. Even for my system, I build most of the software I use from git head and it just works better than whatever's in Ubuntu or Debian on average

bluGill 5y ago

One of the above, depending on my current situation each of the above (and a few others you didn't mention) make the most sense. That is why the cmake for a project shouldn't pull in dependencies, it should just build with whatever happens to be there or throw an error if something is missing. Now there do exist "uber build systems" that are written in cmake. I'm fine with those existing (though I would probably not chose cmake to write it in, that is the author's choice). However when building a project you should not force a method to get dependencies on your users. The package management is a system level concern and you should not mess with it. Note that I just put a lot of hate on systems like Cargo, pip, or hackage. Or google's pull everything into the one repository. Dependencies belong to tools like apt, rpm, yocto, ,snap, conan (when not building packages to install) - Microsoft/Apple are deficient for not providing them. Your program should just attempt to build with what it is given, not try to subvert my system - whatever it is. Yes I know it makes your life easier to not have to deal with more than one [possibly buggy] version of a dependency, but it makes my life worse.

joshuamorton 5y ago
> The package management is a system level concern and you should not mess with it. This makes anything approaching a hermetic/deterministic/reproducible build basically impossible. That's not good. How do I get consistent builds across systems if I have to hope that some (set of, if I'm building on multiple OSes) external dependencies give me the correct version of the package I'm looking for, especially since many of the tools you listed don't, for example, support multiple simultaneous versions of a particular package. I'm also confused by precisely what you mean by package management. Tools like pip cargo and bazel etc. Don't manage compiled binaries, but uncompiled sources. They deal with code dependencies, not dynamic library dependencies. Installing a source file via apt doesn't make a whole lot of sense.
- daemin 5y ago
  Have a script which downloads, installs, and builds dependencies. Have a script which generates build project/solution/script files for the actual build. These scripts can use whatever tools your organisation is most comfortable with, up to and including a mono repo.

gspr 5y ago

> How else would you add/manage external dependencies for your project? Either using a package manager, or doing it "manually" (like you suggest) – a process which can of course also be automated outside of the build system. > Do you manually install them on your local host? That's not reproducible and doesn't support multiple versions for different projects. Of course that's reproducible and supports multiple versions. Any build system worth its salt can be directed to look for dependencies at specific locations. Then multiple versions of things can be installed at multiple locations. This is literally how things have been done for ages. > Do you manually copy all the external deps into your own project repo, and then manually keep them updated with the latest changes? I abhor bundling dependencies, so I'm probably not the right person to ask, but supposing that you do want to bundle dependencies: yes (a process that can be scripted just as easily as it can be handed off to the build system!)

Joker_vD 5y ago
Why do I want the package manager to fetch and install dependencies for a project? I don't want to put all kinds of random junk into my /usr/lib before running "make" in the project's directory stops failing. And I've seen makefiles with "sudo apt install libqt5core libxml2-dev ..." in them and I didn't like that.
- gspr 5y ago
  > Why do I want the package manager to fetch and install dependencies for a project? You don't have to do that. If you separate the task of building from the task of managing software, you can mix and match solutions for each separate task as you wish. > And I've seen makefiles with "sudo apt install libqt5core libxml2-dev ..." in them and I didn't like that. That's obviously terrible too.
howinteresting 5y ago
Build systems that manage dependencies are that automation you're talking about, except they're easy-to-reuse across many projects. That's incredibly valuable. Being able to ship software, especially across many platforms is much more important than maintaining some sort of purist separation between build system and package manager. Your view of affairs, after being dominant for decades, is finally receding into the background, and the world is a better place for it.