8 ms·
Does C++ need a universal package manager?
- singularity2001 9y agoin a way apt and yum are package manager for c++
- rootlocus 9y agoThey're not exactly cross platform though
- claudius 9y agoThey are cross-language though, something I (as a user) personally value much much more highly than a cross-platform package manager which also supports a bunch of random platforms which I don’t care about at all. For developers/software distributors, this may of course look very different.
- rootlocus 9y agoAs a (mostly) windows user, they're completely useless to me. If it works for all linux users, it's not a C++ package manager, it's a linux package manager.
- RayDonnelly 9y agoYou can use `conda` cross-platform to install many C++ libraries.
- pfultz2 9y agoThe package managers that are listed in the post are cross-platform and work on windows.
- rootlocus 9y agoWell sure, but I wasn't talking about them. I was replying to someone who wrote > in a way apt and yum are package manager for c++
- Eridrus 9y agoI find apt completely unsatisfying as a package manager, even when I don't care about OS compatibility, for software development. I doubt yum is any better.
- jeremyjh 9y agoSo, a language-specific package manager allows me to specify different versions of different dependencies in my various projects that I develop on the same machine. The competent ones support reproducible builds by locking dependencies to the exact release I used when I initially added the dependency (until I explicitly update it). apt and yum are completely hostile to reproducible builds.
- exDM69 9y agoNope, they solve different problems. The OS package manager (apt, etc) is there to provide you the OS install with a set of applications and a set of libraries to support those applications. Your development, on the other hand, will depend on a different set of dependency libraries (and this must be reproducible!). This may include stuff like having several different versions of a library (for different projects or different branches or build configs of the same project). This set of libraries will sometimes conflict with what you need for the apps the OS provides. Because you never want to break your day-to-day applications supported by the OS package manager, you'll need a different way of installing your depepdency libs. This is what language package managers do. Right now for C and C++ the alternatives are 1) bundling it with your source 2) use version control (e.g. git submodules) 3) make install --prefix=$HOME/foo and configure build for that 4) single header file libraries. None of these work particularly well compared to a proper development package manager like Cargo.
- Shorel 9y agoIn practice, there are libraries and also dev-libraries which actually add headers and link-objects to use these same libraries. IMO you use submodules only for libraries not already included in your package manager.
- exDM69 9y ago> In practice, there are libraries and also dev-libraries which actually add headers and link-objects to use these same libraries. When developing software, you need both installed. Not every distro has -dev packages to install headers. > IMO you use submodules only for libraries not already included in your package manager. This approach is full of problems, most importantly that this isn't reliably reproducible (without using docker images or something for a reproducible environment). Your package manager may have different libs or different versions of libs than another developer's package manager. But the bigger issue is version requirements mismatches. Say your web browser (or other must-have app) requires libwidget-1.0, but you're writing an app that needs libwidget-2.0 or libwidget-0.8 or both (e.g. your latest supported release was 0.8 and next release is 2.0). You can't install 2.0 or 0.8 because it would break an app you need for daily work. Using your distro package manager for development is a partial solution at best.
- peter-m80 9y agoYes
- emsy 9y agoI'm surprised the author forgot to consider platform architectures, because he has the most important aspects of a package manager down. Unfortunately, it is still terribly easy to write non-portable code with C++. I think this is the main factor why we still don't have a widely used package manager for C++ but your mileage may vary.
- pfultz2 9y ago> I'm surprised the author forgot to consider platform architectures That is mainly for binary distribution. Most C++ package managers currently are source-based because binary compatibility is not a easy problem(there are some package managers trying to solve it though). > Unfortunately, it is still terribly easy to write non-portable code with C++ Yes, it easy to call into a platform-specific API, so cross-platform portability is limited by the testing resources for the package maintainer or library author.
- emsy 9y agoI was talking about source code, not binaries. And I didn't mean platform specific APIs either (though that is a problem as well). I was talking about differences between processor architectures. But while we're at it, try using wstrings or snprintf across Windows/Linux.
- rootlocus 9y agoXKCD comes to mind: https://xkcd.com/927/ https://xkcd.com/927/
- Jgrubb 9y agoI think every programming language would benefit from a universal package manager.
- deleted 9y ago[deleted]
- nurettin 9y agoC++ currently has a package manager. It is called git. Git has submodules for dependency management, tags for release management and binary downloads, gpg signing for authenticity and online services for hosting.
- adrianN 9y agoA package manager is not very useful if your build system doesn't integrate with it.
- nurettin 9y agoYes, that's why you commit your cmake files.
- stochastic_monk 9y agoI agree both that nurettin has a great point and that git is insufficient. A recent failure: I use some projects which fail to compile with clang or more recent gccs but happily compile with gcc4. Homebrew/linuxbrew don't let me have multiple gccs. I use a server which supports the module system and covers gccs from 4 through 7, so I don't have problems there, but I can't compile some things at all on my laptop. Similarly, CMake can work well, but it often breaks for me when I try to build some other peoples' projects, and is much harder to reverse engineer to fix than Makefile.
- humanrebar 9y agoSubmodules don't scale when you have N dependencies with M directory structures.
- rdtsc 9y agoWonder if Nix package manager would work here. It is not strictly tied to the OS unlike deb, rpm etc. It should work in most Linux distributions and MacOS. And it is not a single language. Mono-language package managers work for a while, then all of the sudden needing to deploy other languages / framework and it becomes awkward. Moreover it emphasizes reproducible build and seems to handle dependencies in a sane way.
- steveklabnik 9y agoNix barely works on Windows, that is, only through the WSL. I’d imagine that alone makes it a non-starter for what OP wants; Windows is a very important platform for C++.
- microcolonel 9y agoIt's such a shameful platform too. A lot of C++ stuff I work on and with has a C++11 path, and a Windows path for anything that needs, for example, guaranteed width integer types. Part of the fault for that in particular lies on the ISO folks for waiting twelve years to introduce <cstdint> instead of at least adding it in C++03 (since that's the first C++ after C99); but a big part of it lies in the fact that Microsoft makes you pay to compile things for their platform, so developers get understandably cheap and stay two (or four!) generations of VS behind.
- pjmlp 9y agoOther platform vendors also sell their compilers. For those on the cheap side, Microsoft had the express versions for quite some years now.
- microcolonel 9y agoExpress doesn't do 64-bit though, does it? or did they drop that restriction? > Other platform vendors also sell their compilers. Also, no, Apple's platform compilers are free (though obviously somebody pays for them), and so are the platform compilers for the major free operating systems GNU/Linux, FreeBSD, OpenBSD, Solaris. Sure, you can buy ICC or XL C/C++, but it isn't the only supported way to build and link against those platforms.
- gigatexal 9y agoYes it does.
- revelation 9y agoA package manager for C++ isn't a thing that just downloads files and puts them in predetermined places, maybe with a hip website. It needs to understand ABIs, architectures, standard library runtime versions, symbols and all the other things that make libraries in C++ such a pain. Nobody needs another tool like CMake that just goes "whatever bro GL, here is the compiler/linker command line variable" once you need to do something beyond specifying input files. (This is a big area where lots of improvement is possible. I want to kill someone whenever a linker just spazzes out with "undefined reference". YOU ARE THE MACHINE, GO THROUGH THE LIBRARY SEARCH PATH AND TELL ME WHICH ONE HAS THE SYMBOL.)
- oytis 9y agoNo, please don't. Let's not bring dependency mess to proper programming languages. Not to say C++ binaries normally link to their dependencies dynamically and can't control how they are built (and it's also good).
- ravenstine 9y ago"Proper" programming languages?
- sanjeetsuhag 9y agoOh, boy, here we go.
- cjwoodall 9y agoA "dependency mess" exists both with and without package management. A package management system gives you a playbook to follow with tool support, and "winging it" lets you suffer and clean up the mess in your own clever way. The advantage of course is that that clever way CAN work better for your team (for now)
- oytis 9y agoOf course you _can_ create a dependency mess in a C++ project. Package managers just make it a lot easier. My personal statistics (aka anectodal evidence) says that a typical Rust project has several dozens of direct dependencies, while a typical C/C++ project can make it with half a dozen. And those will most probably be dynamic libraries, which encourages both sides to care about API stability. Some libraries, openssl to name one, are notorious for failing to do so, but at least it is expected.
- cjwoodall 9y agoI also find that in c/c++ I tend to maintain my own library of useful bits and pieces, where in another language I would use 3rd party libraries. There are pros and cons both ways tbh. Some of my pain is from the embedded side where most libraries I use need a special set of flags and are compiled to static libraries or directly into the binary.
- gmueckl 9y agoC++ cannot be covered by package management in the same way interpreted languages are. The problem space is vastly more complex. It starts with optional language features that often get get disabled in projects (most often RTTI and exceptions), continues with incompatible ABIs between compilers and even compiler revisions and does not stop with the presence of sometimes countless compiler flags for optional features and dependencies. For all of these reasons it is not possible in the general case (the one that package management needs to target) to simply distribute binaries. Linux distributions are not a working example here because the libraries are built for the use of the other packages in the same repository. Needs of third parties are never considered because finding library configurations that work for the packages in the distro is hard enough. A platform independent package manager for C++ must effectively degrade into a build system that rebuilds all dependencies from source. And it must allow build parameters for each step to be defined by the dependent libraries or programs, so that the result is something that can be linked together successfully.
- humanrebar 9y ago> ...it must allow build parameters for each step to be defined by the dependent libraries or programs... Which will (given enough code being built) lead to incompatible flags amongst dependent packages. Compilers may or may not, at build or link time, give sensible error messages. It would be a very sophisticated package manager that could warn you of these sorts of conflicts at specification or installation time.
- cjwoodall 9y agoTo be fair some modern compiled languages are covered by package managers. For example, Rust and Cargo. Admittedly there are constructs built into the language and design that have made the plausible since early on.
- kibwen 9y agoEven above any technical aspect, the most important thing about Cargo's success is that it was the official default from day one. There's hardly any question about how packages should build or dependencies should operate. And indeed, Cargo does encourage (though not require) everything to be built from source, which is probably intentional given Mozilla's stance on OSS.
- entelechy 9y agoI'm working on https://buckaroo.pm https://buckaroo.pm a package manager that builds from source and focuses on reproducible builds. To accelerate the build we support caching and sharing artefacts Would love to hear your opinion on that
- pfultz2 9y agoOr rather, I would like to hear your opinion instead. Would buckaroo consider using a standard package metadata similar to what I outlined? I believe buckaroo uses integrated builds so its likely you would need to supplement it with other information such as source files, but hopefully the core information could be reused.
- entelechy 9y agoYes definitely! The biggest time sink when porting a package was to research a projects requirements and dependencies. To our surprise only a handful projects do this correctly. If we could agree on a standard that captures the dependencies and requirements of a C++ project correctly it would be a huge win for the community. Currently there is no buildsystem that enables users to describe the requirements of an project correctly. We choose to support buck as a buildsystem mainly because it allows us to describe nicely the structure of a project. However like any other buildsystem it does not capture the full essence of a C++ project. For instance there is no way to describe the set of compatible compiler and linker flags. Many build systems evolved to be turing complete so we can write logic to handle various combination of options(eg. no-rtti) but this makes it really hard to reason about projects. It turned out to be quite challenging to determine if two projects are compatible with each other - in fact it turned out to be as challenging as solving the halting problem. Is this really necessary? I analyzed now over 300 c++ projects and I still haven't seen one project that truly needed a touring complete buildsystem to be build (Not even LLVM&Clang). So yes - I'm convinced a metadata file as you suggested is the way to go.
- pfultz2 9y ago> Currently there is no buildsystem that enables users to describe the requirements of an project correctly. We want to steer away from standardizing a build system for now. The uses cases are enormous to try to tackle. Instead, it would be simpler to standardized the description of the build environment so the package manager can pass that to the build system. Hopefully, build systems will be updated to read the package metadata so it can properly consume the dependencies correctly. > So yes - I'm convinced a metadata file as you suggested is the way to go. Good to know we are on the right track.
- alexnewman 9y agoFor source packages yes. Deb/RPM are binary packages so not equivalent for devs. st demonstrated how valuable it is for native code. I think they should just adopt cargo.
- MichaelMoser123 9y agoWhat about the C and C++ runtime library? (together with its quirks, platform incompatibilities and different versions for every build of some os). This little dependency alone will require you to recompile every nontrivial piece of software from source, or to have something like debian - it needs the source for every component.
- gumby 9y agoI don’t see why a recompile should be required — that’s why we have platform ABIs. I routinely interlink g++-compiled and clang++-compiled code without issue.
- MichaelMoser123 9y agoTry using a library compiled on a slightly different version of Linux.