6 ms·
So many people have tried and failed to do this. I'm skeptical that these guys have succeeded. The main problem seems to be that because there is no standard C
by IshKebab 8y ago
So many people have tried and failed to do this. I'm skeptical that these guys have succeeded.
The main problem seems to be that because there is no standard C++ build system, every library uses a different one. Wrangling all those together without requiring the library authors to do anything is extremely difficult.
Hell many C++ libraries still use autotools or hand-written Makefiles. I don't really see how it is even possible to automatically download and build those projects.
- blueline 8y ago> I don't really see how it is even possible to automatically download and build those projects. so i mean what e.g. conan does is acknowledge this and require that package creators provide "recipes" that tell conan how to build the project. buckaroo seems to take a similar approach: https://github.com/LoopPerfect/buckaroo/wiki/Creating-a-Package https://github.com/LoopPerfect/buckaroo/wiki/Creating-a-Pack... i will say that in my time using conan to attempt to solve for compiling an application for x86 and a few flavors of ARM, this is NOT a silver bullet.. a huge fraction of third party libraries we tried to pull in needed to have their recipes modified to build correctly.
- entelechy 8y agoAfaik Conan requires a server and you cannot install from arbitrary sources. Furthermore Conan prefers to download precompiled libraries. Unless you run your own artifactory, this may be problematic for people for people concerned about security or ABI compatibility.
- cbuq 8y agoJust to break down your complaints. 1. Conan does not require a server. If a project contains a conanfile.py with build instructions, you can also run `conan create` from a project's source repository (no server required). 2. You can easily force build dependencies with the `--build *` flag
- adgasf 8y agoInteresting. Regarding (1), does this mean you have to download the dependency from GitHub by hand, then put it into the cache before you can "install" it?
- shin_lao 8y agoIt's indeed going to be very, very, hard to retrofit a package manager into C++. You could structure something around CMake. One immediate issue is compilation flags and platforms management. You may need to use some custom flags, for example, you need to compile against a specific architecture. You want those flags to be passed to all libraries uniformly. But the horrible problem is quickly the compatibility matrix. In other words, if you are making available, let's say you are using alib v1 and blib v2, you need to check the two libraries compile, link, and work together nicely. So when you upgrade one of the libraries, you need to check that compatibility again. It's an important problem because if you're working on production software, you need to be able to build all versions and maintain them. And maybe you will have to upgrade only one library. You can't always be "current" everywhere. So very quickly, using a package manager becomes more painful than having the libraries management manually with a couple of custom scripts. I'll be happily be proven wrong by a package manager (and we tried a lot).
- brobdingnagians 8y agocget appears to be structured around CMake, looks like a pretty simple, minimalistic effort that would afford some convenience https://cget.readthedocs.io/en/latest/src/intro.html https://cget.readthedocs.io/en/latest/src/intro.html
- dcuthbertson 8y agoI can't say I'd be a fan of a CMake based package manager. While CMake has been a great tool under Linux environments (I've used it for C/C++ builds, and in mixed C & Go build environments), I've also experienced horrors when CMake is used to generate Visual Studio solution and project files. I mostly work in Windows development. CMake scripts created by a professional build engineer caused a lot of havoc. It injected itself into the project files in such an insidious fashion, that it was nearly impossible to tell if dependencies were correct. It completely broke Intellisense, making code navigation an exercise in frustration. I was not happy, and wound up ripping out the scripts and creating new solution and project files that actually supported the needs of the developers. (Edited to remove typos and improve grammar)
- je42 8y ago
- dana321 8y agolinuxbrew / homebrew does a pretty good job at downloading packages and compiling them. The wildcard is cross-platform. vcpkg doesn't seem to be 100% there and has many bugs for example the folders where includes and libraries are kept changes based on platform. The other problem is static vs dynamic linking, as well as conflicts between different combinations of cmt libs etc. it gets very messy quickly. Its almost like you're spending more time trying to get all the libraries you want working together than actually coding.
- rienbdj 8y agoThe solution in my experience (also the approach at Google) is to always build from source but cache the build artefacts. This is how Buckaroo works too (although it is Buck instead of Bazel).
- entelechy 8y agoyes and you can also distribute your cache [1] Furthermore it uses git to manage the sourcecode and use shallow&shared clones to have quick downloads. [1] https://buckbuild.com/files-and-dirs/buckconfig.html#cache https://buckbuild.com/files-and-dirs/buckconfig.html#cache Lastly I'd like to note that Buck and Bazel are converging to Skylark for describing builds. The differences between Buck and Bazel should become smaller over time
- ghthor 8y agoPants has also converged with Bazel around BUILD file formats.
- spacksmack 8y agoObligatory Spack reference: Spack is also worth checking out if you want to build everything from source. A lot of the national labs use spack now because you can specify all the dependencies for a tool (eg, g++8, boost1.68, opempi3) and it will build out any of the missing tools/compilers for you in one command. It doesn't solve the build system stuff like buckaroo, but it works with whatever build stuff I want to do (cmake, make, sh). https://spack.io/ https://spack.io/
- rienbdj 8y agoThis is the chicken - egg problem of C++ package managers. one approach is to try to wrap all build systems (Conan); the other is to port to a common build system (vcpkg, Hunter, Buckaroo). The wrapping approach makes it very hard to inspect the build and customize things. The porting approach makes it hard to get things moving. I think the porting approach will prove better in the long run. What we really need are automated porting tools to fix this mess!
- entelechy 8y agowe build https://buildinfer.loopperfect.com/ https://buildinfer.loopperfect.com/ to transpile buildsystems automatically.
- entelechy 8y agoOne of the authors here. Yes you are totally right, as build-systems are not standardized it is quite challenging. However we have seen more than 300 ports back in 2017 [1] and saw many emerging patterns. This enabled us to write rules to transpile and optimize 100s of build-systems to buck using buildinfer [2] [3]. We hope that more sane decisions will be made regarding build-systems in new projects. We discovered analyzing over 300 projects that a lot of complexity is not needed and often a non-turing complete language for describing the build is sufficient. In fact we found that over 90% of projects can be described solely by using glob expressions. [1] https://hackernoon.com/lessons-learned-from-porting-300-projects-to-buck-build-ff6463b65142 https://hackernoon.com/lessons-learned-from-porting-300-proj... [2] https://hackernoon.com/announcing-buildinfer-for-c-3dfa3eb15feb https://hackernoon.com/announcing-buildinfer-for-c-3dfa3eb15... [3] https://buildinfer.loopperfect.com/ https://buildinfer.loopperfect.com/
- ghthor 8y agoThat's so freaking cool! I'm not to surprised that 90% can be globs. Were there any projects that you saw that were polyglots? Maybe some type of library that had built in diagnostics and metrics providing prebuilt html or interactive webpages?
- entelechy 8y agoMay projects use some document generators or tools to collect metrics. However some build-systems that were really interesting were OpenCV and OpenSSL. OpenSSL buildsystem is over 6k lines of perl scripts which communicate over pipes. OpenCV uses cmake, python and prolog for it's build. Until this day, I have no clue why there is prolog involved. You can read some bits about it in our initial announcement of buildinfer: https://hackernoon.com/announcing-buildinfer-for-c-3dfa3eb15feb https://hackernoon.com/announcing-buildinfer-for-c-3dfa3eb15...