6 ms·
Simplifying build in C++
- calumrobinson 9y ago> One last item remains (and possibly the most controversial): not using a language other than C++ to describe how to a package a C++ project Totally agree on this point. Why learn all the quirks of the CMake language just to express the structure of my program?
- vvanders 9y agoThis is what makes me love Rust/Cargo so much. You can create pre-build orchestration in your build.rs in the same language as you build your project.
- deleted 9y ago[deleted]
- pornel 9y agoInitially I thought they've lost their mind to replace domain-specific scripting languages with Rust, but I've come around to love it. It has enabled Cargo use its own package ecosystem at build time as well, so build scripts can easily reuse other crates instead of reinventing the wheel for each project. It also saved Cargo from needing to be a build system that works with everything and supports everyone (the midlayer mistake[1]) and instead 3rd party build-time crates and *-sys library wrappers provide the flexibility. [1] https://lwn.net/Articles/336262/ https://lwn.net/Articles/336262/
- masklinn 9y agobuild.rs is a fallback though, the main "language […] to describe how to a package a [Rust] project" is Toml and the Cargo.toml file (http://doc.crates.io/manifest.html http://doc.crates.io/manifest.html), build.rs is the escape hatch if the main manifest file does not suffice. Python uses a Python script as the main entry point to the package, and it's commonly problematic as doing all kinds of weird shit so your package builds (on your machines, until you change your configuration) is a smaller step than looking up what the system already provides, and thus the default. It also makes static package in(tro)spection much more difficult.
- lomnakkus 9y ago> build.rs is a fallback though, the main "language […] to describe how to a package a [Rust] project" is Toml and the Cargo.toml file (http://doc.crates.io/manifest.html http://doc.crates.io/manifest.html), build.rs is the escape hatch if the main manifest file does not suffice. I get the feeling that they're making the exact same mistake that Cabal/cabal-install did in the Haskell world[1] -- they just don't have complex-enough builds and/or a big enough ecosystem yet to know it. (In Cabal/cabal-install there's a "Custom" build type where "anything goes". Unfortunately that means that it's basically impossible for any higher-level tooling to have any idea of what's going on in the build... and hence it thus powerless to introspect anything useful. It's a huge pain.) [1] Not that I blame them, it's just such a tempting thing to do rather than completely redesigning the build system to accomodate everything and provide enough introspection/metadata. So it goes, I guess...
- vvanders 9y agoConsidering ~60% of the use cases for build.rs is wrapping C/C++ builds I'd love to know exactly how you expose metadata for that ;).
- lomnakkus 9y agoThat's a fair point, but even mandating "choose one of Ninja/CMake/etc." would be helpful in terms of at least getting a little bit of metadata out. This is obviously non-trivial, and I'm not sure how much metadata is even retrievable from my examples, but it's better than nothing. (Ninja and CMake operate at sort of different levels, but that's OK for this purpose, I think. At least you can ask things like "if X changed, what else would change?" without actually having to run a compiler.) (AFAIUI Cabal:Custom had exactly the same reasoning, btw. "It's too difficult! Let's just give up and allow anything.") It might be a practical reality that this is just unavoidable if you want to achieve success, but at a meta-level I think deserves thinking about whether builds should actually be allowed the full Turing Complete treatment. At least, theoretically: As long as we continue 'allowing' Custom for practical reasons and without serious thought, there can be no progress. (Btw, I'm quite mad which it comes to this type of thing. I spend way too much time thinking about build systems, etc.) EDIT: This problem manifests EVERYWHERE: Docker builds randomly guess wrong about whether inputs have changed, cabal-install gets it wrong, SBT/Zinc doesn't quite track dependencies correctly, etc. etc. and it's infuriating.
- ridiculous_fish 9y agoRaw C++ is unsuitable as a build language, so you'll have to layer a DSL on top of it. Either way you're learning a new language. As to "why learn all the quirks of CMake," the selling points of CMake are its widespread availability, maturity, and coverage. For example, just on the packaging side, it knows how to create tarballs, .debs, .rpms, Windows .msi files, multiple flavors of Mac installers... Less mature build tools require you to implement this stuff manually. CMake mostly handles it for you (and has the scars to prove it).
- desertrider12 9y agoC++ as it exists now is unsuitable, but I can imagine an alternate history where C++ actually allowed arbitrary compile time execution (better than macros and constexpr). Then someone would have made a library for build system logic, and all the user would have to do is g++ project.cpp.
- fatbob 9y agoHerb Sutter has a proposal for metaclasses, which involve running some compile-time C++-like code. I don't think it would stretch to this, but it's heading in your direction.
- pjmlp 9y agoI do like cmake, however I think that actually C++03 is the one unsuitable, C++17 is quite capable. All those actions can be easily modeled in a libraries coupled with a DSL.
- Jaxan 9y agoI never understood why people insist of having a single language for all parts of development. Most developers already (should) speak a few languages, so one extra is fine, I guess. I strongly believe in DSLs.
- Koshkin 9y agoIt's not that they "insist", it's just that when you look at a modern general-purpose programming language you often ask yourself, why couldn't I use it for such and such particular purpose?
- subb 9y agoThe premise of matching modules and package one-to-one is I think incorrect. The way I see it, a package is just a library with a description of its dependencies, and possibly an already compiled binary. It could contains multiple modules. Plus, you probably need to link with a specific package version, and the module proposal doesn't have any way to specify a module version. So you can't just extract package dependencies from module usage.
- duality 9y agoIt does not satisfy all of the items on the author's wish list, but I have used and enjoyed Bazel[1] as perhaps the nearest build system in existence that provides a notion of package management to C++. It does not hold prebuilt binaries to satisfy dependencies. But it does cache builds. Nor does Bazel use C++ syntax to describe the build, instead using a variant of Python called Skylark, but this feels (to me) much less onerous to pick up than the custom scripting language CMake uses. Bazel also provides a notion of visibility of dependencies that allow you in effect to control which build targets are exposed to other targets as allowed dependencies. And allows the build tool to specify a toolchain (compiler, linker, etc.) and machine architecture. This information is used when building all the dependencies, which in effect satisfies the author's request for compatibility between packages. You might even argue that a (caching) build from source may be better than attempting to serve prebuilt binaries. [1] https://bazel.build https://bazel.build
- chuckcode 9y ago+1 for bazel. It also scales to large builds and handles java as well. We've seen big decreases in build times by using it as it does caching, parallel execution and good dependency analysis. Getting up to speed can take a little doing though as the original docs weren't great and there are some counter intuitive things about it. I think the docs are improving but some things a little odd like everything gets dumped into ~/.cache/bazel without any command line option to change [1]. Feels like the first time I used git where you suspect that the authors designed it to be difficult to keep the lazy away. https://docs.bazel.build/versions/master/output_directories.html https://docs.bazel.build/versions/master/output_directories....
- ridiculous_fish 9y agoWhat software do you build with bazel? How do you distribute it? Would you use it for an open source project? It feels onerous to ask users and contributors to install a moderately esoteric build tool (which in turn requires Java) to build a C++ project.
- chuckcode 9y ago
- llimllib 9y agoSeems like the horse is out of the barn, across the hay field, down the street, and over to the next town on this one.
- maxxxxx 9y agoC++ really needs an ABI standard. It would make builds and package management much easier.
- gpderetta 9y agoIt is impossible to have a standard ABI for a language that is supposed to be compiled down to machine code and run as native executable. The ABI is tied to the OS and specific machine. At best you can have standardized machine specific ABIs. Which is exactly what we have.
- maxxxxx 9y agoMachine specific ABI would already be great. Whenever I need a new library it takes forever to figure out how to integrate it correctly and add to the build. From observation this also trips over most newbies who don't know all the subtleties.
- pjmlp 9y agoI doubt the OS and compiler vendors will ever agree on one. Right now the "best" solutions are COM (aka UWP) on Windows, and the C++ Itanium ABI description. Even C does not have one, just that most mix language ABI with OS ABI, which happen to be the same on OS written in C. If you pick something else, not written in C, for example IBM i, then the ABI is actually TIMI, the OS/400 bytecode format. MBIs, the binary format for modules, might be the answer but until then there are many issues to tackle and agree upon.
- epage 9y agoDecoupling build from dependency management I think is the great feature of conan. https://www.conan.io/ https://www.conan.io/
- arximboldi 9y agoI like Nix. Only problem is it doesn't seem to work on damn Windows. Nix has, in my setup, made the build system topic almost irrelevant, since it allows for simpler build scripts to play well with other dependencies. Plus, it works with stuff in any language and polyglot projects are a bliss. I really hope it catches traction.
- dj-wonk 9y agoI also like Nix. Can you share your particular workflow? Also, how does your workflow handle ‘snapshot’ / temporary dependencies?
- bobajeff 9y agoI still haven't learned how to use a build system yet. For me it's just way easier to put the individual build commands in a .sh file and run it.
- earenndil 9y agoYou should at least use a makefile. It's pretty simple, and mostly the same as shell, but it will automatically detect changes, and only recompile the files that need to be recompiled.
- Skunkleton 9y agoHonestly, I think very few engineers fully learn a build system. The norm seems to be learning just enough to hobble along. That said, you should really try out GNU Make, CMake, or similar.
- bobajeff 9y agoI have...
- lomnakkus 9y agoIf you're using C or C++ you can actually get suprisingly far using this approach + ccache. Most C preprocessors are ridiculously fast at this point.
- dj-wonk 9y agoBesides CMake, Bazel, and Conan, what build system or related project(s) are worth looking at? Which one(s) would you recommend? (See other threads as well)
- DubiousPusher 9y agoI'm not saying whether it's good or bad but for completeness one should probably look at MSBuild. There's a pretty interesting compiler wrapper called "fastbuild" that really lives up to its name. C/C++ build environments are fascinating to me because there are so many. They solve this swarm of overlapping problems in many different ways.
- oscargrouch 9y agoAs i've said in another comment, theres also GN, the build system used for Chrome. It's bazel-like, in C++, so you can have a binary tool, and it outputs ninja build files. https://chromium.googlesource.com/chromium/src/tools/gn/ https://chromium.googlesource.com/chromium/src/tools/gn/ https://chromium.googlesource.com/chromium/src/+/lkgr/tools/gn/docs/quick_start.md https://chromium.googlesource.com/chromium/src/+/lkgr/tools/...
- McP 9y agohttps://buckbuild.com https://buckbuild.com
- geezerjay 9y agoNo love for autotools?