11 ms·

Qb – Zero-configuration build system to quickly build C/C++ projects

48 points by olvy0 4y ago

danny0z 4y ago
We can also try https://github.com/xmake-io/xmake https://github.com/xmake-io/xmake
kevingadd 4y ago
As much as I love zero-configuration software, it seems implausible to do it for C/C++ as anything other than a gimmick. Even a simple program may need to link against a library or two to pull in some APIs unless absolutely every single function you use is in libc and it gets linked by default. If you want to target multiple platforms it gets worse because you may need things like per-platform defines set by the build system or per-platform .c/.h selection. Once you approach that level of complexity you should strongly consider just using existing build ecosystems like cmake - not because they're pleasant, but because the people maintaining them have identified a lot of the sharp edges that you're going to run into. The performance of builds with this seems like it could also scale badly once your app gets big enough, since it doesn't seem to do incremental builds or avoid re-building individual .c files if a source file and its dependencies (headers) haven't changed. Seems handy if you just want to fiddle around with a .c and .h file pair without writing a makefile, though!
- jhgb 4y ago
  I imagine that using Zig to compile C++ projects is probably a better idea?
  - cyber_kinetist 4y ago
    Can you actually write Zig build scripts to build C++ projects? If that’s possible then that might be a good option.
    - jhgb 4y ago
      Judging from https://zig.news/kristoff/make-zig-your-c-c-build-system-28g5 https://zig.news/kristoff/make-zig-your-c-c-build-system-28g... , this should be possible. Considering Zig's features, I wonder if even the automated build logic in this Qb system couldn't be ported into a Zig build script completely.
    - flohofwoe 4y ago
      Yes, here is an example for a non-trivial mixed C/C++ project: https://github.com/floooh/sokol-tools/blob/master/build.zig https://github.com/floooh/sokol-tools/blob/master/build.zig Compared to cmake this means giving up IDE support like Xcode or Visual Studio though, it's really "just" a build system, not a meta-build system which can generate IDE project files. The Zig build system also has some minor warts that don't matter much for a pure Zig project but do matter for C/C++ projects, mainly that build settings of dependencies (like header search paths or linker input) mostly don't automatically 'travel' upwards to the root, instead each dependency and root project needs to be configured on its own. There are plenty of upsides too though, like much simplified cross-compilation, all in all it looks very promising.
      - cyber_kinetist 4y ago
        Can it also generate compile_commands.json automatically? That is the only thing you need for IDE/editor support nowadays.
        flohofwoe 4y ago
        AFAIK no, sounds like a good feature to have though.
- e12e 4y ago
  > Seems handy if you just want to fiddle around with a .c and .h file pair without writing a makefile, though! Doesn't make already do this, with built-in targets? Eg, with hello.c, hello.h, no Makefile: make hello (not sure about header files though, I know it works for simple c files at least).
  - jcelerier 4y ago
    there's a whole lot of default rules in GNU make: https://www.gnu.org/software/make/manual/html_node/Implicit-Rules.html https://www.gnu.org/software/make/manual/html_node/Implicit-... but not sure they are portable
arketyp 4y ago
I admit the C/C++ build setup has deterred me from prototyping at times. I used to solve this by keeping a sandbox project where I replace code in main().
- c0balt 4y ago
  Similiar story for me with rust and js. The cost for the first setup with everything dialed in just right (proper linker, comp flags...) is often deterring for the prototyping stage. Maybe that's a case for boilerplates? Idk, though I really liked the idea of create-react-app and co.
- jhgb 4y ago
  I got lazy and installed https://github.com/ryanmjacobs/c https://github.com/ryanmjacobs/c . (Other than that, I intend to learn Zig including its C++ build capabilities.)
- arinlen 4y ago
  > I admit the C/C++ build setup has deterred me from prototyping at times. I don't understand how that can be a barrier to entry. There are plenty of free IDEs which already create ready-to-go projects, and build systems such as cmake make it quite trivial to setup a C++ project from scratch. Case in point: project(foo LANGUAGES CXX) add_executable(foo main.cpp) https://cmake.org/cmake/help/latest/guide/tutorial/A%20Basic%20Starting%20Point.html https://cmake.org/cmake/help/latest/guide/tutorial/A%20Basic...
HelloNurse 4y ago
It seems aimed at making the trivial use cases easy, the normal use cases hard, and the hard use cases impossible. For example, "exe", "dll" and "lib" are types of projects, not of individual files: no apparent way to have multiple and heterogeneous end products (e.g dynamic libraries + static libraries + test executables + something not C/C++ like Doxygen) or to define variants like compiling the same program as a statically linked big executable or as a thin executable and a bundle of dynamic libraries. Lack of documentation is another serious problem. For example, how do I pass exotic GCC flags and how can I ensure that qb uses GCC instead of MSVC? "Zero configuration" in serious systems means self-configuration, e.g. automatically finding and inspecting available compilers, to provide added value over completely manual configuration; denying configuration, or at least not thinking of it because no toy example has needed it so far, isn't very productive for the end user.
- arinlen 4y ago
  > It seems aimed at making the trivial use cases easy, the normal use cases hard, and the hard use cases impossible. I'd add that mainstream build systems already covered this sort of usecase. For instance, qmake used templates[1]. With CMake anyone can put together their own macro, and have config-free cmake projects. In fact, I've already worked on projects that followed this approach. I guess it's all fine and dandy when all you do is follow the happy path, but that's hardly the case. [1] https://doc.qt.io/qt-5/qmake-variable-reference.html#template https://doc.qt.io/qt-5/qmake-variable-reference.html#templat...
nspattak 4y ago
go code to compile c/c++ code? What is the reason for this complexity? What does this project solve that existing package managers do not? (eg compared to vcpkg, spack or hunter) After a week of fighting with vcpkg, i really doubt that this project is worth anyone's time.
- rwj 4y ago
  Go is surprisingly good in this role. Super easy cross compilation, good platform support, and easy distribution means that it's very easy to get the tool where you need it. For a lot of file munging, Go is serviceable.
  - nspattak 4y ago
    To my understanding, the problem is how to easily and consistently build several versions of software stacks(libraries/executables) with different compilers(primarily MSVC/icc/gcc/clang) on windows/linux/osx for windows/linux/osx/android. I do not see how go (or any other tool) is more or less suited to solving this problem and given the variety of existing tools (and their failure to do it), makes me suspect that this project is merely an alpha stage effort in solving this problem and mostly an exercise in go.
- deleted 4y ago
  [deleted]
erwincoumans 4y ago
The discussion here reminded me about a blog post I made back in 2011, concluding with meta-meta-build systems. Unfortunately, Mike Acton didn't maintain the AltDevBlogADay blog due to technical (build system?) issues here a copy (warning, a wall of text!): ++++++++++++++ Erwin Coumans, 2011: "This is a story about my search for a hassle-free cross-platform open source (meta-/meta-meta-) build system. For our open source Bullet physics engine, I’ve been distributing the source code in a way that should make it as easy as possible to build out-of-the box. This means that for all supported platforms, the user (a developer who downloaded the Bullet SDK) should be able to download and unzip the zipfile or tarball on his machine, and get started as soon as possible. It should be hassle-free for the user but also for me, so I rather don’t manual updating too many files for each release. Aside from all the different platforms, we need to also support various compilers, compiler settings and integrated developer environments (IDEs). Common platforms are Windows, Mac OSX, iPhone, Linux as well as PlayStation 3 and XBox 360. Some common compilers are Visual Studio compiler, GCC, SN Systems and LLVM. Here are a couple of build systems and modifications that I tried or considered to try: Visual Studio project files. Shipping manual generated visual studio files works hassle-free for the user, but isn’t hassle-free for myself especially if you want to support all versions from MSVC 6 up to MSVC 2010. Obviously it is also not cross-platform so it needs to be combined with another build system. For example autotools. autotools: for most unix flavors, autotools does a great job. It is often already installed and is fairly hassle-free, as long as the user doesn’t run into version compatibilities of automake, autoconf or libtool. It is a bit of a hassle for me, because I’m not too familiar with autotools and can only provide most basic support. A bigger issue is that autotools doesn’t support IDEs such as Xcode. Xcode, the environment for Mac OSX and iPhone, requires its own project files. Shipping manual generated project files is a burden, when dealing with versions and platform settings. jam and msvcgen: jam is a fast build system and it is small enough to ship the entire jam source code together with my own project (it is just a few hundred kilobytes with no external dependencies). The msvcgen extension could autogenerate standalone Visual Studio project files from MSVC 6 onwards, so that was a nice extra. For several years I’ve been shipping the Bullet SDK with autotools + jam and msvcgen autogenerated Visual Studio projects for MSVC 6, 7 and 2005. No proper support for XCode, until I noticed cmake. cmake: this is almost a dream meta build system but unfortunately it cannot properly generate distributable build systems. For example the generated Visual Studio project files only work on the local machine. I’ve been hacking the cmake source code so that it does generate distributable project files, and submitted a few patches back to cmake. Unfortunately, the main cmake developers don’t want to support distributable project file generation, and I felt they were very reluctant to discuss further changes to support distributable project files. For example, I need the option to avoid creating copy rules that depend on cmake being installed. Furthermore any environment variables should not be expanded in absolute paths but stay environment variables for shipping project files (for example DXSDK_DIR, ATISTREAMSDKROOT or NVSDKCOMPUTE_ROOT for some recent OpenCL and DirectCompute accelerated optional modules). This means that I need to keep on using and updating a forked version of cmake to generate and distribute MSVC projectfiles. As a side note: the wxWidgets project even created their own build system, just because of this cmake shortcoming. Another shortcoming of cmake is lack of multiple compiler support, so generating projects for both PlayStation PPU and SPU is not possible. After bringing up my build system issue on the PS3 Devnet forums, someone mentioned premake. premake is a very promising meta build system and I keep on thinking of trying it out. I’ve heard people who create PS3 and XBox 360 project files, and it supports most other platforms. Premake can be optionally extended using Lua, but it is not a requirement for Lua to be installed, unlike the Python requirement for scons. Unfortunately, I’ve heard that premake has some issues with dependency tracking, has less features that cmake and it is not widely supported yet. This means that integrating it into another project might become a hassle. This leaves me to either wait until premake matures and becomes more widely used, or writing my own meta-meta build system that can generate both cmake, premake and other project files." ++++++++++++++++ Not a whole lot changed in those 11 years. Bullet focus moved to robotics and reinforment learning and Python/PyBullet and with that: setuptools as one more build system, so you can trivially install using: pip install pybullet
kgeist 4y ago
I once made something similar for my project as an excercise - it parsed the source code for #includes and figured out automatically what needs to be compiled. It was OKayish for my self-contained project which only used the standard library but it wouldn't work if you had to add third-party dependencies. Today I'd rather just use CMake.

48 points by olvy0 4y ago

We can also try https://github.com/xmake-io/xmake https://github.com/xmake-io/xmake

As much as I love zero-configuration software, it seems implausible to do it for C/C++ as anything other than a gimmick. Even a simple program may need to link against a library or two to pull in some APIs unless absolutely every single function you use is in libc and it gets linked by default. If you want to target multiple platforms it gets worse because you may need things like per-platform defines set by the build system or per-platform .c/.h selection. Once you approach that level of complexity you should strongly consider just using existing build ecosystems like cmake - not because they're pleasant, but because the people maintaining them have identified a lot of the sharp edges that you're going to run into. The performance of builds with this seems like it could also scale badly once your app gets big enough, since it doesn't seem to do incremental builds or avoid re-building individual .c files if a source file and its dependencies (headers) haven't changed. Seems handy if you just want to fiddle around with a .c and .h file pair without writing a makefile, though!

jhgb 4y ago
I imagine that using Zig to compile C++ projects is probably a better idea?
- cyber_kinetist 4y ago
  Can you actually write Zig build scripts to build C++ projects? If that’s possible then that might be a good option.
  - jhgb 4y ago
    Judging from https://zig.news/kristoff/make-zig-your-c-c-build-system-28g5 https://zig.news/kristoff/make-zig-your-c-c-build-system-28g... , this should be possible. Considering Zig's features, I wonder if even the automated build logic in this Qb system couldn't be ported into a Zig build script completely.
  - flohofwoe 4y ago
    Yes, here is an example for a non-trivial mixed C/C++ project: https://github.com/floooh/sokol-tools/blob/master/build.zig https://github.com/floooh/sokol-tools/blob/master/build.zig Compared to cmake this means giving up IDE support like Xcode or Visual Studio though, it's really "just" a build system, not a meta-build system which can generate IDE project files. The Zig build system also has some minor warts that don't matter much for a pure Zig project but do matter for C/C++ projects, mainly that build settings of dependencies (like header search paths or linker input) mostly don't automatically 'travel' upwards to the root, instead each dependency and root project needs to be configured on its own. There are plenty of upsides too though, like much simplified cross-compilation, all in all it looks very promising.
    - cyber_kinetist 4y ago
      Can it also generate compile_commands.json automatically? That is the only thing you need for IDE/editor support nowadays.
      - flohofwoe 4y ago
        AFAIK no, sounds like a good feature to have though.
e12e 4y ago
> Seems handy if you just want to fiddle around with a .c and .h file pair without writing a makefile, though! Doesn't make already do this, with built-in targets? Eg, with hello.c, hello.h, no Makefile: make hello (not sure about header files though, I know it works for simple c files at least).
- jcelerier 4y ago
  there's a whole lot of default rules in GNU make: https://www.gnu.org/software/make/manual/html_node/Implicit-Rules.html https://www.gnu.org/software/make/manual/html_node/Implicit-... but not sure they are portable

arketyp 4y ago

I admit the C/C++ build setup has deterred me from prototyping at times. I used to solve this by keeping a sandbox project where I replace code in main().

c0balt 4y ago
Similiar story for me with rust and js. The cost for the first setup with everything dialed in just right (proper linker, comp flags...) is often deterring for the prototyping stage. Maybe that's a case for boilerplates? Idk, though I really liked the idea of create-react-app and co.
jhgb 4y ago
I got lazy and installed https://github.com/ryanmjacobs/c https://github.com/ryanmjacobs/c . (Other than that, I intend to learn Zig including its C++ build capabilities.)
arinlen 4y ago
> I admit the C/C++ build setup has deterred me from prototyping at times. I don't understand how that can be a barrier to entry. There are plenty of free IDEs which already create ready-to-go projects, and build systems such as cmake make it quite trivial to setup a C++ project from scratch. Case in point: project(foo LANGUAGES CXX) add_executable(foo main.cpp) https://cmake.org/cmake/help/latest/guide/tutorial/A%20Basic%20Starting%20Point.html https://cmake.org/cmake/help/latest/guide/tutorial/A%20Basic...

HelloNurse 4y ago

It seems aimed at making the trivial use cases easy, the normal use cases hard, and the hard use cases impossible. For example, "exe", "dll" and "lib" are types of projects, not of individual files: no apparent way to have multiple and heterogeneous end products (e.g dynamic libraries + static libraries + test executables + something not C/C++ like Doxygen) or to define variants like compiling the same program as a statically linked big executable or as a thin executable and a bundle of dynamic libraries. Lack of documentation is another serious problem. For example, how do I pass exotic GCC flags and how can I ensure that qb uses GCC instead of MSVC? "Zero configuration" in serious systems means self-configuration, e.g. automatically finding and inspecting available compilers, to provide added value over completely manual configuration; denying configuration, or at least not thinking of it because no toy example has needed it so far, isn't very productive for the end user.

arinlen 4y ago
> It seems aimed at making the trivial use cases easy, the normal use cases hard, and the hard use cases impossible. I'd add that mainstream build systems already covered this sort of usecase. For instance, qmake used templates[1]. With CMake anyone can put together their own macro, and have config-free cmake projects. In fact, I've already worked on projects that followed this approach. I guess it's all fine and dandy when all you do is follow the happy path, but that's hardly the case. [1] https://doc.qt.io/qt-5/qmake-variable-reference.html#template https://doc.qt.io/qt-5/qmake-variable-reference.html#templat...

nspattak 4y ago

go code to compile c/c++ code? What is the reason for this complexity? What does this project solve that existing package managers do not? (eg compared to vcpkg, spack or hunter) After a week of fighting with vcpkg, i really doubt that this project is worth anyone's time.

rwj 4y ago
Go is surprisingly good in this role. Super easy cross compilation, good platform support, and easy distribution means that it's very easy to get the tool where you need it. For a lot of file munging, Go is serviceable.
- nspattak 4y ago
  To my understanding, the problem is how to easily and consistently build several versions of software stacks(libraries/executables) with different compilers(primarily MSVC/icc/gcc/clang) on windows/linux/osx for windows/linux/osx/android. I do not see how go (or any other tool) is more or less suited to solving this problem and given the variety of existing tools (and their failure to do it), makes me suspect that this project is merely an alpha stage effort in solving this problem and mostly an exercise in go.
deleted 4y ago
[deleted]

erwincoumans 4y ago

The discussion here reminded me about a blog post I made back in 2011, concluding with meta-meta-build systems. Unfortunately, Mike Acton didn't maintain the AltDevBlogADay blog due to technical (build system?) issues here a copy (warning, a wall of text!): ++++++++++++++ Erwin Coumans, 2011: "This is a story about my search for a hassle-free cross-platform open source (meta-/meta-meta-) build system. For our open source Bullet physics engine, I’ve been distributing the source code in a way that should make it as easy as possible to build out-of-the box. This means that for all supported platforms, the user (a developer who downloaded the Bullet SDK) should be able to download and unzip the zipfile or tarball on his machine, and get started as soon as possible. It should be hassle-free for the user but also for me, so I rather don’t manual updating too many files for each release. Aside from all the different platforms, we need to also support various compilers, compiler settings and integrated developer environments (IDEs). Common platforms are Windows, Mac OSX, iPhone, Linux as well as PlayStation 3 and XBox 360. Some common compilers are Visual Studio compiler, GCC, SN Systems and LLVM. Here are a couple of build systems and modifications that I tried or considered to try: Visual Studio project files. Shipping manual generated visual studio files works hassle-free for the user, but isn’t hassle-free for myself especially if you want to support all versions from MSVC 6 up to MSVC 2010. Obviously it is also not cross-platform so it needs to be combined with another build system. For example autotools. autotools: for most unix flavors, autotools does a great job. It is often already installed and is fairly hassle-free, as long as the user doesn’t run into version compatibilities of automake, autoconf or libtool. It is a bit of a hassle for me, because I’m not too familiar with autotools and can only provide most basic support. A bigger issue is that autotools doesn’t support IDEs such as Xcode. Xcode, the environment for Mac OSX and iPhone, requires its own project files. Shipping manual generated project files is a burden, when dealing with versions and platform settings. jam and msvcgen: jam is a fast build system and it is small enough to ship the entire jam source code together with my own project (it is just a few hundred kilobytes with no external dependencies). The msvcgen extension could autogenerate standalone Visual Studio project files from MSVC 6 onwards, so that was a nice extra. For several years I’ve been shipping the Bullet SDK with autotools + jam and msvcgen autogenerated Visual Studio projects for MSVC 6, 7 and 2005. No proper support for XCode, until I noticed cmake. cmake: this is almost a dream meta build system but unfortunately it cannot properly generate distributable build systems. For example the generated Visual Studio project files only work on the local machine. I’ve been hacking the cmake source code so that it does generate distributable project files, and submitted a few patches back to cmake. Unfortunately, the main cmake developers don’t want to support distributable project file generation, and I felt they were very reluctant to discuss further changes to support distributable project files. For example, I need the option to avoid creating copy rules that depend on cmake being installed. Furthermore any environment variables should not be expanded in absolute paths but stay environment variables for shipping project files (for example DXSDK_DIR, ATISTREAMSDKROOT or NVSDKCOMPUTE_ROOT for some recent OpenCL and DirectCompute accelerated optional modules). This means that I need to keep on using and updating a forked version of cmake to generate and distribute MSVC projectfiles. As a side note: the wxWidgets project even created their own build system, just because of this cmake shortcoming. Another shortcoming of cmake is lack of multiple compiler support, so generating projects for both PlayStation PPU and SPU is not possible. After bringing up my build system issue on the PS3 Devnet forums, someone mentioned premake. premake is a very promising meta build system and I keep on thinking of trying it out. I’ve heard people who create PS3 and XBox 360 project files, and it supports most other platforms. Premake can be optionally extended using Lua, but it is not a requirement for Lua to be installed, unlike the Python requirement for scons. Unfortunately, I’ve heard that premake has some issues with dependency tracking, has less features that cmake and it is not widely supported yet. This means that integrating it into another project might become a hassle. This leaves me to either wait until premake matures and becomes more widely used, or writing my own meta-meta build system that can generate both cmake, premake and other project files." ++++++++++++++++ Not a whole lot changed in those 11 years. Bullet focus moved to robotics and reinforment learning and Python/PyBullet and with that: setuptools as one more build system, so you can trivially install using: pip install pybullet

kgeist 4y ago

I once made something similar for my project as an excercise - it parsed the source code for #includes and figured out automatically what needs to be compiled. It was OKayish for my self-contained project which only used the standard library but it wouldn't work if you had to add third-party dependencies. Today I'd rather just use CMake.