8 ms·

A different approach to building C++ projects

45 points by kimmk 4y ago

ipgmonstereater 4y ago
[dead]
jjgreen 4y ago
One .cc/.h pair, one object Not always the case; I have a project with default.o: default.yaml $(LD) -r -b binary -o default.o default.yaml and a default.h containing extern const char _binary_default_yaml_start[]; extern const char _binary_default_yaml_end[]; #define PARAM_YAML _binary_default_yaml_start #define PARAM_YAML_LEN (_binary_default_yaml_end - _binary_default_yaml_start) this used in the main code as fwrite(PARAM_YAML, 1, PARAM_YAML_LEN, stdout); printing the contents of the yaml file to stdout.
- velcrovan 4y ago
  So, you do things in a way that would not support this approach. She’s not saying “this is always how it’s done”, she’s explaining what practices you would need to commit to in order for her approach to be viable: > “If you want something like this to work, you have to commit to a certain amount of consistency in your code base. You might have to throw out a few really nasty hacks that you've done in the past. It's entirely likely that most people are fully unwilling or unable to do this, and so they will continue to suffer. That's on them.”
- quietbritishjim 4y ago
  The point is that the tool is opinionated and demands that this be the case for projects that work with it. Not that the author believes all .cc / .h files work that way. Your use case would be served by C23's #embed [1]. The same thing has been proposed for C++ but repeatedly kicked down the road because the standardisation committee wanted to make it more general even though no one had any demand for that so they didn't know what it would look like. (C++ standardisation in a nutshell.) [1] https://thephd.dev/finally-embed-in-c23 https://thephd.dev/finally-embed-in-c23
  - jjgreen 4y ago
    Wow, I didn't know about #embed, that will make life so much easier -- thanks!
Sakos 4y ago
This is sort of unrelated, but that reminds me that one of my biggest issues with learning C++ was how I was expected to deal with libraries (particularly on Linux, where conventions will even differ between distros) and building the project. Most guides or what have you sort of teach you how to compile a file or two, but you quickly run into issues that are difficult to solve for a complete beginner without a direct source of feedback.
- compacct27 4y ago
  I agree, it’s so horrible. Passing flags to a compiler for some Byzantine rule just to change lord knows what, but it all works now. What on earth.
  - Sakos 4y ago
    I wish somebody would write a book. Even an online book. An all-encompassing book just about how to link/build C++ projects and all the different solutions people use and how they work in practice. Common ways to organize and configure C++ project builds. How Linux distros each differ in where libraries are stored and how to find and link them. Common issues and how to fix them. But it also needs to convey how all these things work and help you create a mental model that allows you to also find your own solutions.
- xnorswap 4y ago
  Every time I've tried to dabble in C++ I've had the same horrible experience. I end up "Randomly" stabbing at things until it works just well enough to get that particular thing done then dropping it all because it was such a painful experience. Compared to something like cargo which works really well, C++ and it's build tools just feel flaky. It may be that I'm just missing a mental model to get to grips with it, but no other major programming language is like that from my experience.
  - moloch-hai 4y ago
    In Behavioral Science this phenomenon is called "learned helplessness". The rabbit will not flee the cage even with the door propped open and no one around. Theon Greyjoy in "Game of Thrones" exhibited this condition. It is a thing to overcome.
jbandela1 4y ago
One of the nice things that Visual C++ has is #pragma comment(lib, "xxx.lib") You can specify it in a header file for the library. That way if you include the header file, the library mentioned will automatically get linked as long as it is somewhere in the library search path. I have found myself wishing that GCC would also get something like this.
- flohofwoe 4y ago
  I wish that would work for all build settings, and be standardized across compilers. Even building complex projects with platform-specific build settings could then be reduced to a simple: cc main.c -o bla
- Joker_vD 4y ago
  I remember back when I was programming in Delphi I could link directly against a .dll, just take a function prototype from the .h file and translate it into a function declaration like this one: function I2C_GetNumChannels(out numChannels: Longword): FT_Result; stdcall; external 'libmpsse.dll'; and that was it; but to do this in MSVC you needed not only the .h header and the .dll itself, you also needed that stupid .lib file that had AFAYCT had literally nothing inside it except symbol entries that said "no, load it dynamically from this .dll on startup, please". So it was a rather common source of amusement for Delphi programmers that paradoxically, it was harder to link a program written in C against a DLL written in C than it was to link a program written in Delphi against a DLL written in C.
  - FpUser 4y ago
    Delphi made an awful lot of things incredibly easy. Com automation for example. It is just too bad Borland had fucked up.
    - pjmlp 4y ago
      It boggles my mind that for how hardline the WinDev is about using COM, they still fail to match Borland, nowadays Embarcadero tooling for COM. For a brief moment they almost had it with .NET Native and C++/CX, and then, first they killed C++/CX in name of C++/WinRT (with VS tooling just like in the good old ATL days), and with UWP's deprecation, CsWinRT also fails quite short of the .NET Native experience in regards to COM. How a OS development team that is so invested into COM APIs, fails to produce tooling better than the competition for 25 years escapes me.
IshKebab 4y ago
This sounds like it would be fine for code that you write yourself. But if you're only compiling code you wrote then C++ build systems are pretty trivial. The hard bit is dependencies.
- wilburm 4y ago
  One big lesson that newer languages like go and rust seemed to have learned is that the tooling, building and dependency management need to be dictated as part of the language ecosystem. Dealing with tons of other C++ projects written by other people (even in the same company) - how to specify dependencies, where their build artifacts can be found, etc - is a HUGE pain in the ass and consumer of my time.
  - bluGill 4y ago
    They all get the tooling wrong though because none can stand the idea that you might want to mix languages, or add their new language to an existing project with existing tooling.
    - kibwen 4y ago
      For Go, mixing languages is uncommon because its FFI suffers an impedance mismatch with its task scheduler. For Rust, mixing languages is extremely common. Rust's entire original reason for existing was to rewrite small parts of a large C++ project.
      - pjmlp 4y ago
        So much common that Google had to create their own integrations for Android and Fuchsia. I bet the announced Chrome efforts will again, require another adaptation.
        kibwen 4y ago
        I'm unsure what this is trying to say. You appear to be in begrudging agreement that Rust is commonly mixed with other languages?
deleted 4y ago
[deleted]
gavinray 4y ago
This is sort of the principle of the "bpt" build tool I think from "vector-of-bool" https://bpt.pizza/ https://bpt.pizza/
ChrisMarshallNY 4y ago
> If you want something like this to work, you have to commit to a certain amount of consistency in your code base. That goes for almost everything, in developing ship code. Today, I am in the initial stages of rewriting an app with a codebase that has “accreted” over two years. It’s kind of a mess (my mess, to be clear). I’ll be adding a great deal of rigor to the new code. I think it will come out great, but I have my work cut out for me.
boris 4y ago
I wish we had a culture that expected every new C/C++ build system to handle a non-trivial project like Boost or Qt (and their dependencies, like ICU and OpenSSL) before it being pitched as the new best thing. It's trivial to make a build system that elegantly handles your own toy projects that follow your preferred style and structure. But the real world of C/C++ is a harsh place with a lot of variability.
- whatisyour 4y ago
  [dead]
- matthewaveryusa 4y ago
  This is wisdom right here. It's tragedy of the commons. I find that the real problem is no one wants to properly learn how their build system works. I don't care if it's make, cmake or bazel -- whatever it is, you need to _learn_ it. I've worked with folks that have 20 years of experience, fantastic C/C++ developers, that look at a makefile and say "Ugh what is this complicated mess, can you do it in cmake or bazel or something" and expect a silver-bullet where-in the makefile build will somehow transform itself into a self-describing intuitive build system by virtue of some sort of hype-osmosis.
  - saidinesh5 4y ago
    > I've worked with folks that have 20 years of experience that look at a makefile and say "Ugh what is this complicated mess, can you do it in cmake or bazel or something" This is so true, it happened to me more than once. A couple of projects ago, we had a complicated build process (7-8 manual build steps that depend on files generated from each other before) for an embedded system. I wrote a little makefile deleting all those 7-8 shell scripts and i was asked to re do it in cmake.. I was like wtf.. each clearly defined step in makefile would turn into multiple unreadable function calls in cmake.. why would anyone want to do that.. Not that Makefiles are perfect, but sometimes, the right tool for the job isn't the shiniest. Make does a job of being "good enough" for a lot of little tasks.
    - 4y ago
bogwog 4y ago
Conan + (any build system) = problem solved Conan has a learning curve, but it’s totally worth it. Anyone making their own build system should get some experience with a state of the art package manager before writing a single line of code, because chances are that it already solves whatever problem is motivating you.
- annowiki 4y ago
  I started as a python programmer and was very used to package managers. I believed in them, I championed them. When I switched to C++ for work I was very disheartened that there wasn't a standard. Conan obviously has promise, I haven't spent much time with it, most of my experience with C++ package managers is with nuget and vcpkg. However, my attitude toward package managers is changing. I increasingly like _not_ using package managers because it makes me (and my company) way way way less likely to bloat our software with unnecessary third party dependencies. I wrote this in another thread: I never believed you should write something yourself if you can find a package for it. My boss told me I should write it all myself, I could probably write it to be faster. I encountered a case where I needed to compare version numbers in python. For the heck of it I wrote the simplest, quickest, most naive solution I could come up with and then timed it against the most recommended version comparison package in python. I blew it away by 20x throughput. I don't believe in package managers anymore. Obviously I'll keep using pip and sqlalchemy in Python, but I'll happily spend the 20-30 minutes it takes adding something like nlohmann-json or md4c to my project over worrying about maintaining a package manager for c++ these days. Precisely because it makes me think twice about adding another dependency.
  - planede 4y ago
    Sometimes you have dependencies with actual value add that you really don't want to replicate. No, I'm totally not writing a yaml parser, thank you very much. I can probably write a good yaml parser, possibly even better than some 3rd party stuff, but yaml parsing is simply not our business. And yaml parsing is probably on the simpler side of things. We need to run torch models, we do need libtorch. We are not rewriting libtorch, that would be silly.
  - 4y ago
db48x 4y ago
Has everyone forgotten about deps files? Run gcc -MD and it will create .d files that record the dependencies between your source (and header) files. You can then use an include directive in your Makefile to pull that information in for make to use. There are a couple of variations on the theme; some people recommend putting the .d files alongside your source files, others recommend a specific “deps” directory for them, etc. See the man page for details, with particular reference to options like -M, -MM, -MF, -MD, and -MMD. Of course, the other alternative is to simply #include _every_ file in your project into a single source file, then compile that. It’ll probably be faster than anything else you do, and eliminates several other foot–guns as well. And it means that your build script can just be a shell script with a single line that runs the compiler on that one file. But these days I greatly prefer Rust, where building is always just “cargo build”. Doesn’t get much easier than that.
- dezgeg 4y ago
  How does gcc -MD help at all in tracking which .cpp files to link in? > Of course, the other alternative is to simply #include _every_ file in your project into a single source file, then compile that. Yeah, no... recompiling the entire project whenever any file is touched is way too slow for any non-trivial project.
  - db48x 4y ago
    Most projects don’t have a lot of .cpp files that they don’t use :) You’re ultimately going to use them all.
    - klyrs 4y ago
      Most people only change one or two files at a time. Those files, and the few that depend on them, must be rebuilt upon a change. Incremental builds only do what's necessary. Your proposal always builds everything no matter how trivial the change. It does not scale.
      - db48x 4y ago
breakds 4y ago
I have been managing all my existing and new projects with nix for a few years and never look back. Guaranteed to build and run on all my machines. nix flake new --template "github:nixvital/flake-templates#cpp-starter-kit" my-project will create a skeleton for my new C++ projects.

45 points by kimmk 4y ago

[dead]

One .cc/.h pair, one object Not always the case; I have a project with default.o: default.yaml $(LD) -r -b binary -o default.o default.yaml and a default.h containing extern const char _binary_default_yaml_start[]; extern const char _binary_default_yaml_end[]; #define PARAM_YAML _binary_default_yaml_start #define PARAM_YAML_LEN (_binary_default_yaml_end - _binary_default_yaml_start) this used in the main code as fwrite(PARAM_YAML, 1, PARAM_YAML_LEN, stdout); printing the contents of the yaml file to stdout.

velcrovan 4y ago
So, you do things in a way that would not support this approach. She’s not saying “this is always how it’s done”, she’s explaining what practices you would need to commit to in order for her approach to be viable: > “If you want something like this to work, you have to commit to a certain amount of consistency in your code base. You might have to throw out a few really nasty hacks that you've done in the past. It's entirely likely that most people are fully unwilling or unable to do this, and so they will continue to suffer. That's on them.”
quietbritishjim 4y ago
The point is that the tool is opinionated and demands that this be the case for projects that work with it. Not that the author believes all .cc / .h files work that way. Your use case would be served by C23's #embed [1]. The same thing has been proposed for C++ but repeatedly kicked down the road because the standardisation committee wanted to make it more general even though no one had any demand for that so they didn't know what it would look like. (C++ standardisation in a nutshell.) [1] https://thephd.dev/finally-embed-in-c23 https://thephd.dev/finally-embed-in-c23
- jjgreen 4y ago
  Wow, I didn't know about #embed, that will make life so much easier -- thanks!

Sakos 4y ago

This is sort of unrelated, but that reminds me that one of my biggest issues with learning C++ was how I was expected to deal with libraries (particularly on Linux, where conventions will even differ between distros) and building the project. Most guides or what have you sort of teach you how to compile a file or two, but you quickly run into issues that are difficult to solve for a complete beginner without a direct source of feedback.

compacct27 4y ago
I agree, it’s so horrible. Passing flags to a compiler for some Byzantine rule just to change lord knows what, but it all works now. What on earth.
- Sakos 4y ago
  I wish somebody would write a book. Even an online book. An all-encompassing book just about how to link/build C++ projects and all the different solutions people use and how they work in practice. Common ways to organize and configure C++ project builds. How Linux distros each differ in where libraries are stored and how to find and link them. Common issues and how to fix them. But it also needs to convey how all these things work and help you create a mental model that allows you to also find your own solutions.
xnorswap 4y ago
Every time I've tried to dabble in C++ I've had the same horrible experience. I end up "Randomly" stabbing at things until it works just well enough to get that particular thing done then dropping it all because it was such a painful experience. Compared to something like cargo which works really well, C++ and it's build tools just feel flaky. It may be that I'm just missing a mental model to get to grips with it, but no other major programming language is like that from my experience.
- moloch-hai 4y ago
  In Behavioral Science this phenomenon is called "learned helplessness". The rabbit will not flee the cage even with the door propped open and no one around. Theon Greyjoy in "Game of Thrones" exhibited this condition. It is a thing to overcome.

jbandela1 4y ago

One of the nice things that Visual C++ has is #pragma comment(lib, "xxx.lib") You can specify it in a header file for the library. That way if you include the header file, the library mentioned will automatically get linked as long as it is somewhere in the library search path. I have found myself wishing that GCC would also get something like this.

flohofwoe 4y ago
I wish that would work for all build settings, and be standardized across compilers. Even building complex projects with platform-specific build settings could then be reduced to a simple: cc main.c -o bla
Joker_vD 4y ago
I remember back when I was programming in Delphi I could link directly against a .dll, just take a function prototype from the .h file and translate it into a function declaration like this one: function I2C_GetNumChannels(out numChannels: Longword): FT_Result; stdcall; external 'libmpsse.dll'; and that was it; but to do this in MSVC you needed not only the .h header and the .dll itself, you also needed that stupid .lib file that had AFAYCT had literally nothing inside it except symbol entries that said "no, load it dynamically from this .dll on startup, please". So it was a rather common source of amusement for Delphi programmers that paradoxically, it was harder to link a program written in C against a DLL written in C than it was to link a program written in Delphi against a DLL written in C.
- FpUser 4y ago
  Delphi made an awful lot of things incredibly easy. Com automation for example. It is just too bad Borland had fucked up.
  - pjmlp 4y ago
    It boggles my mind that for how hardline the WinDev is about using COM, they still fail to match Borland, nowadays Embarcadero tooling for COM. For a brief moment they almost had it with .NET Native and C++/CX, and then, first they killed C++/CX in name of C++/WinRT (with VS tooling just like in the good old ATL days), and with UWP's deprecation, CsWinRT also fails quite short of the .NET Native experience in regards to COM. How a OS development team that is so invested into COM APIs, fails to produce tooling better than the competition for 25 years escapes me.

IshKebab 4y ago

This sounds like it would be fine for code that you write yourself. But if you're only compiling code you wrote then C++ build systems are pretty trivial. The hard bit is dependencies.

wilburm 4y ago
One big lesson that newer languages like go and rust seemed to have learned is that the tooling, building and dependency management need to be dictated as part of the language ecosystem. Dealing with tons of other C++ projects written by other people (even in the same company) - how to specify dependencies, where their build artifacts can be found, etc - is a HUGE pain in the ass and consumer of my time.
- bluGill 4y ago
  They all get the tooling wrong though because none can stand the idea that you might want to mix languages, or add their new language to an existing project with existing tooling.
  - kibwen 4y ago
    For Go, mixing languages is uncommon because its FFI suffers an impedance mismatch with its task scheduler. For Rust, mixing languages is extremely common. Rust's entire original reason for existing was to rewrite small parts of a large C++ project.
    - pjmlp 4y ago
      So much common that Google had to create their own integrations for Android and Fuchsia. I bet the announced Chrome efforts will again, require another adaptation.
      - kibwen 4y ago
        I'm unsure what this is trying to say. You appear to be in begrudging agreement that Rust is commonly mixed with other languages?

deleted 4y ago

[deleted]

gavinray 4y ago

This is sort of the principle of the "bpt" build tool I think from "vector-of-bool" https://bpt.pizza/ https://bpt.pizza/

ChrisMarshallNY 4y ago

> If you want something like this to work, you have to commit to a certain amount of consistency in your code base. That goes for almost everything, in developing ship code. Today, I am in the initial stages of rewriting an app with a codebase that has “accreted” over two years. It’s kind of a mess (my mess, to be clear). I’ll be adding a great deal of rigor to the new code. I think it will come out great, but I have my work cut out for me.

boris 4y ago

I wish we had a culture that expected every new C/C++ build system to handle a non-trivial project like Boost or Qt (and their dependencies, like ICU and OpenSSL) before it being pitched as the new best thing. It's trivial to make a build system that elegantly handles your own toy projects that follow your preferred style and structure. But the real world of C/C++ is a harsh place with a lot of variability.

whatisyour 4y ago
[dead]
matthewaveryusa 4y ago
This is wisdom right here. It's tragedy of the commons. I find that the real problem is no one wants to properly learn how their build system works. I don't care if it's make, cmake or bazel -- whatever it is, you need to _learn_ it. I've worked with folks that have 20 years of experience, fantastic C/C++ developers, that look at a makefile and say "Ugh what is this complicated mess, can you do it in cmake or bazel or something" and expect a silver-bullet where-in the makefile build will somehow transform itself into a self-describing intuitive build system by virtue of some sort of hype-osmosis.
- saidinesh5 4y ago
  > I've worked with folks that have 20 years of experience that look at a makefile and say "Ugh what is this complicated mess, can you do it in cmake or bazel or something" This is so true, it happened to me more than once. A couple of projects ago, we had a complicated build process (7-8 manual build steps that depend on files generated from each other before) for an embedded system. I wrote a little makefile deleting all those 7-8 shell scripts and i was asked to re do it in cmake.. I was like wtf.. each clearly defined step in makefile would turn into multiple unreadable function calls in cmake.. why would anyone want to do that.. Not that Makefiles are perfect, but sometimes, the right tool for the job isn't the shiniest. Make does a job of being "good enough" for a lot of little tasks.
  - 4y ago

bogwog 4y ago

Conan + (any build system) = problem solved Conan has a learning curve, but it’s totally worth it. Anyone making their own build system should get some experience with a state of the art package manager before writing a single line of code, because chances are that it already solves whatever problem is motivating you.

annowiki 4y ago
I started as a python programmer and was very used to package managers. I believed in them, I championed them. When I switched to C++ for work I was very disheartened that there wasn't a standard. Conan obviously has promise, I haven't spent much time with it, most of my experience with C++ package managers is with nuget and vcpkg. However, my attitude toward package managers is changing. I increasingly like _not_ using package managers because it makes me (and my company) way way way less likely to bloat our software with unnecessary third party dependencies. I wrote this in another thread: I never believed you should write something yourself if you can find a package for it. My boss told me I should write it all myself, I could probably write it to be faster. I encountered a case where I needed to compare version numbers in python. For the heck of it I wrote the simplest, quickest, most naive solution I could come up with and then timed it against the most recommended version comparison package in python. I blew it away by 20x throughput. I don't believe in package managers anymore. Obviously I'll keep using pip and sqlalchemy in Python, but I'll happily spend the 20-30 minutes it takes adding something like nlohmann-json or md4c to my project over worrying about maintaining a package manager for c++ these days. Precisely because it makes me think twice about adding another dependency.
- planede 4y ago
  Sometimes you have dependencies with actual value add that you really don't want to replicate. No, I'm totally not writing a yaml parser, thank you very much. I can probably write a good yaml parser, possibly even better than some 3rd party stuff, but yaml parsing is simply not our business. And yaml parsing is probably on the simpler side of things. We need to run torch models, we do need libtorch. We are not rewriting libtorch, that would be silly.
- 4y ago

db48x 4y ago

Has everyone forgotten about deps files? Run gcc -MD and it will create .d files that record the dependencies between your source (and header) files. You can then use an include directive in your Makefile to pull that information in for make to use. There are a couple of variations on the theme; some people recommend putting the .d files alongside your source files, others recommend a specific “deps” directory for them, etc. See the man page for details, with particular reference to options like -M, -MM, -MF, -MD, and -MMD. Of course, the other alternative is to simply #include _every_ file in your project into a single source file, then compile that. It’ll probably be faster than anything else you do, and eliminates several other foot–guns as well. And it means that your build script can just be a shell script with a single line that runs the compiler on that one file. But these days I greatly prefer Rust, where building is always just “cargo build”. Doesn’t get much easier than that.

dezgeg 4y ago
How does gcc -MD help at all in tracking which .cpp files to link in? > Of course, the other alternative is to simply #include _every_ file in your project into a single source file, then compile that. Yeah, no... recompiling the entire project whenever any file is touched is way too slow for any non-trivial project.
- db48x 4y ago
  Most projects don’t have a lot of .cpp files that they don’t use :) You’re ultimately going to use them all.
  - klyrs 4y ago
    Most people only change one or two files at a time. Those files, and the few that depend on them, must be rebuilt upon a change. Incremental builds only do what's necessary. Your proposal always builds everything no matter how trivial the change. It does not scale.
    - db48x 4y ago

breakds 4y ago

I have been managing all my existing and new projects with nix for a few years and never look back. Guaranteed to build and run on all my machines. nix flake new --template "github:nixvital/flake-templates#cpp-starter-kit" my-project will create a skeleton for my new C++ projects.