7 ms·

Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management plat

Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform? Managing dependencies in C++ is a nightmare, and I'm not even talking about the lack of actual modules inside the language itself. It's so bad that people (incl. me) tend to prefer to copy-paste libraries source code instead of setting up a dependency. A Python, Ruby, Node or Java project would never have to head toward such a poor solution instead of using their respective package managers. But in C++ we do because the alternative (e.g. Conan) is such a nightmare to use.

flohofwoe 6y ago
Exactly, and what's especially elegant about single-file-libs is that they solve a real-world problem without requiring any new tooling, and you don't need to convince anybody to use your favourite build system or depedency manager (which is the main problem - you need to have a single standard solution right from the start, any time later is too late). PS: the original motiviation for the STB single-file libs was actually that Windows doesn't have a default place where dependencies are installed: https://github.com/nothings/stb#why-single-file-headers https://github.com/nothings/stb#why-single-file-headers
- humanrebar 6y ago
  It helps with the problem, but doesn't solve it. Just because you have five header files instead of five packages doesn't mean you can use all five headers at the same time. For example, some might require C++03 and some might need C++14 or newer. Also, if all dependencies are single file with no non-standard dependencies, that means none share dependencies they should, like threading, telemetry, or logging libraries.
  - CyberDildonics 6y ago
    Do you have examples of this or is it just a guess? Anything in its own compilation unit would be isolated. Anything written using C++03 should compile without too much trouble in a more modern compiler. Also threading is part of the C++ standard library. I don't know what you mean by sharing dependencies on telemetry and logging. That sounds like adding questionable complexity to libraries that are simple and modular.
    - johannes1234321 6y ago
      > Anything in its own compilation unit would be isolated In isolation yes. But if it interacts you get events like the fact that gcc changed std::string between versions (from refcounted to one with small buffer optimisation for C++11 compliance) also introduction of move semantics and rvalue references can change interpretation of the same class declaration between compilers in different modes.
      - CyberDildonics 6y ago
        I have never heard of people upgrading their compiler and/or standard libraries, but expecting to not have to recompile their compilation units. You seem to be talking about major changes to the fundamental set up of a project while not taking a comparably trivial amount of time recompiling compilation units. How often are you changing major compiler versions and breaking standard library versions that you would have this expectation?
        fnord123 6y ago
        > I have never heard of people upgrading their compiler and/or standard libraries, but expecting to not have to recompile their compilation units. You've never linked to a binary? I have no idea what debian's Qt was build with or libssl or SDL etc were built with but we can link to them all the same. > How often are you changing major compiler versions and breaking standard library versions that you would have this expectation? You don't configure Jenkins to build against multiple compilers/versions of compiler? Isn't that super normal?
        CyberDildonics 6y ago
        You are conflating the C ABI with the C++ ABI. The C ABI is stable, so linking to SDL works fine. The C++ ABI is actually relatively stable as far as I can tell, but ABI breaking changes are announced when they are released by compiler vendors.
        6y ago
  - flohofwoe 6y ago
    ...shared dependencies are more often a problem and not a solution, even when a standard package manager exists (see the deep dependency trees in many Javascript projects where nobody really knows what's actually going on and what code ends up running).
- mschuetz 6y ago
  > PS: the original motiviation for the STB single-file libs was actually that Windows doesn't have a default place where dependencies are installed I've had so many issues with shared libs on linux in the past, I gave up on it and started to include all the necessary source of any third party library in my projects. Makes building and distributing things much easier.
  - fnord123 6y ago
    In other language build environments it's called vendorising and Ruby, Go, and Rust offer tooling to do this for the various benefits (hermetic builds, dealing with down infra, not molesting third party source control, not leaking information about your dependencies to third parties).
_pmf_ 6y ago
> Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform? No, I think there is no other way to see it.
callesgg 6y ago
There is correlation there yes! But sometimes you just want something simple and easy and package managers comes with so much extra stuff that is very useful in more complex projects. Those things can be a major annoyance when you just want something simple.
enriquto 6y ago
> Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform? Single files are a modern and unified package management platform. You copy the file and you use it. It does not get any simpler than that. You are talking as if there was a kind of compromise between some imaginary disadvantages of single files. There are none, and there is no compromise. Single files are alright.
quietbritishjim 6y ago
vcpkg is a decent enough package manager to making using it worthwhile, despite the occasional problem. Certainly, if anyone finds that things have got desperate enough that they're looking the OP's list of single-file libraries then they should just take the one-time hit of setting their project up to use vcpkg. Once that's done, using a supported library reduces to running `vcpkg install foo`, regardless of how many source files it has (or how many transitive dependencies for that matter).
thisiswas 6y ago
I find single file libs superior to typical package managers. What is more ergonomic than - download file, drop into your project, include and start coding, and I can use my favorite build system/way of setting up projects, no need to integrate anything. Can trivially support multiple incompatible versions of the library, nothing needs to be fetched or resolved (once you've got the single file of course), can send and distribute it easily over any channel you want (web, email, free file host, google drive, your own website, etc).
- KptMarchewa 6y ago
  >What is more ergonomic than - download file, drop into your project, include and start coding, and I can use my favorite build system/way of setting up projects, no need to integrate anything. This approach lacks any features that reasonable dependency manager provides, such as providing updates compliant with semver.
- quietbritishjim 6y ago
  There are lots of problems with single-file libraries. Here are a few but I'm sure there are others I've forgotten or haven't thought of. * You have to wait for compilation of the whole library at least once every time you do a build of your program - certainly every time you do a clean build, and potentially even incremental builds if it's header only. * If the library is header only (many of the linked libraries are) then you you potentially have to pay that compilation cost more than once per compilation of your program - once per every one of your source files that include it. * Again this is specific to header-only libraries, but to avoid code bloat you'll need to turn on link-time optimisation which is far slower than just allowing the linker to do its job by only compiling definitions into a single object file. (Admittedly LTO is a good idea anyway, but adding a bunch of duplicated symbols is avoidable extra work for it.) * Some useful libraries are realistically just too big for their authors to write the whole thing in one file (e.g. protobuf, opencv, ... in fact most libraries I use on a regular basis seem to fall into that). They could "release" the library in single-file format, similar to SQLite's amalgam, but then if there are any problems (either a bug in their code or something in your code that makes you want to look at their code) you're now not looking at the original source but some mangled version of it. * If the library is so large that its interface needs to be split over multiple headers (think Boost or OpenCV) then you're now bang out of luck. Hopefully the library has cleanly-enough separated modules you could potentially release these separately (e.g. OpenCV core, imgproc, imgcodecs, highgui, ...) but then you're essentially back to multi-file libraries. * Adding a library with a lot of its own transitive depedencies takes proportionally the amount of effort as the number of those dependencies, rather than being handled automatically. One interesting thing about all of these problems is that they get worse and worse as you need more libraries in your program, or need a larger library for your program. In contrast, using a package manager (I'm thinking particularly vcpkg here) tends to add a one-time cost at the start but allows you to scale your dependency list almost for free. If you're writing your own library, rather than a application, then there are even more problems with this approach, but I won't quite open that can of worms.
mmm_grayons 6y ago
You know what's funny? I used to think this too, but I just started a new project with vcpkg and it works great. I understand that Conan is supposed to be good as well, though slightly less user-friendly. I can just define all of my dependencies in a json manifest, set my cmake toolchain, and run `find_package`, `include_directories`, and `target_link_libraries` as normal. It's great.
fnord123 6y ago
You are not the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform because the single-file lib trend is indeed a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform.
junon 6y ago
This is a bit of a misunderstanding of the C++/C way of doing things. It makes sense in some ways but clearly there are better ways of doing it. Which is precisely why C++20 has modules support. Further, C++ does have a central package manager - whatever the system distributes. C/C++ programs constantly link against the operating system and thus have to get those system headers from somewhere. This means a universal package manager (like Conan) requires buy-in from OS vendors or at least someone who can manage those packages with conviction, else the package manager doesn't make a lot of sense to use. This is contrast to Node, Ruby and Python as they are inherently Cross-Platform in nature and are not coupled tightly to the OS. This is why C/C++, historically, have not had a strong centralized package manager. Anymore, I see less and less package management being used anyway. Most C/C++ projects I work with either vendor in dependencies or use git submodules (the latter I prefer very much). This is because, unlike e.g. the Node community, micro-dependencies are generally not worth the effort to bring in. The usual exception to this sort of structure are codebases that will be distributed via a package manager - in which case, they generally rely on the package manager supplying correct versions of the dependencies. These header-only libs work OK usually because they are able to be included with trivial linkage (something else you normally don't need to care about in scripting environments). Linkage is something you have to care about regardless of if you're writing C, C++ or machine code and since these languages give you pretty much free reign on the flexibility of all of these parameters, it's hard to generalize everything into a nice package like scripting languages can. Usually the functions inside header only libraries will be static inline, and will almost certainly increase compilation time (noticeably so if you use the header in many places). There is work being done to improve this situation, it's not like we're all just sitting here going "yes, we love the way things are and have no idea how to make things better." That's far from the case. Shit just takes time.
jasode 6y ago
>Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform? One counterpoint to your proposed cause & effect is that some observers think Javascript NPM's packaging convenience enables the explosion of single file dependencies. Example discussion: https://news.ycombinator.com/item?id=11348798 https://news.ycombinator.com/item?id=11348798
- klodolph 6y ago
  What’s happening in the JavaScript ecosystem is dependencies are so easy that you’re getting a lot of small libraries, and they’re one file because they’re small. What’s happening in C++ is that you’re getting one-file libraries that are not small, because developers will shoehorn their library into a single file rather rather than deal with a way of managing C++ library dependencies. You might find in C++, for example, a header file with over 10k lines in it and a bunch of preprocessor conditionals. What you might find in JavaScript is a library that contains one or two functions.
pjmlp 6y ago
I see it more as trend for the You Tube generation unwilling to learn how compiled languages work. Maybe we should a couple of You Tube videos about linkers and stuff.
- snazz 6y ago
  This is just a natural consequence of programming becoming more accessible, I think. And let's not pretend that C and C++ package management is "good but misunderstood"—it needs a lot of work.
  - pjmlp 6y ago
    Yeah, work like reading a book.
    - snazz 6y ago
      You can understand how C/C++ builds and dependencies work by reading a book, yes. But you can still criticize it—there's lots of stuff there to criticize! Modern languages have shown that dependency management and builds can be easier, faster, more secure, and more portable.
      - pjmlp 6y ago
        Which is why Conan and vcpkg do exist, and for quite long time OS package managers and installers.
- raphaelj 6y ago
  This does not have anything to do with knowledge. C/C++ build systems are archaic if not broken. I literally wrote a compiler but still spend 15/20% of my development time trying to find the right not user-friendly CMake syntax, understanding while my Conan dependency broke overnight, or writing header files that could be automatically generated by the compiler (à la GHC).
  - 6y ago

flohofwoe 6y ago

Exactly, and what's especially elegant about single-file-libs is that they solve a real-world problem without requiring any new tooling, and you don't need to convince anybody to use your favourite build system or depedency manager (which is the main problem - you need to have a single standard solution right from the start, any time later is too late). PS: the original motiviation for the STB single-file libs was actually that Windows doesn't have a default place where dependencies are installed: https://github.com/nothings/stb#why-single-file-headers https://github.com/nothings/stb#why-single-file-headers

humanrebar 6y ago
It helps with the problem, but doesn't solve it. Just because you have five header files instead of five packages doesn't mean you can use all five headers at the same time. For example, some might require C++03 and some might need C++14 or newer. Also, if all dependencies are single file with no non-standard dependencies, that means none share dependencies they should, like threading, telemetry, or logging libraries.
- CyberDildonics 6y ago
  Do you have examples of this or is it just a guess? Anything in its own compilation unit would be isolated. Anything written using C++03 should compile without too much trouble in a more modern compiler. Also threading is part of the C++ standard library. I don't know what you mean by sharing dependencies on telemetry and logging. That sounds like adding questionable complexity to libraries that are simple and modular.
  - johannes1234321 6y ago
    > Anything in its own compilation unit would be isolated In isolation yes. But if it interacts you get events like the fact that gcc changed std::string between versions (from refcounted to one with small buffer optimisation for C++11 compliance) also introduction of move semantics and rvalue references can change interpretation of the same class declaration between compilers in different modes.
    - CyberDildonics 6y ago
      I have never heard of people upgrading their compiler and/or standard libraries, but expecting to not have to recompile their compilation units. You seem to be talking about major changes to the fundamental set up of a project while not taking a comparably trivial amount of time recompiling compilation units. How often are you changing major compiler versions and breaking standard library versions that you would have this expectation?
      - fnord123 6y ago
        > I have never heard of people upgrading their compiler and/or standard libraries, but expecting to not have to recompile their compilation units. You've never linked to a binary? I have no idea what debian's Qt was build with or libssl or SDL etc were built with but we can link to them all the same. > How often are you changing major compiler versions and breaking standard library versions that you would have this expectation? You don't configure Jenkins to build against multiple compilers/versions of compiler? Isn't that super normal?
        CyberDildonics 6y ago
        You are conflating the C ABI with the C++ ABI. The C ABI is stable, so linking to SDL works fine. The C++ ABI is actually relatively stable as far as I can tell, but ABI breaking changes are announced when they are released by compiler vendors.
        6y ago
- flohofwoe 6y ago
  ...shared dependencies are more often a problem and not a solution, even when a standard package manager exists (see the deep dependency trees in many Javascript projects where nobody really knows what's actually going on and what code ends up running).
mschuetz 6y ago
> PS: the original motiviation for the STB single-file libs was actually that Windows doesn't have a default place where dependencies are installed I've had so many issues with shared libs on linux in the past, I gave up on it and started to include all the necessary source of any third party library in my projects. Makes building and distributing things much easier.
- fnord123 6y ago
  In other language build environments it's called vendorising and Ruby, Go, and Rust offer tooling to do this for the various benefits (hermetic builds, dealing with down infra, not molesting third party source control, not leaking information about your dependencies to third parties).

_pmf_ 6y ago

> Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform? No, I think there is no other way to see it.

callesgg 6y ago

There is correlation there yes! But sometimes you just want something simple and easy and package managers comes with so much extra stuff that is very useful in more complex projects. Those things can be a major annoyance when you just want something simple.

enriquto 6y ago

> Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform? Single files are a modern and unified package management platform. You copy the file and you use it. It does not get any simpler than that. You are talking as if there was a kind of compromise between some imaginary disadvantages of single files. There are none, and there is no compromise. Single files are alright.

quietbritishjim 6y ago

vcpkg is a decent enough package manager to making using it worthwhile, despite the occasional problem. Certainly, if anyone finds that things have got desperate enough that they're looking the OP's list of single-file libraries then they should just take the one-time hit of setting their project up to use vcpkg. Once that's done, using a supported library reduces to running `vcpkg install foo`, regardless of how many source files it has (or how many transitive dependencies for that matter).

thisiswas 6y ago

I find single file libs superior to typical package managers. What is more ergonomic than - download file, drop into your project, include and start coding, and I can use my favorite build system/way of setting up projects, no need to integrate anything. Can trivially support multiple incompatible versions of the library, nothing needs to be fetched or resolved (once you've got the single file of course), can send and distribute it easily over any channel you want (web, email, free file host, google drive, your own website, etc).

KptMarchewa 6y ago
>What is more ergonomic than - download file, drop into your project, include and start coding, and I can use my favorite build system/way of setting up projects, no need to integrate anything. This approach lacks any features that reasonable dependency manager provides, such as providing updates compliant with semver.
quietbritishjim 6y ago
There are lots of problems with single-file libraries. Here are a few but I'm sure there are others I've forgotten or haven't thought of. * You have to wait for compilation of the whole library at least once every time you do a build of your program - certainly every time you do a clean build, and potentially even incremental builds if it's header only. * If the library is header only (many of the linked libraries are) then you you potentially have to pay that compilation cost more than once per compilation of your program - once per every one of your source files that include it. * Again this is specific to header-only libraries, but to avoid code bloat you'll need to turn on link-time optimisation which is far slower than just allowing the linker to do its job by only compiling definitions into a single object file. (Admittedly LTO is a good idea anyway, but adding a bunch of duplicated symbols is avoidable extra work for it.) * Some useful libraries are realistically just too big for their authors to write the whole thing in one file (e.g. protobuf, opencv, ... in fact most libraries I use on a regular basis seem to fall into that). They could "release" the library in single-file format, similar to SQLite's amalgam, but then if there are any problems (either a bug in their code or something in your code that makes you want to look at their code) you're now not looking at the original source but some mangled version of it. * If the library is so large that its interface needs to be split over multiple headers (think Boost or OpenCV) then you're now bang out of luck. Hopefully the library has cleanly-enough separated modules you could potentially release these separately (e.g. OpenCV core, imgproc, imgcodecs, highgui, ...) but then you're essentially back to multi-file libraries. * Adding a library with a lot of its own transitive depedencies takes proportionally the amount of effort as the number of those dependencies, rather than being handled automatically. One interesting thing about all of these problems is that they get worse and worse as you need more libraries in your program, or need a larger library for your program. In contrast, using a package manager (I'm thinking particularly vcpkg here) tends to add a one-time cost at the start but allows you to scale your dependency list almost for free. If you're writing your own library, rather than a application, then there are even more problems with this approach, but I won't quite open that can of worms.

mmm_grayons 6y ago

You know what's funny? I used to think this too, but I just started a new project with vcpkg and it works great. I understand that Conan is supposed to be good as well, though slightly less user-friendly. I can just define all of my dependencies in a json manifest, set my cmake toolchain, and run `find_package`, `include_directories`, and `target_link_libraries` as normal. It's great.

fnord123 6y ago

You are not the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform because the single-file lib trend is indeed a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform.

junon 6y ago

This is a bit of a misunderstanding of the C++/C way of doing things. It makes sense in some ways but clearly there are better ways of doing it. Which is precisely why C++20 has modules support. Further, C++ does have a central package manager - whatever the system distributes. C/C++ programs constantly link against the operating system and thus have to get those system headers from somewhere. This means a universal package manager (like Conan) requires buy-in from OS vendors or at least someone who can manage those packages with conviction, else the package manager doesn't make a lot of sense to use. This is contrast to Node, Ruby and Python as they are inherently Cross-Platform in nature and are not coupled tightly to the OS. This is why C/C++, historically, have not had a strong centralized package manager. Anymore, I see less and less package management being used anyway. Most C/C++ projects I work with either vendor in dependencies or use git submodules (the latter I prefer very much). This is because, unlike e.g. the Node community, micro-dependencies are generally not worth the effort to bring in. The usual exception to this sort of structure are codebases that will be distributed via a package manager - in which case, they generally rely on the package manager supplying correct versions of the dependencies. These header-only libs work OK usually because they are able to be included with trivial linkage (something else you normally don't need to care about in scripting environments). Linkage is something you have to care about regardless of if you're writing C, C++ or machine code and since these languages give you pretty much free reign on the flexibility of all of these parameters, it's hard to generalize everything into a nice package like scripting languages can. Usually the functions inside header only libraries will be static inline, and will almost certainly increase compilation time (noticeably so if you use the header in many places). There is work being done to improve this situation, it's not like we're all just sitting here going "yes, we love the way things are and have no idea how to make things better." That's far from the case. Shit just takes time.

jasode 6y ago

>Am I the only one to see this single-file lib trend as a consequence of the failure of the C++ ecosystem to produce a modern and unified package management platform? One counterpoint to your proposed cause & effect is that some observers think Javascript NPM's packaging convenience enables the explosion of single file dependencies. Example discussion: https://news.ycombinator.com/item?id=11348798 https://news.ycombinator.com/item?id=11348798

klodolph 6y ago
What’s happening in the JavaScript ecosystem is dependencies are so easy that you’re getting a lot of small libraries, and they’re one file because they’re small. What’s happening in C++ is that you’re getting one-file libraries that are not small, because developers will shoehorn their library into a single file rather rather than deal with a way of managing C++ library dependencies. You might find in C++, for example, a header file with over 10k lines in it and a bunch of preprocessor conditionals. What you might find in JavaScript is a library that contains one or two functions.

pjmlp 6y ago

I see it more as trend for the You Tube generation unwilling to learn how compiled languages work. Maybe we should a couple of You Tube videos about linkers and stuff.

snazz 6y ago
This is just a natural consequence of programming becoming more accessible, I think. And let's not pretend that C and C++ package management is "good but misunderstood"—it needs a lot of work.
- pjmlp 6y ago
  Yeah, work like reading a book.
  - snazz 6y ago
    You can understand how C/C++ builds and dependencies work by reading a book, yes. But you can still criticize it—there's lots of stuff there to criticize! Modern languages have shown that dependency management and builds can be easier, faster, more secure, and more portable.
    - pjmlp 6y ago
      Which is why Conan and vcpkg do exist, and for quite long time OS package managers and installers.
raphaelj 6y ago
This does not have anything to do with knowledge. C/C++ build systems are archaic if not broken. I literally wrote a compiler but still spend 15/20% of my development time trying to find the right not user-friendly CMake syntax, understanding while my Conan dependency broke overnight, or writing header files that could be automatically generated by the compiler (à la GHC).
- 6y ago