3 ms·

I've been reading a lot of research papers lately, sometimes with POC repos. When compiling the code, it would often fail because dependencies have changed over

I've been reading a lot of research papers lately, sometimes with POC repos. When compiling the code, it would often fail because dependencies have changed over the years (we're talking about C/C++ code which doesn't have a package manager like you're probably used to). Most of these repos would fail to compile because it expected libraries installed on the system to not have changed in the past 10 years or so. In exactly one of these repos did the author use a git submodule so I was linking directly to the correct version of the library. Granted, those libraries _did not_ do this, so it still failed to compile 7 years later... If everyone used git submodules for deps, you'd end up with a (block) chain (if everyone's commits were signed) for deps.

FrenchyJiby 4y ago
That's pretty cool reproducibility indeed, love it. I once worked in C++, and the lack of a (standard?) package manager was killing me for pretty much that reason. "This says it's using curl, but did they compile it with a weird flag, or was their version just old?", and stuff like that. For the case you describe, a submodule is the next best thing to proper package management, and I'm glad those submodules are there to save you! But in themselves, the submodules don't guarantee build instructions, and the same guarantees of exactness can be had by locking dependencies. I feel like your message mostly reinforces my belief in the NEED for package manager for every such language combo, having seen the immense value in other systems, and the extremes that we have to go to when they're missing. I hear the Nix (and Guix) community has done great steps to manage these stray systems into reproducibility, using submodule chains of the sort you describe, but augmented with standardized build info to make it system-wide pkg management.
- withinboredom 4y ago
  That’s a good point about build instructions, but hopefully they are encoded via a make file. In reality, I only had to make very minor changes to the experimental code to get it to compile, so I really appreciated maintainers keeping backwards compatibility for the most part. Some of this code was over 10 years old, and it still worked… that was pretty impressive since I mostly work with higher level languages where dependencies can change very significantly between versions, or even be abandoned within a decade.
humanrebar 4y ago
> we're talking about C/C++ code which doesn't have a package manager like you're probably used to No, there are 3-4 suitable package managers [1] depending on what your bar for an acceptable package manager is. It's just C and C++ engineers have gone so long without one that they don't even think about it. Honestly, we just need to start expecting more (and contributing more) with respect to unpackaged C and C++. [1] vcpkg, Conan, nix, and spack off the top of my head

FrenchyJiby 4y ago

That's pretty cool reproducibility indeed, love it. I once worked in C++, and the lack of a (standard?) package manager was killing me for pretty much that reason. "This says it's using curl, but did they compile it with a weird flag, or was their version just old?", and stuff like that. For the case you describe, a submodule is the next best thing to proper package management, and I'm glad those submodules are there to save you! But in themselves, the submodules don't guarantee build instructions, and the same guarantees of exactness can be had by locking dependencies. I feel like your message mostly reinforces my belief in the NEED for package manager for every such language combo, having seen the immense value in other systems, and the extremes that we have to go to when they're missing. I hear the Nix (and Guix) community has done great steps to manage these stray systems into reproducibility, using submodule chains of the sort you describe, but augmented with standardized build info to make it system-wide pkg management.

withinboredom 4y ago
That’s a good point about build instructions, but hopefully they are encoded via a make file. In reality, I only had to make very minor changes to the experimental code to get it to compile, so I really appreciated maintainers keeping backwards compatibility for the most part. Some of this code was over 10 years old, and it still worked… that was pretty impressive since I mostly work with higher level languages where dependencies can change very significantly between versions, or even be abandoned within a decade.

humanrebar 4y ago

> we're talking about C/C++ code which doesn't have a package manager like you're probably used to No, there are 3-4 suitable package managers [1] depending on what your bar for an acceptable package manager is. It's just C and C++ engineers have gone so long without one that they don't even think about it. Honestly, we just need to start expecting more (and contributing more) with respect to unpackaged C and C++. [1] vcpkg, Conan, nix, and spack off the top of my head