4 ms·

Dependencies introduce risks, but not using them at all puts you at a competitive disadvantage against those who are using them and thus achieve faster developm

by fmajid 1y ago

Dependencies introduce risks, but not using them at all puts you at a competitive disadvantage against those who are using them and thus achieve faster development time and time to market. What you need is a process to manage dependencies: 1) Only consider open-source dependencies. 2) Introducing new dependencies requires a review. Not just a code review on the pull request introducing it, but checking the license, estimating how much work it would be to rip out, auditing it for history of security vulnerabilities or bugs, whether it is live and still gets updates, how vibrant the community is, and so on. 3) If possible, review your dependencies for possible security issues. Doing this at scale is expensive and the economics of this are still an unsolved problem (I have my ideas: https://blog.majid.info/supply-chain-vetting/ https://blog.majid.info/supply-chain-vetting/). 4) Do not adopt a dependency you are not willing and able to take over the maintenance of, or fork if necessary. At a minimum, it means you have built it from source at least once, not just used binary packages maintained by someone else. 5) Preemptively fork all your dependencies. People can and do delete their repos out of pique, see left-pad.

dotancohen 1y ago
This is amazingly insightful, thank you. I'm going to copy this to my own procedures files, on every dependency I use in the future. The only problem I really see is very tall dependency trees, seen in e.g. the JavaScript world.
chii 1y ago
Both 4) and 5) are very important, but often forgotten. Even for my own personal (small) projects, i've gotten hit with problems when i take an extended leave of absence and then return to a project, only to find my dependencies have become either completely outdated and unusable, or the repo was entirely deleted (with only a remote copy to build with). I've since adopted the "fork" method; the dependency's source is forked (privately), and get the dependency built; this is recursively done with their dependencies (stopping at the language level libraries), just to get familiar with their build chain. Only then, will i feel good enough to add the dependency to my project (and use their publicly released artefacts from their publicly hosted library repository). It's a bit of work, but i think this effort is spent upfront, and removes future effort if the project lives long enough to see the ecosystem move/change directions (and you dont want to follow). Sometimes i do find that source libraries (rather than pre-built binaries) to be better in this sense. I merely need to fork and link the sources into the project, rather than have to learn anything about build chains of dependencies...
Cthulhu_ 1y ago
With 5, forking may be a bit excessive, as keeping your fork up to date adds a big maintenance burden. Vendoring dependencies (just pushing them into git, git LFS, or operating a caching proxy) is valid though, especially for long-lived software projects. Maybe less so for NodeJS based ones as their dependencies use many small files, but there's Yarn and maybe PNPM that will keep dependencies in more convenient archive files instead.
friendzis 1y ago
4 and 5. Not at least once, all of your code should be at the very least buildable with a network cable yoinked out, preferably without any binary artifacts, but that is not always possible.
hahn-kev 1y ago
For #4, while I agree sometimes, I also recognize that SQLite (and other similar types of dependencies) are 100% going to outlive the product I'm building. So it's a bit arrogant to say my product is going to outlive them. I'm not going to build the Linux kernel just because it's a dependency.
- nijave 1y ago
  I think it still applies if consider you can build SQLite from a fork of the source and it probably needs no maintenance. In some ways, you could consider the SQLite package that ships with a given distro/OS a mini-fork Still, surprises happen. Key contributor(s) could become ill, license changes can happen, ownership/stewardship change. Although super popular projects you can _probably_ rely on someone else forking Iirc one of the MariaDB engines was maintained by some singular dude that would go offline for months at a time and maintained it as a hobby.

by fmajid 1y ago

dotancohen 1y ago

This is amazingly insightful, thank you. I'm going to copy this to my own procedures files, on every dependency I use in the future. The only problem I really see is very tall dependency trees, seen in e.g. the JavaScript world.

chii 1y ago

Both 4) and 5) are very important, but often forgotten. Even for my own personal (small) projects, i've gotten hit with problems when i take an extended leave of absence and then return to a project, only to find my dependencies have become either completely outdated and unusable, or the repo was entirely deleted (with only a remote copy to build with). I've since adopted the "fork" method; the dependency's source is forked (privately), and get the dependency built; this is recursively done with their dependencies (stopping at the language level libraries), just to get familiar with their build chain. Only then, will i feel good enough to add the dependency to my project (and use their publicly released artefacts from their publicly hosted library repository). It's a bit of work, but i think this effort is spent upfront, and removes future effort if the project lives long enough to see the ecosystem move/change directions (and you dont want to follow). Sometimes i do find that source libraries (rather than pre-built binaries) to be better in this sense. I merely need to fork and link the sources into the project, rather than have to learn anything about build chains of dependencies...

Cthulhu_ 1y ago

With 5, forking may be a bit excessive, as keeping your fork up to date adds a big maintenance burden. Vendoring dependencies (just pushing them into git, git LFS, or operating a caching proxy) is valid though, especially for long-lived software projects. Maybe less so for NodeJS based ones as their dependencies use many small files, but there's Yarn and maybe PNPM that will keep dependencies in more convenient archive files instead.

friendzis 1y ago

4 and 5. Not at least once, all of your code should be at the very least buildable with a network cable yoinked out, preferably without any binary artifacts, but that is not always possible.

hahn-kev 1y ago

For #4, while I agree sometimes, I also recognize that SQLite (and other similar types of dependencies) are 100% going to outlive the product I'm building. So it's a bit arrogant to say my product is going to outlive them. I'm not going to build the Linux kernel just because it's a dependency.

nijave 1y ago
I think it still applies if consider you can build SQLite from a fork of the source and it probably needs no maintenance. In some ways, you could consider the SQLite package that ships with a given distro/OS a mini-fork Still, surprises happen. Key contributor(s) could become ill, license changes can happen, ownership/stewardship change. Although super popular projects you can _probably_ rely on someone else forking Iirc one of the MariaDB engines was maintained by some singular dude that would go offline for months at a time and maintained it as a hobby.