3 ms·
Vendor by Default (2021)
- pornel 4y agoReading your dependencies' sources is a very good idea. However, vendoring is a clunky way to do it. On top of the baggage it adds to the repository, it also makes you re-review the same dependencies in multiple projects. Reviewing code is hard and tedious, and big dumps will get a careless "LGTM". I prefer the system of sharing reviews: https://lib.rs/cargo-crev https://lib.rs/cargo-crev. I review code of individual dependencies, publish "I've checked the dependency with this hash is fine" in a machine-readable form, and I and people who trust me can reuse this verification across projects.
- cxr 4y ago> On top of the baggage it adds to the repository, it also makes you re-review the same dependencies in multiple projects. Huh? Why would that be the case? (You're saying storing the code of your dependencies directly in your source tree would require you to reread the code for every project where that dependency is used? Or something else?) > big dumps will get a careless "LGTM" Your "will" means "probably will" (i.e., "can"), but when package registries are involved for late-fetching dependencies, that's what goes on, too, every time. You say this, though, as if it's a unique downside to in-tree dependencies.
- pornel 4y agoUnless you have a company-wide monorepo, you'll end up needing to vendor the same dependencies on separate occasions in multiple projects. If you use vendoring as an opportunity to review code of dependencies you're adding, you end up with multiple PRs that include the same code you've already reviewed. In some cases you may remember you've already reviewed code of a certain dependency and can skip it next time, but human memory is imprecise (have you reviewed this exact version or an older one?), and harder to share between team members to avoid duplicate effort across the team (if someone else gets a vendoring PR, how will they know what you've already reviewed before?) Tendency to "LGTM" grows with code size. If you get a massive PR that vendors fifty deps at once, it will be an intimidating wall of code, under pressure to quickly accept the PR to unblock development. A tool that shares and "recycles" prior reviews may reduce amount of code you review. It lets you review one package at a time, and split that work among team members. Depending on your threat model you may also review deps asynchronously, not blocking PRs.
- cxr 4y agoTo reiterate: "LGTM" is what happens right now with most out-of-tree dependencies listed in package.json and fetched JIT. To put it another way: keeping your dependencies in-tree or not doesn't directly impact code size. All the code is there in either case, whether you're checking it out as a consequence of cloning the repo or whether you're pulling it down via npm install. You're either already reviewing it, or you're not. On that specific factor you brought up (tendency to "LGTM"), there is no unique downside to storing the dependencies in-tree. It's illusory thinking to believe that there is.
- deleted 4y ago[deleted]
- daviddever23box 4y agoAs an aside, it is still possible to vendor dependencies in Golang by using go mod vendor, as described here: https://go.dev/ref/mod#go-mod-vendor https://go.dev/ref/mod#go-mod-vendor
- echelon 4y agoThe last place I worked had an enormous Golang monorepo where every dependency was vendored. Literally everything. It was a very large collection of code. Updating dependencies frequently touched a ton of files. One approach to being unable to update pinned dependencies was to duplicate code, which created even more of it. On the plus side, the tooling still seemed to handle the scale. Everything was there for you to look at.
- Cthulhu_ 4y agoThis is why it's generally best to avoid dependencies as much as possible; "a little copying is better than a little dependency". That said, updating dependencies / many code changes because of it isn't necessarily a bad thing. It may need to be restricted to once a month though, unless there's a security incident. But honestly, in Go, dependencies don't change that often so you don't need to update so frequently.
- psnehanshu 4y agoI didn't read through all the article, but I am not convinced. Vendoring isn't scalable for the long term. Everytime you want to use a dependency, you will have the overhead of first understanding the code and then copy-paste. The dividends might be there, but not worth it.
- xyzzy123 4y agoI don't think it's kind to maintainers. IMHO vendoring all the things works for very small (personal) projects or for very large projects (you are google), but not for most of the stuff in between. It's great to be the sort of developer who reads all their dependencies, what's not nice is forcing maintainers to do that too after you have left - we usually want to expend the minimum effort necessary to keep your application vulnerability-free and still-deployable and do not appreciate "artisanal" takes. I want to see locked dependency versions and use of company artefact repositories to avoid unexpected changes and avoid availability concerns (e.g. it gets yanked off npm or github etc). I am not pleased to find vendoring, as I now have to work out what (if any) code has been changed, what version of the upstream it corresponds to, and I now have to potentially merge third party code to do otherwise simple updates. How bad this all is depends on the language and style of vendoring. The absolute worst case is when someone like the author has vendored everything, deleted half of it, pointlessly eslinted it(!) made a few cosmetic changes here and there, ... "haha good luck merging from upstream now" - it adds no business value but increases maintenance costs. Depending on tooling, I may now also have to commit to periodic manual review for dep based CVEs instead of just letting the company dep scanner sort those out. I have no problems with vendoring "for the right reasons" e.g. if you actually needed to change the code to fix a bug or make a change that's fairly specific to your application.
- jeroenhd 4y agoI think that this approach would cut down the amount of JS dependencies significantly. Things like is-even and is-odd come to mind. You don't want another leftpad or colors.js to happen to you and minimising dependencies is the most effective strategy to accomplish that. So many leaf dependencies I've looked into are no more than a Stackoverflow answer in a JS file accompanied by six or seven metadata files (package.json + typescript files + linter config + readme + git config + ...). This file: https://github.com/i-voted-for-trump/is-odd/blob/master/index.js https://github.com/i-voted-for-trump/is-odd/blob/master/inde... is downloaded over 400000 times per week (https://www.npmjs.com/package/is-odd https://www.npmjs.com/package/is-odd) and while I don't have anythimg against the author for publishing a helpers function, I don't see why I would expose my project to risking a supply chain attack for something so minor. Here's another, with millions of downloads: https://github.com/inspect-js/is-date-object/blob/main/index.js https://github.com/inspect-js/is-date-object/blob/main/index... I know that these are all downloaded so ofyen because theyre dependencies of dependencies but I'd appreciate it if bigger libraries would provide a vendored version of their packages that just collects these microdependencies instead of wasting npm's time by making it manage these tiny helper files. Don't vendor stuff like React or Vue or whatever framework you prefer but for the love of God don't add a dependency for 50 lines of code. Sometimes copy/paste is the right solution.
- QuadmasterXLII 4y agoTo be fair, is-odd is a bit of a weird case because schlinkert has a whole web of useless packages that depend on each other and promotes them by shoving them into real open source projects. Bad-faith clout hacking is a real problem in the js ecosystem but not the same problem as excessive good faith dependencies
- retcore 4y agoI thought that this was actually going to be a argument against vacuuming non commercial libraries and arbitrary codes, and a appeal, which I believe is a important consideration, for paid contract library maintenance classification, interop/bakeoff and testing. Clearly the possibility has slipped through the gaps between the plethora of open source foundations already established.
- sourceless 4y agoAnd you get to inherit all the vulnerabilities while sacrificing the ability to automatically/easily upgrade to a fixed version...