3 ms·
Well, whether this is a "win" depends too much on the definition of "win" for me to care to debate. But there are certainly advantages; a vulnerable code path t
by jerf 3y ago
Well, whether this is a "win" depends too much on the definition of "win" for me to care to debate. But there are certainly advantages; a vulnerable code path that is never executed and can not be reached from the external library is now a safe code path. A nerve wracking one to be sure, but safe.
Another advantage of vendoring like that is that you do have some tools for attacking the code now. If you have solid test coverage on your own library, you can now run your library and run code coverage tools on the newly-vendored library. Fixing the library may be a challenge, but you might be able to just go hacking into it and deleting anything your code doesn't touch relatively easily. Depends on the structure of the vendored library, of course. If you end up deleting all the vulnerable code, you do have a clear win. If you discover you were hitting some of it after all you may end up with an even more clear win.
Or, to put it another way, forking a library in general for public maintenance is a big commitment. Vendoring it and hacking it to just the bits you need is much smaller. Even if you don't actually perform this pruning, just the act of making that pruning easier is itself progress.
There are disadvantages too, certainly, but it's not all bad.
- leoedin 3y agoThis is actually a great outcome. All dependencies are a security risk - the argument for not vendoring every dependency is that you should keep external dependencies up to date to avoid known security issues. If the external dependency has nobody watching it any more then that argument disappears, and that external dependency becomes a liability. The fact the dependency was marked as abandoned and started getting flagged in security reports is what you want. That lets you make an informed decision about whether to vendor it (which removes the "bad actor sneaks something in" risk) or do something else. It's great that the Rust build system allows that so easily.
- the_mitsuhiko 3y agoAuthor here: the result is now that the debt got absorbed into a project that uses the fork exclusively. It’s less likely that folks will look for security issues. It would in my mind have been better for folks to use an abandoned shared library where at the very least if a security incident popped up we would have all been notified and dealt with it at once.
- jerf 3y agoThe debt has been absorbed and I don't deny that, but it has at least been absorbed in a way that makes paying it locally easier. Given that the description of the problem includes generally not being affected by its issue I read that to be the rather common case in which one is just using a relatively small fraction of the library. To be concrete, let's say it's an image library, and it turns out there's a PNG parsing vulnerability, but your code only ever uses it for GIFs. If you vendor it you also have the opportunity to go in and just yank PNG, JPEG, TIFF, etc. support, and depending on the situation, plugin support (maybe you can trivially tweak your code to go directly to the JPEG support without passing through anything else). I'm not saying this is a perfect solution with no downsides, but it's not all downside either. Of course, you need the aforementioned solid test coverage. Without that you're just flying blind. That's another thing about the debt metaphor... it can compound, but unlike most monetary debt, code debt can compound on you unexpectedly.
- actionfromafar 3y agoEven if it's maintained, you don't know how good it is security wise.
- keybored 3y agoVendoring a pinned dependency just looks like dependency theater. You can let it sit there as a pinned dependency and it will do just as much harm as if you vendor it. I get the sense (only a sense because the author is a bit coy, but sibling comment by him seems to confirm) that this was more to appease these tools that look for “bad dependencies”.
- cmrdporcupine 3y agoWorth pointing out that all third party deps at Google have to be vendored. By policy. And generally only one version of said dep is permitted for the entire massive monorepo. (This may have changed since I was there.) And each third_party dep has a set of designated responsible individuals / OWNERS. This has the effect of keeping things relatively sane. But this enforced discipline can work well at an org like Google that has plenty of time and $$ and isn't dedicated to a "move fast and break things" philosophy. Not sure how well it would work for a startup.
- baq 3y agoI’m completely guessing here but this looks like one relatively technical reason Google keeps killing products.
- cmrdporcupine 3y agoThat guess is not accurate. There are plenty of other reasons though.
- bee_rider 3y agoThis seems like a unique solution for a Google-sized programming enterprise, right? Google is big enough and has enough programmers that they can have their own internal ecosystem that looks a lot like the whole open source universe (other than not being open to the outside world).
- cmrdporcupine 3y agoI mean, they're all open source dependencies. Just moderated.
- guitarbill 3y agoYou don't discuss why they do it or the downsides. "Google does this" is not good enough of an argument. For the why, could be that the build would be dog slow if a network requests was made for every one of a bajillion dependencies. Or to avoid breaking the build if something is janked externally. Could be because C++ has no modules/libraries. Could be for the lawyer's sake. Is any of that applicable? There are also other ways of vendoring Rust crates, e.g. mirroring them to an internal registry. That can have fewer downsides.
- theptip 3y agoI wonder if there are tools that library maintainers can use to calculate transitive code coverage of their libraries, eg which other lib test suites use them? Could be an interesting test strategy, in some sense it’s directed fuzz testing. (With other libraries providing the random-ish invocations.)