3 ms·

Worth pointing out that all third party deps at Google have to be vendored. By policy. And generally only one version of said dep is permitted for the entire ma

Worth pointing out that all third party deps at Google have to be vendored. By policy. And generally only one version of said dep is permitted for the entire massive monorepo. (This may have changed since I was there.) And each third_party dep has a set of designated responsible individuals / OWNERS. This has the effect of keeping things relatively sane. But this enforced discipline can work well at an org like Google that has plenty of time and $$ and isn't dedicated to a "move fast and break things" philosophy. Not sure how well it would work for a startup.

baq 3y ago
I’m completely guessing here but this looks like one relatively technical reason Google keeps killing products.
- cmrdporcupine 3y ago
  That guess is not accurate. There are plenty of other reasons though.
bee_rider 3y ago
This seems like a unique solution for a Google-sized programming enterprise, right? Google is big enough and has enough programmers that they can have their own internal ecosystem that looks a lot like the whole open source universe (other than not being open to the outside world).
- cmrdporcupine 3y ago
  I mean, they're all open source dependencies. Just moderated.
guitarbill 3y ago
You don't discuss why they do it or the downsides. "Google does this" is not good enough of an argument. For the why, could be that the build would be dog slow if a network requests was made for every one of a bajillion dependencies. Or to avoid breaking the build if something is janked externally. Could be because C++ has no modules/libraries. Could be for the lawyer's sake. Is any of that applicable? There are also other ways of vendoring Rust crates, e.g. mirroring them to an internal registry. That can have fewer downsides.
- cmrdporcupine 3y ago
  Security is one big reason. Avoiding link time or runtime conflicts is another. And, yes, having outgoing network calls for builds is a problem. Not because of performance concerns, but because it's a security and reliability nightmare. But consistency across the whole codebase is drastically important. There are many many reasons but overall sanity and security concerns are paramount. But also, read the original article referred to here... and the pain points seen there. Be careful about what you depend on.
  - zozbot234 3y ago
    > Security is one big reason. Avoiding link time or runtime conflicts is another. I mean, most Linux distributions do the same thing - anything that's "vendored" as part of some project is supposed to be patched out from the source and pointed to the single system-wide version of that dependency. And Debian is packaging lots of Rust crates and Golang modules as build-time dependencies within their archive.
- SAI_Peregrinus 3y ago
  Rust has the `cargo vendor` command to vendor all crates.io and git dependencies for a project.
oneshtein 3y ago
Looks a lot like a Linux distribution, for example, Fedora, which has the same policy to reduce maintenance burden.
- ogoffart 3y ago
  The policy in Fedora is precisely that upstream must not vendor (bundle) their dependencies https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling https://docs.fedoraproject.org/en-US/packaging-guidelines/#b... https://docs.fedoraproject.org/en-US/packaging-guidelines/Rust/#_vendored_dependencies https://docs.fedoraproject.org/en-US/packaging-guidelines/Ru...
  - oneshtein 3y ago
    Yes, exactly. All dependencies must be bundled by Fedora, as separate packages, with only 1 version per dependency allowed,

baq 3y ago

I’m completely guessing here but this looks like one relatively technical reason Google keeps killing products.

cmrdporcupine 3y ago
That guess is not accurate. There are plenty of other reasons though.

bee_rider 3y ago

This seems like a unique solution for a Google-sized programming enterprise, right? Google is big enough and has enough programmers that they can have their own internal ecosystem that looks a lot like the whole open source universe (other than not being open to the outside world).

cmrdporcupine 3y ago
I mean, they're all open source dependencies. Just moderated.

guitarbill 3y ago

You don't discuss why they do it or the downsides. "Google does this" is not good enough of an argument. For the why, could be that the build would be dog slow if a network requests was made for every one of a bajillion dependencies. Or to avoid breaking the build if something is janked externally. Could be because C++ has no modules/libraries. Could be for the lawyer's sake. Is any of that applicable? There are also other ways of vendoring Rust crates, e.g. mirroring them to an internal registry. That can have fewer downsides.

cmrdporcupine 3y ago
Security is one big reason. Avoiding link time or runtime conflicts is another. And, yes, having outgoing network calls for builds is a problem. Not because of performance concerns, but because it's a security and reliability nightmare. But consistency across the whole codebase is drastically important. There are many many reasons but overall sanity and security concerns are paramount. But also, read the original article referred to here... and the pain points seen there. Be careful about what you depend on.
- zozbot234 3y ago
  > Security is one big reason. Avoiding link time or runtime conflicts is another. I mean, most Linux distributions do the same thing - anything that's "vendored" as part of some project is supposed to be patched out from the source and pointed to the single system-wide version of that dependency. And Debian is packaging lots of Rust crates and Golang modules as build-time dependencies within their archive.
SAI_Peregrinus 3y ago
Rust has the `cargo vendor` command to vendor all crates.io and git dependencies for a project.

oneshtein 3y ago

Looks a lot like a Linux distribution, for example, Fedora, which has the same policy to reduce maintenance burden.

ogoffart 3y ago
The policy in Fedora is precisely that upstream must not vendor (bundle) their dependencies https://docs.fedoraproject.org/en-US/packaging-guidelines/#bundling https://docs.fedoraproject.org/en-US/packaging-guidelines/#b... https://docs.fedoraproject.org/en-US/packaging-guidelines/Rust/#_vendored_dependencies https://docs.fedoraproject.org/en-US/packaging-guidelines/Ru...
- oneshtein 3y ago
  Yes, exactly. All dependencies must be bundled by Fedora, as separate packages, with only 1 version per dependency allowed,