4 ms·

The more interesting story here IMO is the seemingly extreme prevalence of NIH syndrome at Google, and whether or not that was a smart investment for Google com

The more interesting story here IMO is the seemingly extreme prevalence of NIH syndrome at Google, and whether or not that was a smart investment for Google compared to, say, contributing to GitLab or something. I would love to hear the reasoning behind going custom, and if that ended up truly being worth the investment. Or maybe I'm misremembering the timeline on when certain things became available. shrug

arccy 2y ago
based on my experience with gitlab... anything is better than it at scale
- floating-io 2y ago
  That... is probably true. =)
acac10 2y ago
I think the takeaway from your comment is that you really do not understand the sheer scale of volume of changes done by +100k engineers, their heavy reliance on tools that do cluster-based testing (not local dev machine) and the tens of thousands of tools that perform auto-commits. It’s a lot easier to shit on a big corp. Intellectually lazy, too.
- floating-io 2y ago
  Are you saying that GitHub and GitLab.com don't have that kind of volume? (edit: the point being that at least one of those tools existed when they started Piper if I have the timeline correct).
  - Xeamek 2y ago
    >Are you saying that GitHub and GitLab.com don't have that kind of volume? On a single project? Verry much doubt either of them do
    - floating-io 2y ago
      Yeah, that's moving the goalposts. The point is that either product should theoretically have the ability to handle Google's load. They were both built for scale. If I have the timeline correct, they existed at that point. I would expect those organizations to have positively salivated at the possibility of having Google as a customer; they would have bent over backwards to make it support Google's workflows. The ROI of such a choice vs going custom is the interesting question. Of course, if I have the timeline wrong, then the discussion is moot.
      - tylerhou 2y ago
        There is a big difference between one VCS handling 1,000 QPS and 1,000,000 VCS instances each handling 0.001 QPS. A system built for one is not necessarily suitable for the other.
        floating-io 2y ago
        See my response to bananapub below.
      - Xeamek 2y ago
        Its not moving the goalpost, it's part of google's requirements that make their case unique (at least more than average)
      - kccqzy 2y ago
        It's not moving the goalpost because you didn't understand where the goalpost was in the first place. Lots of tiny repos like GitHub? Easy sharding problem. They don't even need Paxos. One large monorepo? Everything changes. Let us not forget that GitHub cannot even handle the Homebrew repo be cloned and updated by Mac users and asked them to switch to a CDN.
bananapub 2y ago
> The more interesting story here IMO is the seemingly extreme prevalence of NIH syndrome at Google, and whether or not that was a smart investment for Google compared to, say, contributing to GitLab or something. this is the perfect HN comment - so confident but with so little thought. essentially all google code - something like a billion LOC - are in one repository. there's something like 100 000 people who use source control, plus jillions of systems doing automatic commits all the time. it's atomic on each commit. so that's several QPS globally synchronising a huge data store, with the entire dev tooling system hanging off it. git is ... very very much not suitable for such systems. gitlab is just a simple web app reading from a bunch of small and rarely changing git repositories. Piper is also the successor to Perforce, which is a completely different model to git.
- floating-io 2y ago
  It's interesting how people treat the monorepo as some kind of religion that MUST be kept in this discussion. How repos are organized is a choice. Google could just as easily (financially speaking) have chosen to break up that repo and use more standard tooling. They did not choose that. Did they have (possibly perfectly valid) reasons for that? Sure. That does not render it as their only possible path. Rather than switch their source control strategy to something that would work well with existing products, they chose to build a brand new product to support their existing strategy. This is, IMO, not an obviously sane decision from an outside perspective. Why? Because source control has nothing to do with their core business, and in addition to the cost of developing and deploying that solution, they now have to bear the cost of maintaining it, which is likely not insignificant. That's how it looks from an outside perspective. The question I'm asking is not "why didn't they use git?!". The question I'm asking is, "what is the ROI of building and maintaining their own solution versus biting the bullet and pivoting so they can use pre-existing products?" Haven't seen an answer to that yet. People are too busy implying or outright stating that I'm stupid for asking the question in the first place.
  - bananapub 2y ago
    > It's interesting how people treat the monorepo as some kind of religion that MUST be kept in this discussion. not at all, it's just what google decided to do on balance. not only did they decide it was the best choice for them, the cost of switching would be ... astronomical. the amount of tooling that assumes it can find all code in the one place is very very large.
kccqzy 2y ago
GitHub uses Git. Git itself is ill suited to Google's size. Years ago Google has tried something called git5 which aimed to bring git to the Google monorepo. It failed miserably because Git itself isn't scalable. Lest you think they didn't try hard enough, Facebook tried the same thing and came to the exact same conclusion that Git isn't scalable. Nowadays both companies use mercurial which is much better. They couldn't even switch to Git let alone a forge based on Git.
- dak3 2y ago
  Mercurial wasn't scalable either but the Mercurial community was much more receptive to changes to improve scalability. So Meta pushed Mercurial as far as possible with a single server solution while developing -> https://github.com/facebook/sapling https://github.com/facebook/sapling.
  - kccqzy 2y ago
    Yes. Mercurial wasn't scalable enough but now has been made scalable enough to be used at Google for individuals.

arccy 2y ago

based on my experience with gitlab... anything is better than it at scale

floating-io 2y ago
That... is probably true. =)

acac10 2y ago

I think the takeaway from your comment is that you really do not understand the sheer scale of volume of changes done by +100k engineers, their heavy reliance on tools that do cluster-based testing (not local dev machine) and the tens of thousands of tools that perform auto-commits. It’s a lot easier to shit on a big corp. Intellectually lazy, too.

floating-io 2y ago
Are you saying that GitHub and GitLab.com don't have that kind of volume? (edit: the point being that at least one of those tools existed when they started Piper if I have the timeline correct).
- Xeamek 2y ago
  >Are you saying that GitHub and GitLab.com don't have that kind of volume? On a single project? Verry much doubt either of them do
  - floating-io 2y ago
    Yeah, that's moving the goalposts. The point is that either product should theoretically have the ability to handle Google's load. They were both built for scale. If I have the timeline correct, they existed at that point. I would expect those organizations to have positively salivated at the possibility of having Google as a customer; they would have bent over backwards to make it support Google's workflows. The ROI of such a choice vs going custom is the interesting question. Of course, if I have the timeline wrong, then the discussion is moot.
    - tylerhou 2y ago
      There is a big difference between one VCS handling 1,000 QPS and 1,000,000 VCS instances each handling 0.001 QPS. A system built for one is not necessarily suitable for the other.
      - floating-io 2y ago
        See my response to bananapub below.
    - Xeamek 2y ago
      Its not moving the goalpost, it's part of google's requirements that make their case unique (at least more than average)
    - kccqzy 2y ago
      It's not moving the goalpost because you didn't understand where the goalpost was in the first place. Lots of tiny repos like GitHub? Easy sharding problem. They don't even need Paxos. One large monorepo? Everything changes. Let us not forget that GitHub cannot even handle the Homebrew repo be cloned and updated by Mac users and asked them to switch to a CDN.

bananapub 2y ago

> The more interesting story here IMO is the seemingly extreme prevalence of NIH syndrome at Google, and whether or not that was a smart investment for Google compared to, say, contributing to GitLab or something. this is the perfect HN comment - so confident but with so little thought. essentially all google code - something like a billion LOC - are in one repository. there's something like 100 000 people who use source control, plus jillions of systems doing automatic commits all the time. it's atomic on each commit. so that's several QPS globally synchronising a huge data store, with the entire dev tooling system hanging off it. git is ... very very much not suitable for such systems. gitlab is just a simple web app reading from a bunch of small and rarely changing git repositories. Piper is also the successor to Perforce, which is a completely different model to git.

floating-io 2y ago
It's interesting how people treat the monorepo as some kind of religion that MUST be kept in this discussion. How repos are organized is a choice. Google could just as easily (financially speaking) have chosen to break up that repo and use more standard tooling. They did not choose that. Did they have (possibly perfectly valid) reasons for that? Sure. That does not render it as their only possible path. Rather than switch their source control strategy to something that would work well with existing products, they chose to build a brand new product to support their existing strategy. This is, IMO, not an obviously sane decision from an outside perspective. Why? Because source control has nothing to do with their core business, and in addition to the cost of developing and deploying that solution, they now have to bear the cost of maintaining it, which is likely not insignificant. That's how it looks from an outside perspective. The question I'm asking is not "why didn't they use git?!". The question I'm asking is, "what is the ROI of building and maintaining their own solution versus biting the bullet and pivoting so they can use pre-existing products?" Haven't seen an answer to that yet. People are too busy implying or outright stating that I'm stupid for asking the question in the first place.
- bananapub 2y ago
  > It's interesting how people treat the monorepo as some kind of religion that MUST be kept in this discussion. not at all, it's just what google decided to do on balance. not only did they decide it was the best choice for them, the cost of switching would be ... astronomical. the amount of tooling that assumes it can find all code in the one place is very very large.

kccqzy 2y ago

GitHub uses Git. Git itself is ill suited to Google's size. Years ago Google has tried something called git5 which aimed to bring git to the Google monorepo. It failed miserably because Git itself isn't scalable. Lest you think they didn't try hard enough, Facebook tried the same thing and came to the exact same conclusion that Git isn't scalable. Nowadays both companies use mercurial which is much better. They couldn't even switch to Git let alone a forge based on Git.

dak3 2y ago
Mercurial wasn't scalable either but the Mercurial community was much more receptive to changes to improve scalability. So Meta pushed Mercurial as far as possible with a single server solution while developing -> https://github.com/facebook/sapling https://github.com/facebook/sapling.
- kccqzy 2y ago
  Yes. Mercurial wasn't scalable enough but now has been made scalable enough to be used at Google for individuals.