19 ms·

> Google can't spin off tools successfully because their internal codebases are deeply reliant on assumptions about Google's infrastructure and Google internal

> Google can't spin off tools successfully because their internal codebases are deeply reliant on assumptions about Google's infrastructure and Google internal libraries, etc. (for example: Google Reader could not be spun off This is a not a likely speculation. Did reader really require anything special that doesn’t already exist as a Google Cloud API? Or majority of their killed products honestly? Or to look another way, do you think anyone who has a cloud services platform (Microsoft, Amazon, Google) can really afford having a separate, divergent infrastructure internally? Haven’t these services started for utilizing excess capacity and economies of scale of existing infrastructure? Also you are idealizing code reuse, especially at scale. When you take a dependency the arrow goes both ways and now there is a cost for depended breaking things too. Each integration potentially creates non-trivial friction points, if not we would have open source code of the world converge towards a single codebase organically already.

andrewem 6y ago
It's probably true that now (in 2020) Google has cloud services that do everything which Reader needed, but it wasn't true when Reader was shut down in 2013. For instance, cloud Bigtable launched to the public in 2015. (I don't know whether current Google products mainly rely on the public services, or internal near-equivalents to the public services, or significantly different internal services. I assume it's mainly either the first or the second.)
- acituan 6y ago
  Well the point is there is no “secret sauce infra”, let alone Google specific infra as the OP speculates. If we were to claim timing as a factor, AWS was available since 2006.
  - lrem 6y ago
    You really missed the point. It does not matter if the "secret sauce infra" is better than equivalents. What matters is that the code you have is based on it. And since literally everything is based on Stubby, which has a hard dependency on Borg, once you remove the bits of the backend's code that are secret-sauce-dependent you're left with a Makefile ;) Disclaimer: I work in Google, but the above is easily inferred from the SRE books.
    - acituan 6y ago
      > It does not matter if the "secret sauce infra" is better than equivalents. That is what the OP suggested. > What matters is that the code you have is based on it It is the nature of infrastructure is to be independent from business logic. I’m not saying any migration would be trivial, but it is also not as cost prohibitive as you seem to be arguing for. RPC is a very old and generic concept, so unless reader was depending on Google specific RPC mechanics, the cost of swapping one RPC tech with another is not going to be to the extent of warranting uttering of concepts like anti-trust.
      - dodobirdlord 6y ago
        The distinction is less important at Google, where all of the code is in a monorepo and all of the infrastructure was custom-built to support the products.
        acituan 6y ago
        Monorepo doesn’t mean a monolith, it is about code repositories, not about the extent of software reuse. You could convert the entirety of open source code into a monorepo too, but it wouldn’t mean they magically reuse each other. > all of the infrastructure was custom-built to support the products. That tells me you might not have experienced software development in a large company. I don’t know the specifics of Google, but custom tailoring infra is a combinatorially explosively problem if you have hundreds/thousands of products you need to “custom tailor” to. And if you could, would the result be really custom tailored anymore or general enough to have universal use? Your argument is self contradictory.
dodobirdlord 6y ago
> This is a not a likely speculation. Did reader really require anything special that doesn’t already exist as a Google Cloud API? Or majority of their killed products honestly? Yes, absolutely. The Google software stack is custom end-to-end, down to a customized Linux kernel and a customized C++ compiler. If you stripped out all of the Google-specific code of something like Reader there wouldn't be anything left at the end. > Or to look another way, do you think anyone who has a cloud services platform (Microsoft, Amazon, Google) can really afford having a separate, divergent infrastructure internally? All three of the companies you mentioned had an internal platform before they had a cloud offering, and all three still have an internal platform that is widely used. > Haven’t these services started for utilizing excess capacity and economies of scale of existing infrastructure? No, the "excess capacity" part is a myth. It's a pervasive myth, especially about AWS, but it's not actually true.
- acituan 6y ago
  > The Google software stack is custom end-to-end, down to a customized Linux kernel and a customized C++ compiler. Are you really suggesting that the existence of an RSS aggregator hinges on custom linux kernels and C++ compilers? > all three still have an internal platform that is widely used. And your point is? None can afford running two divergent platforms simultaneously. > No, the "excess capacity" part is a myth. It's a pervasive myth, especially about AWS, but it's not actually true. Citation needed. (Also I didn’t say only excess capacity, also economies of scale.)
  - joshuamorton 6y ago
    Pick a piece of infrastructure: server (k8s), container (docker), RPC protocol (GRPC), user auth/ACLs, etc. and now understand that the internal version is different, and you need to replace it. At that point, you're rewriting the entire system, except for some of the business logic (and even a lot of the business logic will need to be rewritten). > And your point is? None can afford running two divergent platforms simultaneously. I think what they're saying is that all three do run divergent platforms simultaneously, today, right now. So clearly they can afford it.

andrewem 6y ago

It's probably true that now (in 2020) Google has cloud services that do everything which Reader needed, but it wasn't true when Reader was shut down in 2013. For instance, cloud Bigtable launched to the public in 2015. (I don't know whether current Google products mainly rely on the public services, or internal near-equivalents to the public services, or significantly different internal services. I assume it's mainly either the first or the second.)

acituan 6y ago
Well the point is there is no “secret sauce infra”, let alone Google specific infra as the OP speculates. If we were to claim timing as a factor, AWS was available since 2006.
- lrem 6y ago
  You really missed the point. It does not matter if the "secret sauce infra" is better than equivalents. What matters is that the code you have is based on it. And since literally everything is based on Stubby, which has a hard dependency on Borg, once you remove the bits of the backend's code that are secret-sauce-dependent you're left with a Makefile ;) Disclaimer: I work in Google, but the above is easily inferred from the SRE books.
  - acituan 6y ago
    > It does not matter if the "secret sauce infra" is better than equivalents. That is what the OP suggested. > What matters is that the code you have is based on it It is the nature of infrastructure is to be independent from business logic. I’m not saying any migration would be trivial, but it is also not as cost prohibitive as you seem to be arguing for. RPC is a very old and generic concept, so unless reader was depending on Google specific RPC mechanics, the cost of swapping one RPC tech with another is not going to be to the extent of warranting uttering of concepts like anti-trust.
    - dodobirdlord 6y ago
      The distinction is less important at Google, where all of the code is in a monorepo and all of the infrastructure was custom-built to support the products.
      - acituan 6y ago
        Monorepo doesn’t mean a monolith, it is about code repositories, not about the extent of software reuse. You could convert the entirety of open source code into a monorepo too, but it wouldn’t mean they magically reuse each other. > all of the infrastructure was custom-built to support the products. That tells me you might not have experienced software development in a large company. I don’t know the specifics of Google, but custom tailoring infra is a combinatorially explosively problem if you have hundreds/thousands of products you need to “custom tailor” to. And if you could, would the result be really custom tailored anymore or general enough to have universal use? Your argument is self contradictory.

dodobirdlord 6y ago

> This is a not a likely speculation. Did reader really require anything special that doesn’t already exist as a Google Cloud API? Or majority of their killed products honestly? Yes, absolutely. The Google software stack is custom end-to-end, down to a customized Linux kernel and a customized C++ compiler. If you stripped out all of the Google-specific code of something like Reader there wouldn't be anything left at the end. > Or to look another way, do you think anyone who has a cloud services platform (Microsoft, Amazon, Google) can really afford having a separate, divergent infrastructure internally? All three of the companies you mentioned had an internal platform before they had a cloud offering, and all three still have an internal platform that is widely used. > Haven’t these services started for utilizing excess capacity and economies of scale of existing infrastructure? No, the "excess capacity" part is a myth. It's a pervasive myth, especially about AWS, but it's not actually true.

acituan 6y ago
> The Google software stack is custom end-to-end, down to a customized Linux kernel and a customized C++ compiler. Are you really suggesting that the existence of an RSS aggregator hinges on custom linux kernels and C++ compilers? > all three still have an internal platform that is widely used. And your point is? None can afford running two divergent platforms simultaneously. > No, the "excess capacity" part is a myth. It's a pervasive myth, especially about AWS, but it's not actually true. Citation needed. (Also I didn’t say only excess capacity, also economies of scale.)
- joshuamorton 6y ago
  Pick a piece of infrastructure: server (k8s), container (docker), RPC protocol (GRPC), user auth/ACLs, etc. and now understand that the internal version is different, and you need to replace it. At that point, you're rewriting the entire system, except for some of the business logic (and even a lot of the business logic will need to be rewritten). > And your point is? None can afford running two divergent platforms simultaneously. I think what they're saying is that all three do run divergent platforms simultaneously, today, right now. So clearly they can afford it.