4 ms·

This is what continues to shock me about the tech industry. Google started revolutionizing the way data centers work more than 15 years ago, and it's a huge rea

This is what continues to shock me about the tech industry. Google started revolutionizing the way data centers work more than 15 years ago, and it's a huge reason why they are able to execute on ambitious projects so well and serve up content that has a ton of compute behind it yet do so incredibly fast yet also incredibly cheaply. That cuts right to the bottom line, yet nobody else has copied google yet. I think it's a huge indication of just how immature the entire industry truly is. It's difficult enough to get anything done these days, let alone get something exceptional done well. We're only just a little bit out of the deep depths of the "software crisis" where the ability to run a successful software project was a hit or miss thing. But we're still in the software crisis shallows, and it'll be a long time before we make land fall.

threeseed 11y ago
> nobody else has copied google yet Why would they ? Almost nobody is operating at Google's scale and interconnects like Infiniband are rarely saturated to the point that exotic solutions are needed. Fast RPC is nice and all but (a) not that many companies are using microservices and (b) not that many are scaling out to the billions of requests/second. And the 20 or so companies who are operating at Google's scale are hardly floundering around blind. Facebook has done very interesting work in the infrastructure space, LinkedIn is amazing on the core software side and Netflix has pioneered the use of the cloud.
- mikecb 11y ago
  I think this is short-sighted. Everyone who operates with any portion of their stack on any cloud provider is "operating at Google's scale" since even a small service is affected both by the infrastructure on which it is run and all of the other services run on that infrastructure. If you're running in a cloud, you need to plan and develop like you're a product team at facebook or google, and cloud providers need to act like they are providing infrastructure for a cohesive system, not a bunch of individual pieces.
  - threeseed 11y ago
    It's not short sighted. It's called not prematurely optimising. There is a world of difference between the issues Facebook, Google, LinkedIn etc are dealing with and the rest of us. Nobody is worrying whether 10% of their CPU/Memory/Network capacity is going unused if we only have a handful of servers. Or overly worrying about interconnect bandwidth beyond a certain point. It is fascinating and useful to see how to scale but we aren't all in the dark ages simply because we don't follow their architectural approaches.
    - mikecb 11y ago
      No one is saying you should build this in your back office. My comment was limited (by the first bit) to people utilizing public clouds and those who run public clouds. If you are, you are affected by these issues; if you are not, then there's an opportunity cost there, whether or not you believe the early optimization is greater or lesser.
- dekhn 11y ago
  The two examples I think that matter are Amazon and Microsoft. I don't think they copied Google- I think they independently realized, when they decided to enter the cloud service, that they had to scale up and out their data centers tremendously. And when you do that, you quickly learn that deploying more hosts is easier than deploying the ports to interconnect those hosts with a globally nonblocking fabric. Amazon and MSFT have both published nice papers showing that they "got it" about 3-5 years ago. I think they operate within an order of magnitude of Google in terms of total external and internal traffic, but that's just a guess because none of these companies publish enough detail to say for sure.
  - Shebanator 11y ago
    As a googler, I would agree with this bucketing. Amazon and Microsoft are in the same ballpark scale-wise, but companies like LinkedIn and Netflix have lower scaling needs (this isn't a knock on them, just a technical reality). Netflix would likely be in the same ballpark if they didn't leverage Amazon. Facebook is also hovering near the line, and has probably crossed over.

threeseed 11y ago

> nobody else has copied google yet Why would they ? Almost nobody is operating at Google's scale and interconnects like Infiniband are rarely saturated to the point that exotic solutions are needed. Fast RPC is nice and all but (a) not that many companies are using microservices and (b) not that many are scaling out to the billions of requests/second. And the 20 or so companies who are operating at Google's scale are hardly floundering around blind. Facebook has done very interesting work in the infrastructure space, LinkedIn is amazing on the core software side and Netflix has pioneered the use of the cloud.

mikecb 11y ago
I think this is short-sighted. Everyone who operates with any portion of their stack on any cloud provider is "operating at Google's scale" since even a small service is affected both by the infrastructure on which it is run and all of the other services run on that infrastructure. If you're running in a cloud, you need to plan and develop like you're a product team at facebook or google, and cloud providers need to act like they are providing infrastructure for a cohesive system, not a bunch of individual pieces.
- threeseed 11y ago
  It's not short sighted. It's called not prematurely optimising. There is a world of difference between the issues Facebook, Google, LinkedIn etc are dealing with and the rest of us. Nobody is worrying whether 10% of their CPU/Memory/Network capacity is going unused if we only have a handful of servers. Or overly worrying about interconnect bandwidth beyond a certain point. It is fascinating and useful to see how to scale but we aren't all in the dark ages simply because we don't follow their architectural approaches.
  - mikecb 11y ago
    No one is saying you should build this in your back office. My comment was limited (by the first bit) to people utilizing public clouds and those who run public clouds. If you are, you are affected by these issues; if you are not, then there's an opportunity cost there, whether or not you believe the early optimization is greater or lesser.
dekhn 11y ago
The two examples I think that matter are Amazon and Microsoft. I don't think they copied Google- I think they independently realized, when they decided to enter the cloud service, that they had to scale up and out their data centers tremendously. And when you do that, you quickly learn that deploying more hosts is easier than deploying the ports to interconnect those hosts with a globally nonblocking fabric. Amazon and MSFT have both published nice papers showing that they "got it" about 3-5 years ago. I think they operate within an order of magnitude of Google in terms of total external and internal traffic, but that's just a guess because none of these companies publish enough detail to say for sure.
- Shebanator 11y ago
  As a googler, I would agree with this bucketing. Amazon and Microsoft are in the same ballpark scale-wise, but companies like LinkedIn and Netflix have lower scaling needs (this isn't a knock on them, just a technical reality). Netflix would likely be in the same ballpark if they didn't leverage Amazon. Facebook is also hovering near the line, and has probably crossed over.