6 ms·

As a datacenter technician for Google, this makes me happy to see other people recognize this.

by aliasxneo 9y ago

kuschku 9y ago
It’s just sad that we’ll never see any of that openly available, or described.
- j-walker 9y ago
  There's a lot of openly available networking hardware on the Open Compute Project.
  - kuschku 9y ago
    Sure, but that's not even close to the "every board and switch is custom designed" story.
    - pyvpx 9y ago
      there isn't as much exotic "stuff" as you might think. they have the same ODMs build switches and line cards to their specifications, with chips from the same vendors (Broadcom, Cavium, etc) only they can easily hire a team and sign the NDA and write/optimize/perfect the forwarding plane and control plane implementations. hell, the IS-IS implementation in Quagga is usable because of Google :)
- jankey 9y ago
  It's not a lot, but there are a bunch of pictures on google.com/about/datacenters, including a 'street view' inside our DC in North Carolina. There are also a bunch of papers on the network architecture, https://research.google.com/pubs/Networking.html https://research.google.com/pubs/Networking.html . But yes, a lot is closed for the obvious reason that this is really core to the business Google is running. (I'm a datacenter engineer at Google)
- gcardone_ 9y ago
  Disclaimer: I'm an SRE at Google, my opinions are my own. We actually release a decent amount of information, see [0]. I particularly recommend "Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network" [1]. [0] https://research.google.com/pubs/Networking.html https://research.google.com/pubs/Networking.html [1] https://research.google.com/pubs/archive/46286.pdf https://research.google.com/pubs/archive/46286.pdf
winslow 9y ago
Could you shed some light on why google moved away from their shipping container design to a more traditional yet still custom data center approach? Thanks!
- pavs 9y ago
  Not working for Google, but I think it's for multiple reasons. - hardware has gotten a lot faster, which helps the second reason. - Google giving away free GGC nodes to ISPs (if you meet their requirements). The distribution of these nodes has gotten a lot wider since container days. - Google is building more smaller scale (a dozen racks or so) traditional data centers closer to ISPs. I don't think shipping container design was never a very good idea, to begin with - it was an interesting experiment that didn't scale as well as they thought it would.

As a datacenter technician for Google, this makes me happy to see other people recognize this.

kuschku 9y ago

It’s just sad that we’ll never see any of that openly available, or described.

j-walker 9y ago
There's a lot of openly available networking hardware on the Open Compute Project.
- kuschku 9y ago
  Sure, but that's not even close to the "every board and switch is custom designed" story.
  - pyvpx 9y ago
    there isn't as much exotic "stuff" as you might think. they have the same ODMs build switches and line cards to their specifications, with chips from the same vendors (Broadcom, Cavium, etc) only they can easily hire a team and sign the NDA and write/optimize/perfect the forwarding plane and control plane implementations. hell, the IS-IS implementation in Quagga is usable because of Google :)
jankey 9y ago
It's not a lot, but there are a bunch of pictures on google.com/about/datacenters, including a 'street view' inside our DC in North Carolina. There are also a bunch of papers on the network architecture, https://research.google.com/pubs/Networking.html https://research.google.com/pubs/Networking.html . But yes, a lot is closed for the obvious reason that this is really core to the business Google is running. (I'm a datacenter engineer at Google)
gcardone_ 9y ago
Disclaimer: I'm an SRE at Google, my opinions are my own. We actually release a decent amount of information, see [0]. I particularly recommend "Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network" [1]. [0] https://research.google.com/pubs/Networking.html https://research.google.com/pubs/Networking.html [1] https://research.google.com/pubs/archive/46286.pdf https://research.google.com/pubs/archive/46286.pdf

winslow 9y ago

Could you shed some light on why google moved away from their shipping container design to a more traditional yet still custom data center approach? Thanks!

pavs 9y ago
Not working for Google, but I think it's for multiple reasons. - hardware has gotten a lot faster, which helps the second reason. - Google giving away free GGC nodes to ISPs (if you meet their requirements). The distribution of these nodes has gotten a lot wider since container days. - Google is building more smaller scale (a dozen racks or so) traditional data centers closer to ISPs. I don't think shipping container design was never a very good idea, to begin with - it was an interesting experiment that didn't scale as well as they thought it would.