5 ms·

Brought to you by the future: "Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, m

by ttul 4y ago

Brought to you by the future: "Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired."

rpcope1 4y ago
So something like an optical FPGA?
- nighthawk454 4y ago
  not really, only the interconnects between nodes are optical
- formerly_proven 4y ago
  Multiplexer for optical paths, which allows you to change the point-to-point wiring. Much like physical circuit switching in a 50s telephone exchange, actually.
sanxiyn 4y ago
As the paper explains, optical circuit switches are not new in TPU v4 and not the main topic of this paper. Google was already using it for networking and published about it last year. For details, see https://arxiv.org/abs/2208.10041 https://arxiv.org/abs/2208.10041.
pclmulqdq 4y ago
The future of the 90's. Optical matrix switches like this have been around for a long time. These aren't doing packet switching (which honestly would be the future if done optically), it's more of a layer 1 thing - the switch replaces you plugging and unplugging a cable. Bell labs was building these kinds of switches back in the day.
- jeffbee 4y ago
  Like other Google datacenter technologies the innovation of their OCS is that it's a cheap hack that nobody would tolerate outside of Google. Their whole secret sauce is that everything just barely works.
  - dekhn 4y ago
    You think reconfigurable MEMS optical switches are cheap hacks? Um, no.
    - jeffbee 4y ago
      The whole paper is focused on cost efficiency. I think that's in keeping with Google's history, where for example they were the first to use ultra short range optics, because that is an OK way to save a dime.
      - dekhn 4y ago
        There's a difference between cost efficiency and "cheap". In this case, we're talking about advanced photonic systems running in a data center to tie together world-class computing equipment; they didn't go for "cheap". The total investment is in the tens of billions of dollars.
        pclmulqdq 4y ago
        They are "cheap hacks" compared to a packet switched network that runs at that speed. Also, this is another reason why Google can't sell TPUs to other people - nobody would put up with managing this sort of switching network for something they bought. The equivalent for NVidia is to use HDR/NDR Infiniband, and it allows you to run a multi-tenant cluster a lot more efficiently, at no practical loss of performance (due to the marginally higher latency).
        dekhn 4y ago
abcdabcd987 4y ago
On a related note, Google also uses optical circuit switches in their datacenter network. See the paper form SIGCOMM'22 [1]. [1] Jupiter Evolving: Transforming Google’s Datacenter Network via Optical Circuit Switches and Software-Defined Networking. https://research.google/pubs/pub51587/ https://research.google/pubs/pub51587/

by ttul 4y ago

rpcope1 4y ago

So something like an optical FPGA?

nighthawk454 4y ago
not really, only the interconnects between nodes are optical
formerly_proven 4y ago
Multiplexer for optical paths, which allows you to change the point-to-point wiring. Much like physical circuit switching in a 50s telephone exchange, actually.

sanxiyn 4y ago

As the paper explains, optical circuit switches are not new in TPU v4 and not the main topic of this paper. Google was already using it for networking and published about it last year. For details, see https://arxiv.org/abs/2208.10041 https://arxiv.org/abs/2208.10041.

pclmulqdq 4y ago

The future of the 90's. Optical matrix switches like this have been around for a long time. These aren't doing packet switching (which honestly would be the future if done optically), it's more of a layer 1 thing - the switch replaces you plugging and unplugging a cable. Bell labs was building these kinds of switches back in the day.

jeffbee 4y ago
Like other Google datacenter technologies the innovation of their OCS is that it's a cheap hack that nobody would tolerate outside of Google. Their whole secret sauce is that everything just barely works.
- dekhn 4y ago
  You think reconfigurable MEMS optical switches are cheap hacks? Um, no.
  - jeffbee 4y ago
    The whole paper is focused on cost efficiency. I think that's in keeping with Google's history, where for example they were the first to use ultra short range optics, because that is an OK way to save a dime.
    - dekhn 4y ago
      There's a difference between cost efficiency and "cheap". In this case, we're talking about advanced photonic systems running in a data center to tie together world-class computing equipment; they didn't go for "cheap". The total investment is in the tens of billions of dollars.
      - pclmulqdq 4y ago
        They are "cheap hacks" compared to a packet switched network that runs at that speed. Also, this is another reason why Google can't sell TPUs to other people - nobody would put up with managing this sort of switching network for something they bought. The equivalent for NVidia is to use HDR/NDR Infiniband, and it allows you to run a multi-tenant cluster a lot more efficiently, at no practical loss of performance (due to the marginally higher latency).
        dekhn 4y ago

abcdabcd987 4y ago

On a related note, Google also uses optical circuit switches in their datacenter network. See the paper form SIGCOMM'22 [1]. [1] Jupiter Evolving: Transforming Google’s Datacenter Network via Optical Circuit Switches and Software-Defined Networking. https://research.google/pubs/pub51587/ https://research.google/pubs/pub51587/