5 ms·
TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning
- ttul 4y agoBrought to you by the future: "Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired."
- rpcope1 4y agoSo something like an optical FPGA?
- nighthawk454 4y agonot really, only the interconnects between nodes are optical
- formerly_proven 4y agoMultiplexer for optical paths, which allows you to change the point-to-point wiring. Much like physical circuit switching in a 50s telephone exchange, actually.
- sanxiyn 4y agoAs the paper explains, optical circuit switches are not new in TPU v4 and not the main topic of this paper. Google was already using it for networking and published about it last year. For details, see https://arxiv.org/abs/2208.10041 https://arxiv.org/abs/2208.10041.
- pclmulqdq 4y agoThe future of the 90's. Optical matrix switches like this have been around for a long time. These aren't doing packet switching (which honestly would be the future if done optically), it's more of a layer 1 thing - the switch replaces you plugging and unplugging a cable. Bell labs was building these kinds of switches back in the day.
- jeffbee 4y agoLike other Google datacenter technologies the innovation of their OCS is that it's a cheap hack that nobody would tolerate outside of Google. Their whole secret sauce is that everything just barely works.
- dekhn 4y agoYou think reconfigurable MEMS optical switches are cheap hacks? Um, no.
- jeffbee 4y agoThe whole paper is focused on cost efficiency. I think that's in keeping with Google's history, where for example they were the first to use ultra short range optics, because that is an OK way to save a dime.
- dekhn 4y agoThere's a difference between cost efficiency and "cheap". In this case, we're talking about advanced photonic systems running in a data center to tie together world-class computing equipment; they didn't go for "cheap". The total investment is in the tens of billions of dollars.
- pclmulqdq 4y agoThey are "cheap hacks" compared to a packet switched network that runs at that speed. Also, this is another reason why Google can't sell TPUs to other people - nobody would put up with managing this sort of switching network for something they bought. The equivalent for NVidia is to use HDR/NDR Infiniband, and it allows you to run a multi-tenant cluster a lot more efficiently, at no practical loss of performance (due to the marginally higher latency).
- dekhn 4y ago
- abcdabcd987 4y agoOn a related note, Google also uses optical circuit switches in their datacenter network. See the paper form SIGCOMM'22 [1]. [1] Jupiter Evolving: Transforming Google’s Datacenter Network via Optical Circuit Switches and Software-Defined Networking. https://research.google/pubs/pub51587/ https://research.google/pubs/pub51587/
- nighthawk454 4y agosome interesting tidbits: > stretched our ML supercomputer scale .. to 4096 TPU v4 nodes > The Google tradition is to write retrospective papers ... TPU v4s and A100s deployed in 2020 and both use 7nm technology > The appropriate H100 match would be a successor to TPU v4 deployed in a similar time frame and technology (e.g., in 2023 and 4 nm). > TPU v4 supercomputers [are] the workhorses of large language models (LLMs) like LaMDA, MUM, and PaLM]. These features allowed the 540B parameter PaLM model to sustain a remarkable 57.8% of the peak hardware floating point performance over 50 days while training on TPU v4 supercomputers > Google has deployed dozens of TPU v4 supercomputers for both internal use and for external use via Google Cloud > Moreover, the large size of the TPU v4 supercomputer and its reliance on OCSes looks prescient given that the design began two years before the paper was published that has stoked the enthusiasm for LLMs
- CGamesPlay 4y agoIs there any way to purchase anything like a TPU? I guess the Cerebras Andromeda product is one, but I don't know if those are sold or leased. Any others? https://www.cerebras.net/andromeda/ https://www.cerebras.net/andromeda/
- wiz21c 4y agoFrom the page : "Andromeda, a 13.5 Million Core AI Supercomputer". Blown away by the number of cores (I considered myself lucky to have 2 10000+ cores GPU in my workstation) I then realized that the word "core" is singular in the sentence. Is it just a mistake or does it mean something else ? (genuine question, English is not my first language) EDIT: Ahhh a bit below on the page it is written "13.5 million AI-optimized cores" and there it's plural. So it was probably just a mistake.
- fastball 4y agoIt is not a mistake, that is how you phrase it in English when the noun ("core" in this case) is being used as part of a compound adjective. The convention is to keep the noun singular. e.g. "He commanded a ten thousand man army." (not men) "Andromeda, a trillion star galaxy, is 2.5 million lightyears away." (not stars) etc
- tobr 4y agoNot a native English-speaking person, but shouldn’t it be "He commanded a ten-thousand-man army" and "Andromeda, a trillion-star galaxy, is 2.5 million lightyears away"?
- fastball 4y agoI am a native speaking English-speaking person, and although I think you are correct in that compound adjectives have historically been connected with hyphens, that seems to have fallen out-of-fashion somewhat.
- 4y ago
- einpoklum 4y agoI hate the enormous waste of human ability, ingenuity and effort in the creation of proprietary technologies like this. You've made a chip? Offer it for everyone to use. Same goes for Amazon and Apple. It's not as though it's a chip that's only usable for Google-specific work.
- omeysalvi 4y agoIf people are not allowed to monetize their innovations, there is no incentive to innovate. While this needs to have its limits, sharing it to everyone immediately upon creation is not an answer.
- alfor 4y agoIt’s because Google is a monopoly it doesn’t operate on normal economic incentives. It’s goal is only to keep the monopoly: appear bening, keep tech advance in house, share it’s spying network with government so the government don’t regulate them, win-win.
- alphabetting 4y ago>keep tech advance in house they put out way more R&D research than other big firms. GPT-4 would most likely not exist today without them.
- joiojoio 4y agoGood news! They're being offered on Google Cloud: https://cloud.google.com/tpu/docs/system-architecture-tpu-vm#tpu_v4 https://cloud.google.com/tpu/docs/system-architecture-tpu-vm... So you don't have to be frustrated anymore.
- deleted 4y ago[deleted]