3 ms·
A few notes: Nvidia always announces their hardware in advance of availability, while Google typically announces some time after they've started using it intern
by modeless 3y ago
A few notes: Nvidia always announces their hardware in advance of availability, while Google typically announces some time after they've started using it internally. Also TPU has been more cost-focused than Nvidia, since Google uses these chips internally to serve their web traffic while Nvidia is supply constrained and can name their price right now, plus they don't pay the electricity bills.
There are also a few things wrong with the specs you quoted. 282 GB is split between two GPUs, it's 144 GB per GPU (not sure where the extra 6 GB went). The TPU pod throughput number you quoted is interconnect bandwidth which you compared to GH200's memory bandwidth. Those numbers are not comparable.
I believe GH200 does beat TPU v5p per chip, however the differences are not anywhere near as large as your comparison suggested. And it's likely that TPU v5p is dramatically more cost effective but we don't have Google's internal numbers to prove that.
- nabla9 3y ago> 282 GB is split between two GPUs There is only one GPU in GH200 and it gets all 282 GB HBM3e memory. There is another 480 GB LPDDR5X memory for the another chip (Grace CPU, not GPU). GH200 interconnect bandwidth trough NVLink is 900 GB/s.
- modeless 3y agoI don't believe this is true. GH200's GPU has 144GB of HBM3e according to their datasheet. https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-superchip https://resources.nvidia.com/en-us-grace-cpu/grace-hopper-su... The 282 GB figure comes from a "dual configuration" server with two CPU chips and two GPU chips: > the new platform will be available in a wide range of configurations. The dual configuration [...] comprises a single server with 144 Arm Neoverse cores, eight petaflops of AI performance and 282GB of the latest HBM3e memory technology. Note that one Grace Hopper CPU has 72 cores so this configuration clearly has two CPUs and two GPUs in one server, and the 282 GB HBM3e total is split between the two GPUs with 144 GB each (again, not sure where the extra 6 GB went, maybe disabled for yield issues?). https://nvidianews.nvidia.com/news/gh200-grace-hopper-superchip-with-hbm3e-memory https://nvidianews.nvidia.com/news/gh200-grace-hopper-superc...
- m00x 3y agoThat's mostly true. Nvidia also spends a lot of time internally testing their cards before shipping them out. Nothing would be worse than a massive hardware recall. If anything, Google would have an easier time going live sooner than Nvidia since they own all the TPUs and they're onsite.