3 ms·

Isn't it less for Google using their TPU compared to everyone else using nvidia?

jeffbee 1y ago
Maybe, but I would suggest that larger factors arise from utilization and overhead. These are going to be the competitive advantages that AI operators need to win the arms race.
westurner 1y ago
TPUs are more efficient at LLMs because they do more TOPS/KWh. "OpenTPU: Open-Source Reimplementation of Google Tensor Processing Unit (TPU)" (2025) https://news.ycombinator.com/item?id=44111452 https://news.ycombinator.com/item?id=44111452 "A Comparison of the Cerebras Wafer-Scale Integration Technology with Nvidia GPU-based Systems for Artificial Intelligence" (2025) https://arxiv.org/html/2503.11698v1 https://arxiv.org/html/2503.11698v1 From https://news.ycombinator.com/item?id=44648575 https://news.ycombinator.com/item?id=44648575 : > "Next-generation datacenters consume zero water for cooling" (2024) https://news.ycombinator.com/item?id=42376406 https://news.ycombinator.com/item?id=42376406 >> this design will avoid the need for more than 125 million liters of water per year per datacenter "Microsoft’s Datacenter Community Pledge: To build and operate digital infrastructure that addresses societal challenges and creates benefits for communities" (2024-06) https://blogs.microsoft.com/blog/2024/06/02/microsofts-datacenter-community-pledge-to-build-and-operate-digital-infrastructure-that-addresses-societal-challenges-and-creates-benefits-for-communities/ https://blogs.microsoft.com/blog/2024/06/02/microsofts-datac... : > We will design and operate our datacenters to support society’s climate goals and become carbon negative, water positive and zero waste before 2030. [...] > By 2025, we will procure 100% renewable energy on a global scale, both significantly expanding and decarbonizing local electricity grids. > Our datacenter designs are more water efficient than traditional enterprise datacenters, and our plan by 2030 is to replenish more water than we consume locally. Here's this about CNT cooling: "Cyberpower begins selling desktop PCs with carbon nanotube CPU cooling" (2025) https://news.ycombinator.com/item?id=44899495 https://news.ycombinator.com/item?id=44899495 "A carbon-nanotube-based tensor processing unit" (2024) https://news.ycombinator.com/item?id=41322070 https://news.ycombinator.com/item?id=41322070 Graphene semiconductors should be at least 10X more energy efficient; but how much less water would graphene-based chips waste?

Maybe, but I would suggest that larger factors arise from utilization and overhead. These are going to be the competitive advantages that AI operators need to win the arms race.

westurner 1y ago

TPUs are more efficient at LLMs because they do more TOPS/KWh. "OpenTPU: Open-Source Reimplementation of Google Tensor Processing Unit (TPU)" (2025) https://news.ycombinator.com/item?id=44111452 https://news.ycombinator.com/item?id=44111452 "A Comparison of the Cerebras Wafer-Scale Integration Technology with Nvidia GPU-based Systems for Artificial Intelligence" (2025) https://arxiv.org/html/2503.11698v1 https://arxiv.org/html/2503.11698v1 From https://news.ycombinator.com/item?id=44648575 https://news.ycombinator.com/item?id=44648575 : > "Next-generation datacenters consume zero water for cooling" (2024) https://news.ycombinator.com/item?id=42376406 https://news.ycombinator.com/item?id=42376406 >> this design will avoid the need for more than 125 million liters of water per year per datacenter "Microsoft’s Datacenter Community Pledge: To build and operate digital infrastructure that addresses societal challenges and creates benefits for communities" (2024-06) https://blogs.microsoft.com/blog/2024/06/02/microsofts-datacenter-community-pledge-to-build-and-operate-digital-infrastructure-that-addresses-societal-challenges-and-creates-benefits-for-communities/ https://blogs.microsoft.com/blog/2024/06/02/microsofts-datac... : > We will design and operate our datacenters to support society’s climate goals and become carbon negative, water positive and zero waste before 2030. [...] > By 2025, we will procure 100% renewable energy on a global scale, both significantly expanding and decarbonizing local electricity grids. > Our datacenter designs are more water efficient than traditional enterprise datacenters, and our plan by 2030 is to replenish more water than we consume locally. Here's this about CNT cooling: "Cyberpower begins selling desktop PCs with carbon nanotube CPU cooling" (2025) https://news.ycombinator.com/item?id=44899495 https://news.ycombinator.com/item?id=44899495 "A carbon-nanotube-based tensor processing unit" (2024) https://news.ycombinator.com/item?id=41322070 https://news.ycombinator.com/item?id=41322070 Graphene semiconductors should be at least 10X more energy efficient; but how much less water would graphene-based chips waste?