8 ms·
How thermal management is changing in the age of the kilowatt chip
- ksec 3y agoAssuming the upcoming Zen 5c was capped at 192 Core because of Bandwidth and not Thermal. We could have had 256 Core + IOD ( 70W ), if every core were to use 3.6W that is nearly 1000W for the CPU Socket. In a 2U 2Node system, this is a potential of 1024 vCPU in a single server.
- gfv 3y agoIn datacenters, you're mostly limited by the power (and thus cooling). Most commercial DCs only let you use up to about 10kW per rack. For standard 40U racks it's just 250W/RU, give or take. There are niche expensive datacenters with higher power density, but as it stands, exotic multi-kW hardware at scale makes sense if you either save a ton on per-node licensing, or you need extreme bandwidth and/or low latency.
- hinkley 3y agoI have been told by tech writers that Google discovered that at some point electricians will refuse to route more power into a building. So even if you created a separate thermal plant, you still have issues.
- repiret 3y agoSome electricians just assume you’re on a budget. There’s always some limiting factor, and there’s always some (possibly crazy expensive) way to resolve it and get a bit more power until you run into the next limiting factor.
- kuchenbecker 3y agoYou build another datacenter rather than make the existing bigger.
- hinkley 3y agoWhy would they assume Google or Amazon for that matter, are on a budget?
- eternauta3k 3y agoReally? I imagine datacenters are not reaching powers anywhere near those of e.g. arc furnaces.
- zamfi 3y agoArc furnace: peak of about 250 MW to melt the steel [1] Datacenter: seems to cap out at around 850 MW [2] Same ballpark I guess? Probably both are limited by inexpensive power availability + other connectivity factors (road/rail, fiber). [1]: “Therefore, a 300-tonne, 300 MVA EAF will require approximately 132 MWh of energy to melt the steel, and a "power-on time" (the time that steel is being melted with an arc) of approximately 37 minutes.” via https://en.m.wikipedia.org/wiki/Electric_arc_furnace https://en.m.wikipedia.org/wiki/Electric_arc_furnace [2]: https://www.racksolutions.com/news/blog/how-many-servers-does-a-data-center-have/amp/ https://www.racksolutions.com/news/blog/how-many-servers-doe...
- hinkley 3y agoI knew they were putting ridiculous power into these buildings but that’s absurd. How many backup generators does that need??
- ksec 3y ago>Most commercial DCs only let you use up to about 10kW per rack. I think that was the case in 2020; >By 2020, that was up to 8–10 kW per rack. Note, though, that two-thirds of U.S. data centers surveyed said that they were already experiencing peak demands in the 16–20 kW per rack range. The latest numbers from 2022 show 10% of data centers reporting rack densities of 20–29 kW per rack, 7% at 30–39 kW per rack, 3% at 40–49 kW per rack and 5% at 50 kW or greater. We dont have 2023 numbers and we are coming to 2024. But it is clear that demands for high power density is growing. ( And hopefully at a much faster pace )
- wmf 3y agoYou can already overclock a 96C Threadripper 7000 to 1000W.
- layer8 3y agoMaybe we’ll be able to heat our homes with self-hosted AI in a few years.
- lisper 3y agoYou can already heat your house with bitcoin mining rigs. (Hm, that's actually not such a bad idea!)
- aftbit 3y agoIf you're going to run an electric space heater anyway, might as well do some mining instead. :P
- lisper 3y agoThe more I think about this the more I think there could be a real business model here. Make a device that looks and acts like a space heater but is in fact a miner loaded with your private keys. Market it as a "smart heater" that requires a wifi connection to operate. Sell it as a loss leader, or maybe even give it away for free. It doesn't have to be state-of-the-art hardware, so it can be cheap to make. Hmm...
- idiotsecant 3y agoIt's been done. Space heaters are like 200 bucks. Hard to compete with that.
- talldatethrow 3y agoSpace heaters are $20* at Walmart for basically as much energy draw as a typical household socket can handle.
- beAbU 3y agoMining hardware is really expensive still, and they are constantly replaced by better models. I have a massive surplus of solar in summer, and on my local auction sites some old miners became available. I considered picking them up and putting them to use using up my surplus "free" electricity. I gave up when the mining ROI calculation came back with multiple years, even when you factor in a $0 electicity cost. This might've worked 3-5 years ago (maybe more), but I don't see that happening now to be honest.
- joe_the_user 3y agoOK, So what's the limit of a system you could run from your own house (or apartment)? A single standard outlet yields 1500-1800 watts but there are higher voltage/amperage outlets in many houses.
- ygra 3y ago3.5 kW here at least (16 A × 230 V). If you replace the stove with a computer, you get three phases, which supplies a bit more.
- LeonM 3y agoI think you are confusing a breaker group with your total residential connection. Assuming you are in Europe, and you have 2.5mm² cabling (which is the standard for residential applications) then you are indeed limited to 16A per group. However, there is nothing preventing you from using multiple groups for one appliance. This is actually typical for high-power appliances, such as induction cooking. Ultimately it is your main fuse that limits your total power consumption, which for most European countries this is typically rated at 25A (5750W), but on request you can usually have this raised to 35A, 50A or even 80A, if supply is sufficient.
- CapitalistCartr 3y agoIf you're in USA or Canada, your house is probably over-wired and can handle some extra load. Plugging into the electric dryer outlet is good for 5760 continuous watts. The stove outlet is 9600 continuous watts. An electrician could easily add an additional one.
- xattt 3y agoTankless electric hot water heaters can be rated for 24-36 ㎾ (max current draw 150 A)! These are wired with 3-4 8-gauge wires in parallel and connected to 40A breakers. The only limit to household wiring would be the capacity of the distribution coming into your home.
- deleted 3y ago
- brucethemoose2 3y agoIf y'all missed it, the Cerebras CS-2 teardown is amazing: https://www.youtube.com/watch?v=pzyZpauU3Ig https://www.youtube.com/watch?v=pzyZpauU3Ig The engineering to handle that power density is insane. Technically its less power per mm^2, but the chip is the size of a dinner plate. EDIT: The video was taken down, but looks like the web archive got it: https://web.archive.org/web/20230812020202/https://www.youtube.com/watch?v=pzyZpauU3Ig https://web.archive.org/web/20230812020202/https://www.youtu... As well as Vimeo (thanks morcheeba): https://vimeo.com/853557623 https://vimeo.com/853557623
- webstrand 3y agoThat video is no longer available?
- brucethemoose2 3y agoHuh. I copied the link from here: https://news.ycombinator.com/item?id=37096214 https://news.ycombinator.com/item?id=37096214 There are reposts with that same link. Maybe it revealed too much and was taken down?
- morcheeba 3y agoIs it the same as this video? https://vimeo.com/853557623 https://vimeo.com/853557623
- brucethemoose2 3y agoYeah, that's it!
- rwmj 3y agoThat video and the cooling system is insane. Insanely cool even. Thanks for posting. I wonder, how do all the contacts in the 20,000A power distribution plate that is bolted on top of the wafer-scale die line up? The engineering involved in just making that part work must be crazy.
- avereveard 3y agothis site has picture for everything but the current article.
- jl6 3y agoOne hopes this kind of heat density in a single system is still more efficient than operating multiple less-power-hungry systems.
- brianolson 3y agoIt's not. It's not about efficiency. Compute-per-watt will certainly be better in other systems. This is about pushing a small system as fast as possible because it's easier to program for a small system. A few problems are 'embarrassingly parallel', but lots have substantial overhead as parallelism increases so running each core as fast as possible is a win for some problems.
- bluenose69 3y agoI had to smile at the sentence "It essentially boils down to how much surface area you have to dissipate the heat" in the article :-)
- hinkley 3y agoI can’t help but think that someday we will see chips designed like Sierpinski gaskets, with “holes” used for thermal transport and the solid parts used for computation. Such chips would behave more or less like a toroidal structure. Now that chiplets are maturing that is a little less far-fetched.
- dist-epoch 3y agoWhy don't they build data centers where it's really cold, for example north of Canada, and just pipe outside air in? If you train an AI, you don't need low latency/high bandwidth Internet access.
- deleted 3y ago[deleted]
- skirmish 3y agoGoogle does in Finland, https://templ.io/blog/hamina-google-data-center/ https://templ.io/blog/hamina-google-data-center/ Sea water is corrosive, hard to use for cooling.
- appointment 3y agoSea water in the eastern Gulf of Finland is about 0.4% salt by weight, compared to 3.5% salt in the ocean.
- yetanotherloss 3y agoThey do, Google had/has one in the Dalles in Oregon next to the dam and there are several in line Canada and Finland. Part of the problem is consistent humidity management, but the other is that air, even really cold air, isn't dense enough to move specific heat as effectively as the same volume of a denser working fluid. For a practical example in the other direction, look at combined cycle gas turbines which use heated air for the primary turbine and the recapture as much denser steam for the secondary.
- icegreentea2 3y agoThe "north" of Canada (assuming you mean a place that has some semblance of infrastructure) doesn't get -that- cold in the summer. Yellowknife and Whitehorse both have average summer temperatures around 20C. In any case, I think you'd still need substantial cooling infrastructure to deal with the summer. Your capital costs are going to waaaay higher. While your thermal management costs will go down, your payroll will probably be higher.
- ChuckMcM 3y agoGetting closer to Jim Gray's "smoking hairy golf ball" (Jim also forecasted that chips would eventually become spherical to limit the lengths of the data paths.) It will be interesting to see if commercial data centers re-fit with cooling water transport under the floor or above the machines (riskier). This will be a challenge for DCs without a lot of space under there, presumably they could boost the floor height after a door transition. Still how many of them have 20 - 40kW of power allocated per rack. Its one of the few times I miss being at Google because they approached this sort of problem very creatively and with an effectively unlimited budget to try different things. I'm sure their data centers are very much different from my time there!
- icefo 3y agoI had the chance to visit the new datacenter of my college and on the server exhaust side there is a radiator as tall as the rack with cold water in it. All the pipes are under the floor. IIRC they mainly put power hungry compute nodes for the clusters in this new datacenter and I remember that servers full of GPUs had crazy power draw. The water then goes through an heat exchanger to help generate hot water to heat the campus and for the taps.
- akira2501 3y agoIt reminds me of Seymour Cray's solution. Well, series of solutions, to run Freon directly through the frame of the Cray-1. https://www.youtube.com/watch?v=vtOA1vuoDgQ&t=1530s https://www.youtube.com/watch?v=vtOA1vuoDgQ&t=1530s
- deleted 3y ago[deleted]
- morphle 3y agohttps://vimeo.com/731037615 https://vimeo.com/731037615 for a discussion of wafer scale integration design issues. The best way to cool is to go for immersive cooling. Even better is to eliminate most of the heat of the wires with free space optics.
- bgnn 3y agoThe core voltage for 7nm is 0.75V. The max over-voltage would ve 0.9V or so but the lifetime of the device will be limited to less than a year if they don't cool the devices to 50-60C or so in that case. BCI aging is no joke. So I assume they are using around 0.9V and 2.2kA meaning ~2kW.