3 ms·
The flip side of this is meta having a hack that keeps their GPUs busy so that the power draw is more stable during llm training (eg don't want a huge power dro
by wrsh07 1y ago
The flip side of this is meta having a hack that keeps their GPUs busy so that the power draw is more stable during llm training (eg don't want a huge power drop when synchronizing batches)
- jack_riminton 1y agoI thought of this too. Iirc it was bigger problem because surging spikes in power and cooling were harder and more costly to account for. I'm not au fait with network data centres though, how similar are they in terms of their demands?
- wrsh07 1y agoI think the issue is exactly the spikiness because of how AC electricity works (whereas if the data centered were DC - eg wired through a battery - it wouldn't be an issue) I expect you're right that GPU data centers are a particularly extreme example
- Scipio_Afri 1y agoHave a link to an article talking about them doing this?
- wrsh07 1y agoI tried desperately to source this even before seeing your request. My current guess is that I heard it on a podcast (either a Dwarkesh interview or an episode of something else - maybe transistor radio? - featuring Dylan Patel). I'll try to re listen to top candidates in the next two weeks (a little behind on current episodes because I'm near the end of an audiobook) and will try to ping back if I find it. If too long has elapsed, update your profile so I can find out how to message you!
- ikekkdcjkfke 1y agoThat goes against separation of concerns. A separate utility must be created for that specific purpose, not hidden in some other part of the system