3 ms·
Wouldn’t that situation arise when a company is training their top end model? Facebook/Google/DeepSeek probably trained on thousands of collocated GPUs. The big
by 0cf8612b2e1e 1y ago
Wouldn’t that situation arise when a company is training their top end model? Facebook/Google/DeepSeek probably trained on thousands of collocated GPUs. The bigger the cluster, the bigger the sync delays between batches as the model data gets shunted back and forth.
- deleted 1y ago[deleted]
- lstodd 1y agobasically it's a load-spreading problem. this can be mitigated entirely by control plane alone. unless you think they really do a cluster-per-datacenter and in that case, I cannot believe the DC in question were not designed for peak and transient loads both in AC and power supply. Besides it would be stupud.
- jwatte 1y agoYes, the data center was designed for peak load. Unfortunately, the data center is hooked up to the grid. De-coupling that connection is what this article is literally about. Then again, some data centers just use re-generation from giant flywheels, where the grid powers the flywheel to build up inertia, and thus load can be smoothed that way. The flywheels need to keep running at 60 Hz for at least 30 seconds to give the Diesel generators time to start, should the grid fail. Run a flywheel at the main transformer, run batteries at your PDC and add supercaps at each point of load, and you may very well be able to show a much smoother load to the grid.