4 ms·
Waterwave Could Quench AIs' Thirst for GPU Memory
- gleenn 3y agoUnfortunately a reality vague decision of the technique. They say you can train multiple models at once but wouldn't this require N times more memory? They also say that models that require heavy compute won't parallelize well which also seems just a bit absurd because aren't all those shaders all firing off perfectly in parallel to do all the big matrix math?
- fulafel 3y agoAnyone have a link to the source paper?
- alecco 3y agoIt's paywalled and has no citations... https://ieeexplore.ieee.org/document/10130297 https://ieeexplore.ieee.org/document/10130297 https://www.computer.org/csdl/journal/tc/5555/01/10130297/1Nm2KDyyWyI https://www.computer.org/csdl/journal/tc/5555/01/10130297/1N...
- light_hue_1 3y agoThe person who wrote this article doesn't understand what the problem is, and how this paper doesn't address it at all. The paper has a better way to share one GPU between small training jobs. The real world problem is training huge models for a long time across many GPUs. The problem the paper solves is one that is practically irrelevant aside from some niche cases.
- anotherthrway 3y agoThe problem to solve is GPU utilization rate. Training a “huge model” on n GPUs will have a cyclical pattern of idle GPUs and GPUs are expensive. This approach is likely scheduling GPU access across t training jobs to insure GPU is always in use. Maximizing GPU utility is a key requirement for training as a service. The biggest issue is that the failure mode of this approach (one fault hoses the entire pipeline of T jobs) is exactly opposing the goals of its most likely users: Training SaaS providers.