5 ms·
Is 100GB of parameters really that large? 128GB of RAM on a server class machine is not unusual. Seems such a model could fit entirely in RAM.
by native_samples 4y ago
Is 100GB of parameters really that large? 128GB of RAM on a server class machine is not unusual. Seems such a model could fit entirely in RAM.
- andbberger 4y agoGPU memory is generally much smaller and more expensive
- kristjansson 4y agoTo elaborate on the sibling comment: main memory is much bigger, but CPUs are much, much slower. It would be a challenge to merely run a model like this on CPU, and totally infeasible to train one. So the challenge is to fit into the memory of a single GPU you can afford, coordinate multiple GPUs, or efficiently page from main memory into GPU.