23 ms·
Welcome to the party! I joined ML because I realized I could help. You can too. I bet you’re both already thinking of clever ways to deal with massive models fr
by sillysaurusx 5y ago
Welcome to the party! I joined ML because I realized I could help. You can too. I bet you’re both already thinking of clever ways to deal with massive models from an infrastructure standpoint. That’s just one of hundreds of interesting problems.
- native_samples 5y agoIs 100GB of parameters really that large? 128GB of RAM on a server class machine is not unusual. Seems such a model could fit entirely in RAM.
- andbberger 5y agoGPU memory is generally much smaller and more expensive
- kristjansson 5y agoTo elaborate on the sibling comment: main memory is much bigger, but CPUs are much, much slower. It would be a challenge to merely run a model like this on CPU, and totally infeasible to train one. So the challenge is to fit into the memory of a single GPU you can afford, coordinate multiple GPUs, or efficiently page from main memory into GPU.