5 ms·
Maybe possible with a fabulous GPU, but still likely not, and if it did work it would take a horrendously long time. The real blocker is gonna be GPU memory.
by oofbey 5y ago
Maybe possible with a fabulous GPU, but still likely not, and if it did work it would take a horrendously long time. The real blocker is gonna be GPU memory. With an RTX 3090 you have 24 GB of GPU RAM and _might_ be able to try it, but I'm still not sure it would fit. The key model has 3.5 billion parameters, which at 16-bit requires 7GB of GPU-memory for each copy. Training requires 3 or 4 copies of the model, depending on the algorithm you use. And then you need memory for the data and activations, which you can reduce with a small batch size. But if it did fit, on a single GPU with a small batch size, you're probably looking at years of training time.
Even an RTX 3080 is a complete non-starter.
- manquer 5y agoSomething like the Quadro RTX 8000 may theoretically work, it does have 48GB of RAM [1]. [1] https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/quadro-product-literature/quadro-rtx-8000-us-nvidia-946977-r1-web.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/design-vi...
- oofbey 5y agoSure, with a $5k card like that it would be physically capable, but still unreasonably slow. FWIW RTX 8000 is previous-generation - the current gen is the A6000, which is also $5k, and similarly spec'd but faster. If you're looking for a deep learning card, they're actually a great value - I've got a bunch of them.
- jazzyjackson 5y agoI wonder whether it could be adapted to run on Apple M1 Ultra hardware, which can have 128GB of on-chip "unified" memory, can't find right away how much of that is available to the GPU or the neural cores, but if its even half of it, perhaps Apple will have the AI market cornered.