4 ms·
How possible is it to run these models on a gaming GPU?
by modeless 4y ago
How possible is it to run these models on a gaming GPU?
- coolspot 4y agoYes, if your gaming rig is NVidia DGX-1 workstation. https://youtu.be/5TRr2oWeSw0 https://youtu.be/5TRr2oWeSw0
- infinityio 4y agoa very rough approximation is 2GB of vram for every billion fp16 model parameters, so the lower end models may be just about achievable on high-end cards
- sp332 4y agoIf you're patient, https://github.com/FMInference/FlexGen https://github.com/FMInference/FlexGen lets you trade off GPU RAM for system RAM or even disk space.