3 ms·
Supposedly double the model size so 14gb. RTX 4090 might be able to handle it. You can use lambdalabs to rent a server gpu for one of the larger models.
by JimmyRuska 4y ago
Supposedly double the model size so 14gb. RTX 4090 might be able to handle it. You can use lambdalabs to rent a server gpu for one of the larger models.
- TaylorAlexander 4y agoI don't know if it matters but the 7B parameter checkpoint is 13.5GB in size. Someone with 24GB VRAM struggled to run it: https://github.com/facebookresearch/llama/issues/55 https://github.com/facebookresearch/llama/issues/55