5 ms·
In int8 7B needs only 9GB of VRAM and 13B needs only 20GB on a single GPU. https://github.com/oobabooga/text-generation-webui/issues/147#issuecomment-1454841458
by MacsHeadroom 4y ago
In int8 7B needs only 9GB of VRAM and 13B needs only 20GB on a single GPU. https://github.com/oobabooga/text-generation-webui/issues/147#issuecomment-1454841458 https://github.com/oobabooga/text-generation-webui/issues/14...