3 ms·
>~6GiB RAM That's for the 7B model. The 30B model needs 24GB quantized (or 64GB for the unquantized model).
by yyyk 4y ago
>~6GiB RAM
That's for the 7B model.
The 30B model needs 24GB quantized (or 64GB for the unquantized model).
loading thread…