3 ms·
According to the huggingface blog[0], this model requires 640GB for BF16/FP16 and 320GB for GPTQ/int4. [0] https://huggingface.co/blog/falcon-180b https://hugg
by SparkyMcUnicorn 3y ago
According to the huggingface blog[0], this model requires 640GB for BF16/FP16 and 320GB for GPTQ/int4.
[0] https://huggingface.co/blog/falcon-180b https://huggingface.co/blog/falcon-180b