4 ms·
How big are these models that are downloaded? Is 7B 7 gigabytes of data?
by ralphc 3y ago
How big are these models that are downloaded? Is 7B 7 gigabytes of data?
- notpublic 3y agoIf the model is stored in 32bits, it will be ~4x param size You can see the actual size from hugging face. For example, https://huggingface.co/WizardLM/WizardLM-7B-V1.0/tree/main https://huggingface.co/WizardLM/WizardLM-7B-V1.0/tree/main Size with quantization, https://github.com/ggerganov/llama.cpp#quantization https://github.com/ggerganov/llama.cpp#quantization
- raincole 3y agoIt's actually very straightforward. 7B = 7 Billions parameters. If 1 parameter takes 1 byte then a 7B model would be about 7GB in size. Usually 1 parameter takes 4 bytes tho (a 32-bit float), so it would be aboult 28GB. You can use 16-bit, 8-bit or even 4-bit float.