3 ms·
"Parameter count" is the "GHz" of AI models: the number you're most likely to see but least likely to need. All of the models compared (in the table on the hugg
by zamadatix 2y ago
"Parameter count" is the "GHz" of AI models: the number you're most likely to see but least likely to need. All of the models compared (in the table on the huggingface link) are 1-2 billion parameters but the models range in actual size by more than a factor of 10.
- int_19h 2y agoBecause of different quantization. However, parameter count is generally the more interesting number so long as quantization isn't too extreme (as it is here). E.g. FP32 is 4x the size of 8-bit quant, but the difference is close to non-existent in most cases.
- orbital-decay 2y ago>so long as quantization isn't too extreme (as it is here) This is true for post-training quantization, not for quantization-aware training, and not for something like BitNet. Here they claim comparable performance per parameter count as normal models, that's the entire point.
- charcircuit 2y agoTPS is the Ghz of AI models. Both are related to the the propagation time of data.
- idonotknowwhy 2y agoThen i guess vocab is the IPC. 10k mistral tokens are about 8k llama3 tokens