3 ms·
headline hundred billion parameter, none of the official models are over 10 billion parameters. Curious.
by QuadmasterXLII 7mo ago
headline hundred billion parameter, none of the official models are over 10 billion parameters. Curious.
- Tuna-Fish 7mo agoThe project is an inference framework which should support 100B parameter model at 5-7tok/s on CPU. No one has quantized a 100B parameter model to 1 trit, but this existing is an incentive for someone to do so.
- est 7mo ago> quantized a 100B parameter model to 1 trit I had the same question, after some debates with Chatgpt, it's not the "quantize" for post-training we often witness these days, you have to use 1 trit in the beginning since pre-train.