3 ms·
because how huge glm 4.7 is https://huggingface.co/zai-org/GLM-4.7 https://huggingface.co/zai-org/GLM-4.7
by disiplus 9mo ago
because how huge glm 4.7 is https://huggingface.co/zai-org/GLM-4.7 https://huggingface.co/zai-org/GLM-4.7
- omneity 9mo agoExcept this is GLM 4.7 Flash which has 32B total params, 3B active. It should fit with a decent context window of 40k or so in 20GB of ram at 4b weights quantization and you can save even more by quantizing the activations and KV cache to 8bit.
- disiplus 9mo agoyes, but the parrent link was to the big glm 4.7 that had a bunch of ggufs, the new one at the point of posting did not, nor does it now. im waiting for unsloth guys for the 4.7 flash