3 ms·
I'm waiting for FP8 quant, preferably from Google.
by zkmon 5mo ago
I'm waiting for FP8 quant, preferably from Google.
- mdp2021 5mo agoIf you accept the "ggml-org": https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/main https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/tree/mai... https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/blob/main/gemma-4-12B-it-Q8_0.gguf https://huggingface.co/ggml-org/gemma-4-12B-it-GGUF/blob/mai...
- zkmon 5mo agoDo they run well on vLLM?