3 ms·
I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.
by bronxbomber92 2mo ago
I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.
- ddzzz 2mo agoTry quantizing yourself, just a suggestion. https://github.com/pytorch/executorch/tree/main/examples/models/muse-glimmer#:~:text=For%20a%20custom%20local%20quantization,%20export_solo%20also%20accepts%20a%20consolidated%20BF16%20checkpoint%20with%20--checkpoint-dir%20and%20a%20recipe%20selected%20by%20--quant-recipe). https://github.com/pytorch/executorch/tree/main/examples/mod...