5 ms·
One thing I'm happy about is that the gguf file format is getting widely used, here endorsed by Google. I vastly prefer working with gguf and have adopted it fo
by andy99 3y ago
One thing I'm happy about is that the gguf file format is getting widely used, here endorsed by Google. I vastly prefer working with gguf and have adopted it for my project, hopefully it gets the critical mass to become a "native" format and we can stop with pytorch checkpoints.
- brucethemoose2 3y agoOn the contrary, it's not one-size-fits-all. It doesn't support everything other backends need (like certain quantization schemes). If it somehow does, then its not universally compatible and that's just going to create more confusion.
- behnamoh 3y agoIt supports grammar and that's more than enough in practice.
- brucethemoose2 3y agoGrammar has nothing to do with gguf, as it's logit processing (and that is being integrated into other backends as well).