3 ms·
On the contrary, it's not one-size-fits-all. It doesn't support everything other backends need (like certain quantization schemes). If it somehow does, then it
by brucethemoose2 3y ago
On the contrary, it's not one-size-fits-all.
It doesn't support everything other backends need (like certain quantization schemes). If it somehow does, then its not universally compatible and that's just going to create more confusion.
- behnamoh 3y agoIt supports grammar and that's more than enough in practice.
- brucethemoose2 3y agoGrammar has nothing to do with gguf, as it's logit processing (and that is being integrated into other backends as well).