3 ms·
ONNX doesn't support the same level of quantization as GGML. So basically GGML will run on hardware with less memory.
by ianpurton 3y ago
ONNX doesn't support the same level of quantization as GGML.
So basically GGML will run on hardware with less memory.
- regularfry 3y agoOr alternatively, bigger models with the same memory (just quantised harder).