3 ms·
The GGUF quantisations do run on most recent hardware, albeit at increasingly concerning quality tradeoffs.
by Filligree 2y ago
The GGUF quantisations do run on most recent hardware, albeit at increasingly concerning quality tradeoffs.
- tripplyons 2y agoI haven't noticed any quality degradation with the 8-bit GGUF for Flux Dev, but I'm sure the smaller quantizations perform worse.