Looking at the different quantization levels examples, I'm quite impressed. The change from f16 to q8_0 seems to be more of a change in direction than loss o

Looking at the different quantization levels examples, I'm quite impressed. The change from f16 to q8_0 seems to be more of a change in direction than loss of quality. The q5_1 result seems indistinguishable from the q8_0. So you're losing determinism with the higher precision models, but potentially quite usable.