4 ms·
This matches the general consensus at the LocalLLama subreddit that Q8 is basically indistinguishable from the full model.
by staticman2 2y ago
This matches the general consensus at the LocalLLama subreddit that Q8 is basically indistinguishable from the full model.