5 ms·
That is just one quantization method, and its really Q3_K_L and smaller (the first four points on the graph for each model) that butcher the model.
by brucethemoose2 3y ago
That is just one quantization method, and its really Q3_K_L and smaller (the first four points on the graph for each model) that butcher the model.