> Will a model that small suffer heavily from a 4bit quantization? From my experience with quantized 7B llama models, avoid 3B if you can. Without benchmarks,

> Will a model that small suffer heavily from a 4bit quantization? From my experience with quantized 7B llama models, avoid 3B if you can. Without benchmarks, I think this is a decent rule of thumb.