Hm… the q4 34B code llama (which is used here) performs quite poorly in my experience. Using a high quantised larger model gives you an unrealistic impression

Hm… the q4 34B code llama (which is used here) performs quite poorly in my experience. Using a high quantised larger model gives you an unrealistic impression that smaller models and larger models are roughly equivalently capable… but it’s a trade off. The larger codellama model is categorically better, if you don’t lobotomise it. It’d be better if instead of making opinionated choices (which aren’t great) it guided you on how to select an appropriate model…