Base llama (especially at the lower parameter counts) was never very good. You need a Q/A finetune with the exact formatting. And even then... Trivia is not th

Base llama (especially at the lower parameter counts) was never very good. You need a Q/A finetune with the exact formatting. And even then... Trivia is not their strong suit.