3 ms·
Llama3 8b is in no way equivalent to Gemini Flash or gpt-4o mini.
by ilaksh 2y ago
Llama3 8b is in no way equivalent to Gemini Flash or gpt-4o mini.
- ilaksh 2y agoMini is 82% MMLU and 8b is 68%
- wkat4242 2y agoI'm really missing a middle-ground llama3 model. Llama 2 had a 12b but Llama3 is 8 or 70. 12 or 13 would be pretty ideal for a 16gb card :(
- ilaksh 2y agoMistral just released a totally open 12b model
- wkat4242 2y agoYeah I saw! I hope it comes to ollama soon.
- vineyardmike 2y agoGemma 27b? Command R (32b) would be ideal middle grounds but won’t fit in a 16gb card. There are a handful of 12gb like the new mistral though. I doubt that 12b overs much improvement over a 7b to compare to 70b. Seems like an entirely different class. You probably want to limit yourself if you do have a 16gb card because you still need to fit the context window in memory too.
- wkat4242 2y agoI tried gemma2 but it has to be quantified to hell to fit in a 16gb card and then it performs a lot worse than llama3 8b unforutnately. I hope the new mistral comes out soon for ollama. True about the context window but with llama3 this was not a problem as it has such a small context window anyway.