2 ms·
32B (also 34B) is a neat size for prosumer hardware and more compatible with running the llm alongside other apps context: Qwen 1.5 72B is quite good. It ranks
by tosh 3y ago
32B (also 34B) is a neat size for prosumer hardware and more compatible with running the llm alongside other apps
context: Qwen 1.5 72B is quite good. It ranks higher than Mistral-Medium (proprietary) on Chabot Arena
edit:
other interesting models in this weight (ha) class:
* Yi-34B (https://huggingface.co/01-ai/Yi-34B https://huggingface.co/01-ai/Yi-34B)
* Command-R 35B (https://huggingface.co/CohereForAI/c4ai-command-r-v01 https://huggingface.co/CohereForAI/c4ai-command-r-v01)