4 ms·
Qwen1.5-32B: Fitting the Capstone of the Qwen1.5 Language Model Series
- deleted 3y ago[deleted]
- nilsherzig 3y agoI would like to see how it competes against command-r (non plus)
- jpetrucc 3y agoInteresting to see a "modern" ~30b sized model - will have to give this a try!
- tosh 3y ago32B (also 34B) is a neat size for prosumer hardware and more compatible with running the llm alongside other apps context: Qwen 1.5 72B is quite good. It ranks higher than Mistral-Medium (proprietary) on Chabot Arena edit: other interesting models in this weight (ha) class: * Yi-34B (https://huggingface.co/01-ai/Yi-34B https://huggingface.co/01-ai/Yi-34B) * Command-R 35B (https://huggingface.co/CohereForAI/c4ai-command-r-v01 https://huggingface.co/CohereForAI/c4ai-command-r-v01)
- wing-_-nuts 3y agoI guess I look forward to seeing 'TheBloke' do a finetune with vram reqs. If my 3090 could fit a good quality quant I'd be interested. I tried to run a mixtral with vram / sys ram offloading and got garbage ouput, but that's probably down to some mistake I made.
- kken 3y agoTry Ollama or LM Studio. Mixtral and its finetunes work perfectly for me on my RTX3090 with offloading.
- tosh 3y agoGGUF: https://huggingface.co/Qwen/Qwen1.5-32B-Chat-GGUF https://huggingface.co/Qwen/Qwen1.5-32B-Chat-GGUF AWQ: https://huggingface.co/Qwen/Qwen1.5-32B-Chat-AWQ https://huggingface.co/Qwen/Qwen1.5-32B-Chat-AWQ