3 ms·

Is this wrong? It says 65.8GB. If it's wrong, what source should I be using instead? https://llm.extractum.io/model/Qwen%2FQwen2.5-Coder-32B-Instruct,6nvrT0uDy

• exe34 2y ago
  the ollama one is probably quantised.
  • mistercheph 2y ago
    Ollama's default quantization is q4_0 which is quite bad, but you can go to ollama's model page to see all the quantizations they have available, you can do e.g. "ollama run qwen2.5-coder:32b-instruct-q8_0" which will need ~35G + space for context