If you go to huggingface.co, you can tell it what specs you have and when you go to a model, it'll show you what variations of that model are likely to run well

If you go to huggingface.co, you can tell it what specs you have and when you go to a model, it'll show you what variations of that model are likely to run well. So if you go to this[0] random model, on the right there is a list of quantifications based on bits, and those you can run will be shown in green. [0] https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruc...