3 ms·
How large models can this run? Generally, is there a resource that has a table of how much VRAM each of the open source models require, especially at different
by abdullahkhalids 3y ago
How large models can this run?
Generally, is there a resource that has a table of how much VRAM each of the open source models require, especially at different quantization levels? I am pretty confused about these Mixture of Experts models like Mixtral 8x7B.