3 ms·
60+ GB VRAM, unless you quantize. If you quantize sure, but then it will not really be comparable to what others are using.
by ShinyLeftPad 2mo ago
60+ GB VRAM, unless you quantize. If you quantize sure, but then it will not really be comparable to what others are using.