3 ms·
Sure, just offload to system RAM, and don't use your system driver's fallback, but a specific implementation like MultiGPU. It won't speed it up, but using a q
by washadjeffmad 1y ago
Sure, just offload to system RAM, and don't use your system driver's fallback, but a specific implementation like MultiGPU.
It won't speed it up, but using a quantization that fits in VRAM will prevent the offload penalty.