4 ms·
That's amazing, any chance of running it on my trusty GTX 1060 6gb, or that's not enough VRAM?
by patricktlo 4y ago
That's amazing, any chance of running it on my trusty GTX 1060 6gb, or that's not enough VRAM?
- yacine_ 4y agoFrom my mental math, you can run 4bit quant 7B, but the quality for that might not be what you're looking for.