3 ms·
I was just wishing I had 24gb rather than 16gb so I could run q4 comfortably. Unsloth q4 variants are 15.4gb-17.6gb so it seems at least well worth trying on a
by fwipsy 6mo ago
I was just wishing I had 24gb rather than 16gb so I could run q4 comfortably. Unsloth q4 variants are 15.4gb-17.6gb so it seems at least well worth trying on a 4090.