4 ms·

Any idea why it’s so slow? the entire model should fit in the vram of one card.

• graceful6800 2mo ago
  The NVFP4 quant is completely broken, so I'm not shocked that other quants aren't fully there yet. Give it some time to cook.