30 ms·
It runs very quickly on my RTX 5090 fwiw. Whole thing is loading entirely into vRAM with a ~130k context size (the max) fitting as well.
by wincy 2mo ago
It runs very quickly on my RTX 5090 fwiw. Whole thing is loading entirely into vRAM with a ~130k context size (the max) fitting as well.
- codazoda 2mo agoThat's a $5k 32GB card for anyone who doesn't know all these off the top of their heads (like myself).