2 ms·
Speculative decoding was on in all of my tests, the gain came from moving the round's inputs off host memory.
by wowitsbase 2mo ago
Speculative decoding was on in all of my tests, the gain came from moving the round's inputs off host memory.
2 ms·