3 ms·
I wonder whether the memory and gpu requirements have factored in optimisations like speculative sampling for inference?
by jacksonzheng 3y ago
I wonder whether the memory and gpu requirements have factored in optimisations like speculative sampling for inference?