3 ms·
Along with everything else. These parameters can make speculative decoding less accurate increasing the inference cost.
by charcircuit 3mo ago
Along with everything else. These parameters can make speculative decoding less accurate increasing the inference cost.