4 ms·
and which is why the speed up is proportional to context length so starting near parity then, theoretically, see 100x at 100k tokens
by jimmySixDOF 3y ago
and which is why the speed up is proportional to context length so starting near parity then, theoretically, see 100x at 100k tokens