3 ms·
input tokens are processed at 10-50 times the speed of output tokens since you can process then in batches and not one at a time like output tokens
by tokenmaxxinej 5mo ago
input tokens are processed at 10-50 times the speed of output tokens since you can process then in batches and not one at a time like output tokens