4 ms·
You measure how many tokens are output in a period of time and take the average?
by csjh 3y ago
You measure how many tokens are output in a period of time and take the average?
- atharv_jaju 3y agoOh my bad, I meant how to code it?
- csjh 3y agoPython pseudocode: ```py output = prompt("My favourite animal is ") # returns a generator start = time.time() tokens = 0 for token in output: tokens += 1 print(token) print(f"Outputted {tokens / (time.time() - start)} tokens/second") ```