3 ms·
If you're talking inference speed, it's probably just because simple words are a single token, but complex words or code are much more tokens so it'll output th
by Tiberium 3y ago
If you're talking inference speed, it's probably just because simple words are a single token, but complex words or code are much more tokens so it'll output them "slower" (while in reality each token will take the same time)
- explaininjs 3y agoNo, there are significant differences in the per-token time that seem directly correlated with “perplexity” (not necessarily in the exact definition, but so to speak).
- explaininjs 3y agoWho the heck downvoted this. I’ve literally collected this data myself from my production AI applications. But it doesn’t gel with people’s preconceived notions so they downvote ?