2 ms·
I'd have been more surprised if there wasn't streaming. All the LLMs I have ever used, stream tokens in real-time. I'm talking about the time spent per token.
by LoganDark 3y ago
I'd have been more surprised if there wasn't streaming. All the LLMs I have ever used, stream tokens in real-time. I'm talking about the time spent per token.