3 ms·
What % of time, for a an average session, do you think is app overhead vs waiting for tokens? And there's your answer for why it's not a priority.
by nomel 1mo ago
What % of time, for a an average session, do you think is app overhead vs waiting for tokens? And there's your answer for why it's not a priority.
- nikanj 1mo agoFrom OpenAIs perspective, resources on your computer are free and wasting them is inconsequential
- tmp10423288442 1mo agoKind of. Eventually it gets too slow even for the OpenAI engineers using it, and then they need to fix it.
- andai 1mo agoJon Blow's response to this take was, "yes, which is why you have to work even harder to hide latency", instead of adding more on top.
- kingstnap 1mo ago[dead]