3 ms·
I don't think that's true. Latency is critical when the limiting factor is that you know what you want to do but in order to do it, you need to issue many reque
by johnfn 3y ago
I don't think that's true. Latency is critical when the limiting factor is that you know what you want to do but in order to do it, you need to issue many requests all at once. With GPT4 I can issue requests at a high enough level that the limiting factor is actually me thinking of what the next request will be.