3 ms·
the latency is really killer for this kind of usecase.. any plans to figure out how to cut it down?
by swyx 4y ago
the latency is really killer for this kind of usecase.. any plans to figure out how to cut it down?
- rushingcreek 4y agothe answer should start generating almost immediately like with ChatGPT. right now our infrastructure is being hugged a bit, and we are scaling it up.