3 ms·
Who cares about a few ms more latency on an LLM API? Maybe for voice, but most other use-cases are quite latency insensitive.
by KeplerBoy 2mo ago
Who cares about a few ms more latency on an LLM API? Maybe for voice, but most other use-cases are quite latency insensitive.
- goalieca 2mo agoYeah, the customer support use case is already grinding me even worse than offshoring to India.