3 ms·
Latency means this still makes no sense to me. Perhaps some batch background processing job such as research or something but that's stretching.
by KoolKat23 8mo ago
Latency means this still makes no sense to me. Perhaps some batch background processing job such as research or something but that's stretching.
- mike_hearn 8mo agoI think the most providers all give high latency batch APIs significant discounts. A lot of AI workloads feel batch-oriented to me, or could be once they move beyond the prototype and testing phases. Chat will end up being a small fraction of load in the long term.
- KoolKat23 8mo agoThat would imply there's still capacity here on earth for this type of traffic.