3 ms·
The cloud AI providers have unacceptable variation in response time for things that need a predictable runtime. Even if they did offer a defined latency produc
by cameronh90 1y ago
The cloud AI providers have unacceptable variation in response time for things that need a predictable runtime.
Even if they did offer a defined latency product, you’re relying on a lot of infrastructure between your application and their GPU.
That’s not always tolerable.