2 ms·
It does matter if you run your ML models in production. After we upgraded to 3.12 the average response time ~4-6ms decreased to stable sub 2ms. Latency decrease
by evolutionas 3y ago
It does matter if you run your ML models in production. After we upgraded to 3.12 the average response time ~4-6ms decreased to stable sub 2ms. Latency decrease for p95 and p99 were even more significant