3 ms·
Have you tried batch inference on the GPU? In my experience it increases throughput by an order of magnitude.
by Jack000 7y ago
Have you tried batch inference on the GPU? In my experience it increases throughput by an order of magnitude.
loading thread…