3 ms·
Why would scheduling be a thing in this case? I might be missing something here. With continuous batching, you don't wait for entire previous batch to finish.
by ankit219 9mo ago
Why would scheduling be a thing in this case? I might be missing something here.
With continuous batching, you don't wait for entire previous batch to finish. The request goes in as one finishes. Hence the wait time is negligible.