4 ms·
I know. That was my point. Throughput doesn't scale on CPU as well as it does on GPU.
by qeternity 1y ago
I know. That was my point.
Throughput doesn't scale on CPU as well as it does on GPU.
- p12tic 1y agoWe both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.