3 ms·
Depends on the application. If you can process "offline" for an hour and then cache the results, CPU inference is fine. GPUs are expensive.
by dheera 3y ago
Depends on the application.
If you can process "offline" for an hour and then cache the results, CPU inference is fine.
GPUs are expensive.
- idontpost 3y ago> If you can process "offline" for an hour and then cache the results, CPU inference is fine. > GPUs are expensive. Depends on the GPU. I've found T4 GPUs to be cheaper than CPU compute on AWS when testing throughput per $ of spend.
- geuis 3y agoVery true. This is the approach I've been taking. My project isn't human-interactive and I'm fine with it taking a couple hours for each run to finish. The downside is that it makes development and testing super slow without the speedup you get from having local GPU power.
- deleted 3y ago[deleted]