4 ms·
Are you running inference on CPU or GPU?
by vvladymyrov 5y ago
Are you running inference on CPU or GPU?
- ramoz 5y agoCPU, GPU doesn't work out well in our case. There is the data transfer cost as well as memory constraint & the model blows up in memory for every inference call.
- ramoz 5y agox2 Gunicorn workers, MKL mapped to half physical cores for each... for some reason the model (tensorflow) performs better on half vs 1 worker