3 ms·
I just tried some pre-trained models to classify images, using a CPU, it takes about 10s to 15s per image. I did not know it was so inefficient.
by jokoon 3y ago
I just tried some pre-trained models to classify images, using a CPU, it takes about 10s to 15s per image.
I did not know it was so inefficient.
- chaostheory 3y agoIsn’t the problem that you were relying more on the CPU rather than a decent GPU?
- codethief 3y agoWould you share some details? This indeed sounds very slow, so I suppose there should be some easy ways to speed things up. Just to give you an idea of what's possible: A couple years ago I worked on live object recognition & classification (using Python and Tensorflow) and got to about ~30 FPS on an Nvidia Jetson Nano (i.e. using the GPU) and still ~12 FPS on an average laptop (using only the CPU).
- jokoon 3y agohttps://replicate.com/pharmapsychotic/clip-interrogator https://replicate.com/pharmapsychotic/clip-interrogator using: cfg.apply_low_vram_defaults() interrogate_fast() I tried lighter models like vit32/laion400 and others etc all are very very slow to load or use (model list: https://github.com/mlfoundations/open_clip https://github.com/mlfoundations/open_clip) I'm desperately looking for something more modest and light.
- quickthrower2 3y agoI am trying to understand your point. Are you saying it could be more efficient on a CPU using a different language than Python? Or are you saying CPUs are slow, GPUs should be used? Or are you saying classifying images algorithms can be improved more?
- riotnrrd 3y agoThe neural network inference (and training) is all done in compiled C libraries called by Python. The inefficiencies you're seeing are because you're doing it on a CPU, probably in one thread, rather than on a GPU with tens of thousands of "threads."
- deleted 3y ago[deleted]