3 ms·
Here's a use case: I was training a neutral net, and wanted to do some preprocessing (similar to image resizing, but without an existing C function). Inputs are
by lostdog 5y ago
Here's a use case: I was training a neutral net, and wanted to do some preprocessing (similar to image resizing, but without an existing C function). Inputs are batched, so the preprocessing is trivially parallelizable. I tried to multithread it in python, and got no speedup at all.
That was a really sad moment, and I've never felt good about python since.
- isoprophlex 5y agoYeah, I had something similar. I wanted "as you wait for the GPU to churn through this batch, start reading the next batch from disk & preprocessing it on the CPU" Getting this to work turned out so ass backwards it made me sad Also I pity the fool who tries to connect a debugger to code using multiprocessing.Pool()...
- redeyed 5y ago> Also I pity the fool who tries to connect a debugger to code using multiprocessing.Pool() PyCharm works fine
- redeyed 5y agoYeah, had the exact problem. I solved it by using python threads and C function (cv2.resize)