> Would it be worth it if one has to consider that the data needs to be moved to the GPU and the result be retrieved Depends on your batch size. If the computa

> Would it be worth it if one has to consider that the data needs to be moved to the GPU and the result be retrieved Depends on your batch size. If the computation on the CPU is less than lets say 200 ms it's probably not worth it. Also consider that integrated GPUs don't have separate memory, I'm not sure, but they might not have a high cost of moving data to memory.