If only the backward pass is slow that is strange. Maybe it is an issue with the CUDA implementation. A 2x2 stride 2 pooling should be simple, but if it's using

If only the backward pass is slow that is strange. Maybe it is an issue with the CUDA implementation. A 2x2 stride 2 pooling should be simple, but if it's using a parametrized general max pooling kernel then the implementation would be more complex than a specialised one. How much slower is it than the forward pass?