5 ms·

For image resizing, wouldn’t you use the GPU itself (assuming the server has one)? Does Python have a library that works with the GPU for image processing? I h

For image resizing, wouldn’t you use the GPU itself (assuming the server has one)? Does Python have a library that works with the GPU for image processing? I have a Skylake Xeon server with an integrated GPU. Any way to use that with Python?

igordebatur 9y ago
There was a discussion about GPUs under the first article of the series, https://news.ycombinator.com/item?id=14712146 https://news.ycombinator.com/item?id=14712146 In terms of building a SaaS, CPUs seem more reliable
kozikow 9y ago
> Does Python have a library that works with the GPU for image processing? tensorflow. It's not just deep learning library and you can implement any tensor operations there. 99% image processing is just tensor operations.
- make3 9y ago
  except just loading tensorflow takes half an hour on some machines. pytorch is better in that regards. both are probably too hard to install though, cuda and cudnn are really unpleasant to install, unless you are using conda
  - smhx 9y ago
    you dont need to install CUDA or CuDNN to install pytorch. We package and ship all required dependencies with our binaries.
    - make3 9y ago
      he's talking about gpus. you do need to install cuda to get pytorch to work on gpus yes
      - smhx 9y ago
        I am the lead devel of pytorch. No you dont need to install CUDA to get pytorch to work on GPUs, we ship the relevant bits with our binaries. All you need is a working NVIDIA driver.
        make3 9y ago
        really? very cool, thanks a lot, I didn't know
        make3 9y ago
        sm for soumith?
- mozumder 9y ago
  Intriguing option but I'm dealing with the integrated GPU here. It looks like TensorFlow is for NVidia discrete GPUs. PythonOpenCL is probably the best option: https://mathema.tician.de/software/pyopencl/ https://mathema.tician.de/software/pyopencl/ One thing about integrated GPUs for server is that there's no transfer time, since the GPUs share memory with the CPU.
  - Marat_Dukhan 9y ago
    There is transfer time. You source image is in cacheable CPU memory. Integrated GPUs normally work with uncacheable memory allocated in a special region of system memory. Some GPUs can access cacheable memory too, but it is much slower (because it has to maintain coherency with CPU caches), and requires that you allocate such cacheable memory using special OpenCL driver calls, not your normal malloc. So, in practice, you would do a copy to GPU-optimized buffer (in shared with CPU, but uncacheable memory).
stephencanon 9y ago
There's a fairly common use case for resizing where you have a large image already in CPU memory, and want to create a thumbnail or otherwise downsize. Copying the large source image to GPU memory can be expensive enough that it's faster to simply do the resize on the CPU, since the data is already there.
- deleted 9y ago
  [deleted]
wongarsu 9y ago
For low-quality resizing algorithms I suspect a good SIMD algorithm will be faster than transferring the image to the GPU and getting the result back.

igordebatur 9y ago

There was a discussion about GPUs under the first article of the series, https://news.ycombinator.com/item?id=14712146 https://news.ycombinator.com/item?id=14712146 In terms of building a SaaS, CPUs seem more reliable

kozikow 9y ago

> Does Python have a library that works with the GPU for image processing? tensorflow. It's not just deep learning library and you can implement any tensor operations there. 99% image processing is just tensor operations.

make3 9y ago
except just loading tensorflow takes half an hour on some machines. pytorch is better in that regards. both are probably too hard to install though, cuda and cudnn are really unpleasant to install, unless you are using conda
- smhx 9y ago
  you dont need to install CUDA or CuDNN to install pytorch. We package and ship all required dependencies with our binaries.
  - make3 9y ago
    he's talking about gpus. you do need to install cuda to get pytorch to work on gpus yes
    - smhx 9y ago
      I am the lead devel of pytorch. No you dont need to install CUDA to get pytorch to work on GPUs, we ship the relevant bits with our binaries. All you need is a working NVIDIA driver.
      - make3 9y ago
        really? very cool, thanks a lot, I didn't know
      - make3 9y ago
        sm for soumith?
mozumder 9y ago
Intriguing option but I'm dealing with the integrated GPU here. It looks like TensorFlow is for NVidia discrete GPUs. PythonOpenCL is probably the best option: https://mathema.tician.de/software/pyopencl/ https://mathema.tician.de/software/pyopencl/ One thing about integrated GPUs for server is that there's no transfer time, since the GPUs share memory with the CPU.
- Marat_Dukhan 9y ago
  There is transfer time. You source image is in cacheable CPU memory. Integrated GPUs normally work with uncacheable memory allocated in a special region of system memory. Some GPUs can access cacheable memory too, but it is much slower (because it has to maintain coherency with CPU caches), and requires that you allocate such cacheable memory using special OpenCL driver calls, not your normal malloc. So, in practice, you would do a copy to GPU-optimized buffer (in shared with CPU, but uncacheable memory).

stephencanon 9y ago

There's a fairly common use case for resizing where you have a large image already in CPU memory, and want to create a thumbnail or otherwise downsize. Copying the large source image to GPU memory can be expensive enough that it's faster to simply do the resize on the CPU, since the data is already there.

deleted 9y ago
[deleted]

wongarsu 9y ago

For low-quality resizing algorithms I suspect a good SIMD algorithm will be faster than transferring the image to the GPU and getting the result back.