4 ms·

14 sec per image is excessive, is there either an implementation or algorithm that can handle do this in less than 500ms per image?

seam_carver 8y ago
It's 14 seconds to remove 200 seams on the 512-by-342 bench image, but I note potential optimizations in the notebook. Right now, the seam_carve function in scikit-image accepts an energy matrix, which it then uses to generate the accumulated energy matrix. Forward energy calculates the accumulated energy matrix directly, but I calculate the energy matrix backward from it to be able to use scikit's seam_carve function. If I write my own seam removal function or change scikit's, I think I could be able to make the speed comparable to backwards energy. Furthermore, the accumulated energy matrix calculation step in scikit uses a double for loop, which might be able to replaced with some np stuff for a speedup for both forward and backwards energy. I should probably raise this as an issue on GitHub... thanks!
- seam_carver 8y ago
  I actually realized I was doing a lot of unnecessary computation in my dynamic programming step. I removed it and from 14 s it's now 8 s.
  - seam_carver 8y ago
    After talking with some friends, basically I could get this to be a lot faster if I just wrote it in Cython. (the skimage.transform.seam_carve function is written in Cython). Also, I realize now that the double for loop in Cython is actually fast, so that means that the skimage implementation of dynamic programming is optimal. But I can still change it to accept the accumulated energy matrix!
WhiteNoiz3 8y ago
For anyone interested, Photoshop has had this feature for a long time - Content Aware Scaling. It's also much faster.
- seam_carver 8y ago
  My goal right now is to get as close as I can to Photoshop speed; I'm fairly certain that the source code for Photoshop is not available online...
  - seam_carver 8y ago
    Also I should just write this in Cython since I do a lot of unnecessary stuff when using numpy.

seam_carver 8y ago

It's 14 seconds to remove 200 seams on the 512-by-342 bench image, but I note potential optimizations in the notebook. Right now, the seam_carve function in scikit-image accepts an energy matrix, which it then uses to generate the accumulated energy matrix. Forward energy calculates the accumulated energy matrix directly, but I calculate the energy matrix backward from it to be able to use scikit's seam_carve function. If I write my own seam removal function or change scikit's, I think I could be able to make the speed comparable to backwards energy. Furthermore, the accumulated energy matrix calculation step in scikit uses a double for loop, which might be able to replaced with some np stuff for a speedup for both forward and backwards energy. I should probably raise this as an issue on GitHub... thanks!

seam_carver 8y ago
I actually realized I was doing a lot of unnecessary computation in my dynamic programming step. I removed it and from 14 s it's now 8 s.
- seam_carver 8y ago
  After talking with some friends, basically I could get this to be a lot faster if I just wrote it in Cython. (the skimage.transform.seam_carve function is written in Cython). Also, I realize now that the double for loop in Cython is actually fast, so that means that the skimage implementation of dynamic programming is optimal. But I can still change it to accept the accumulated energy matrix!

WhiteNoiz3 8y ago

For anyone interested, Photoshop has had this feature for a long time - Content Aware Scaling. It's also much faster.

seam_carver 8y ago
My goal right now is to get as close as I can to Photoshop speed; I'm fairly certain that the source code for Photoshop is not available online...
- seam_carver 8y ago
  Also I should just write this in Cython since I do a lot of unnecessary stuff when using numpy.