You could use a one-hot encoding to represent the resizing operation that’s desired ([0, 0, 1] for horizontal shrink, [0, 1, 0] for vertical shrink, and [0, 1,

You could use a one-hot encoding to represent the resizing operation that’s desired ([0, 0, 1] for horizontal shrink, [0, 1, 0] for vertical shrink, and [0, 1, 1] for shrinking both, then use first component of 1 for expansion). The input to a CNN model could then be the input image and the one-hot encoded resize instruction. And the “label” could be the pixel map of the chosen seam-carving seam that produces the resize step. The CNN output will be a softmax pixel map, optimized to reproduce the seam-carving result. Then at runtime, you would just have to add a slight post-processing step that ensures the CNN’s predicted seam doesn’t violate any constraints for the intended output shape of the resize operation chosen. You could use the existing seam carving algorithm to generate the training data, but would need to be careful the training set covers a huge variety of image statistics to avoid overfitting resizing to a particular subset of image features (or conversely, you could try to learn “better” than seam carving for specialized resizing on subclasses of images, like specialized for faces, cityscapes, etc.)