3 ms·

Someday the field will figure out that not all images of interest are squares. That will be a great advance. I realize some people have hacked up personal branc

by natch 6y ago

Someday the field will figure out that not all images of interest are squares. That will be a great advance. I realize some people have hacked up personal branches of projects to support non-square rectangles but it really needs to become mainstream or else we’re going to stay stuck in this “AI square winter.”

p1esk 6y ago
I don't get it - is there anything preventing you using non-square images today?
- janhenr 6y ago
  No, in production rectangle images are reshaped and resized to square ones.
  - ebalit 6y ago
    If you want to be able to run inference in batches, you have to normalize to a given input size. It often means a square input size if you want to minimally distort both vertical and horizontal images. You can also use variable input shape in production if you need to. It will be harder to optimize, especially for throughput, but it's not impossible.
- natch 6y ago
  Just the massive inconvenience of having to track down and implement the changes necessary to add non-square image support on most platforms. So if you think that is a minor thing that can be dismissed as nothing, then no. Or one can always crop, or resize, both of which throw away information, or pad, for which the underlying behavior is undocumented. I just don’t think any of these are great options. Do you? If the answer is “you can always write your own” that is true, but it’s just underlining my point that the problem is not yet solved.
  - p1esk 6y ago
    support on most platforms What do you mean by "platforms"?
    - natch 6y ago
      Deep learning frameworks: Keras, Pytorch, Tensorflow, Core ML, etc.
      - p1esk 6y ago
        You might be confused. There's nothing in any of those platforms that dictate you use square images.
        natch 6y ago
        It's possible I am confused, yes. But I'm just going by what is in all the documentation and tutorials I have encountered. It may have been solved in a lab somewhere, but the solution hasn't made it out into code usable by mere mortals, as far as I can tell. You may be applying a very special meaning of the words "nothing" and "dictate" it's just that it's very well hidden how to do this. I'm not alone. Here are examples of other people struggling with non-square images and not succeeding: https://stats.stackexchange.com/questions/240690/non-square-images-for-image-classification https://stats.stackexchange.com/questions/240690/non-square-... https://github.com/tanakataiki/ssd_kerasV2/issues/10 https://github.com/tanakataiki/ssd_kerasV2/issues/10 https://github.com/allanzelener/YAD2K/issues/51 https://github.com/allanzelener/YAD2K/issues/51 https://github.com/eriklindernoren/PyTorch-YOLOv3/issues/277 https://github.com/eriklindernoren/PyTorch-YOLOv3/issues/277 https://stackoverflow.com/questions/49893741/tensorflow-cnn-for-non-square-image https://stackoverflow.com/questions/49893741/tensorflow-cnn-... https://github.com/ml-hongkong/keras-transfer-learning-for-oxford102/issues/3 https://github.com/ml-hongkong/keras-transfer-learning-for-o...
        p1esk 6y ago
        You are confusing platforms with individual models. A platform, such as Pytorch or Tensorflow, does not care what shape is your input. You design and train a model for the whatever inputs you want. On the other hand, if someone trained a specific model architecture (e.g. YOLOv3 or Resnet-50) on some dataset of square images, then yes, this particular pre-trained model will expect the same input size and shape as what it was trained on. Does it make sense? If you take a beginner level course on deep learning (e.g. Stanford CS231n or FastAI course) you will immediately realize there's nothing (in any sense of the word) that prevents you from using any input shape to train your model. If you don't want to learn the tools you use, you will need to find someone who will train a model on your images. If you're willing to pay for it you will find plenty of help. However, if you want to neither learn nor pay, then what exactly are you complaining about?

by natch 6y ago

p1esk 6y ago

I don't get it - is there anything preventing you using non-square images today?

janhenr 6y ago
No, in production rectangle images are reshaped and resized to square ones.
- ebalit 6y ago
  If you want to be able to run inference in batches, you have to normalize to a given input size. It often means a square input size if you want to minimally distort both vertical and horizontal images. You can also use variable input shape in production if you need to. It will be harder to optimize, especially for throughput, but it's not impossible.
natch 6y ago
Just the massive inconvenience of having to track down and implement the changes necessary to add non-square image support on most platforms. So if you think that is a minor thing that can be dismissed as nothing, then no. Or one can always crop, or resize, both of which throw away information, or pad, for which the underlying behavior is undocumented. I just don’t think any of these are great options. Do you? If the answer is “you can always write your own” that is true, but it’s just underlining my point that the problem is not yet solved.
- p1esk 6y ago
  support on most platforms What do you mean by "platforms"?
  - natch 6y ago
    Deep learning frameworks: Keras, Pytorch, Tensorflow, Core ML, etc.
    - p1esk 6y ago
      You might be confused. There's nothing in any of those platforms that dictate you use square images.
      - natch 6y ago
        It's possible I am confused, yes. But I'm just going by what is in all the documentation and tutorials I have encountered. It may have been solved in a lab somewhere, but the solution hasn't made it out into code usable by mere mortals, as far as I can tell. You may be applying a very special meaning of the words "nothing" and "dictate" it's just that it's very well hidden how to do this. I'm not alone. Here are examples of other people struggling with non-square images and not succeeding: https://stats.stackexchange.com/questions/240690/non-square-images-for-image-classification https://stats.stackexchange.com/questions/240690/non-square-... https://github.com/tanakataiki/ssd_kerasV2/issues/10 https://github.com/tanakataiki/ssd_kerasV2/issues/10 https://github.com/allanzelener/YAD2K/issues/51 https://github.com/allanzelener/YAD2K/issues/51 https://github.com/eriklindernoren/PyTorch-YOLOv3/issues/277 https://github.com/eriklindernoren/PyTorch-YOLOv3/issues/277 https://stackoverflow.com/questions/49893741/tensorflow-cnn-for-non-square-image https://stackoverflow.com/questions/49893741/tensorflow-cnn-... https://github.com/ml-hongkong/keras-transfer-learning-for-oxford102/issues/3 https://github.com/ml-hongkong/keras-transfer-learning-for-o...
        p1esk 6y ago
        You are confusing platforms with individual models. A platform, such as Pytorch or Tensorflow, does not care what shape is your input. You design and train a model for the whatever inputs you want. On the other hand, if someone trained a specific model architecture (e.g. YOLOv3 or Resnet-50) on some dataset of square images, then yes, this particular pre-trained model will expect the same input size and shape as what it was trained on. Does it make sense? If you take a beginner level course on deep learning (e.g. Stanford CS231n or FastAI course) you will immediately realize there's nothing (in any sense of the word) that prevents you from using any input shape to train your model. If you don't want to learn the tools you use, you will need to find someone who will train a model on your images. If you're willing to pay for it you will find plenty of help. However, if you want to neither learn nor pay, then what exactly are you complaining about?