4 ms·

Usually, it is not. There have been some attempts to combine image and text data into a hybrid model but I’m not sure how widespread it is. Ex: http://cbonnett

Usually, it is not. There have been some attempts to combine image and text data into a hybrid model but I’m not sure how widespread it is. Ex: http://cbonnett.github.io/Insight.html http://cbonnett.github.io/Insight.html

jcims 6y ago
I’ve been thinking of enduring some exit data (lens focal length, aperture, gravity vector, focus distance, etc) directly into the image as a band of scale bars across the bottom to inject a reference frame into the image directly. Not sure if it will do anything, just curious if it would help.
- mxuribe 6y ago
  Is something like Steganography[0] what you had in mind? (I'll assume your intent is less about hiding data, and more about inserting metadata "inline" for convenience, right?) If so, that sounds pretty handy! [0] = https://en.wikipedia.org/wiki/Steganography https://en.wikipedia.org/wiki/Steganography
- nl 6y ago
  You can inject any data (in numeric form if you want) as a second data input into your neural network (assuming you are doing something custom). For example we do this to represent the position of specific sub-image parts we extract in an original image. > just curious if it would help Depends what you are trying to do. For example normal CNNs aren't rotation invariant[1] so if you know a gravity vector it can be useful to make your image upright. (Whilst CNN's aren't rotation invariant, it's common practice to augment training data by applying some rotation to the same image, so depending on how the CNN was training it may be fine) [1] https://stats.stackexchange.com/questions/239076/about-cnn-kernels-and-scale-rotation-invariance https://stats.stackexchange.com/questions/239076/about-cnn-k..., https://stackoverflow.com/questions/41069903/why-rotation-invariant-neural-networks-are-not-used-in-winners-of-the-popular-co https://stackoverflow.com/questions/41069903/why-rotation-in...

jcims 6y ago

I’ve been thinking of enduring some exit data (lens focal length, aperture, gravity vector, focus distance, etc) directly into the image as a band of scale bars across the bottom to inject a reference frame into the image directly. Not sure if it will do anything, just curious if it would help.

mxuribe 6y ago
Is something like Steganography[0] what you had in mind? (I'll assume your intent is less about hiding data, and more about inserting metadata "inline" for convenience, right?) If so, that sounds pretty handy! [0] = https://en.wikipedia.org/wiki/Steganography https://en.wikipedia.org/wiki/Steganography
nl 6y ago
You can inject any data (in numeric form if you want) as a second data input into your neural network (assuming you are doing something custom). For example we do this to represent the position of specific sub-image parts we extract in an original image. > just curious if it would help Depends what you are trying to do. For example normal CNNs aren't rotation invariant[1] so if you know a gravity vector it can be useful to make your image upright. (Whilst CNN's aren't rotation invariant, it's common practice to augment training data by applying some rotation to the same image, so depending on how the CNN was training it may be fine) [1] https://stats.stackexchange.com/questions/239076/about-cnn-kernels-and-scale-rotation-invariance https://stats.stackexchange.com/questions/239076/about-cnn-k..., https://stackoverflow.com/questions/41069903/why-rotation-invariant-neural-networks-are-not-used-in-winners-of-the-popular-co https://stackoverflow.com/questions/41069903/why-rotation-in...