4 ms·
There is a misunderstanding here. It's not the algorithm that emphasizes textures over shapes, it's the dataset. Texture is a more informative feature than shap
by modeless 7y ago
There is a misunderstanding here. It's not the algorithm that emphasizes textures over shapes, it's the dataset. Texture is a more informative feature than shape in the datasets used.
"AI" can detect shapes just fine; simply use a different dataset and you'll get a different result with the same algorithm. I mean, just look at the classic MNIST: it's all shape, no texture, and neural nets work great.
- Majromax 7y ago> Texture is a more informative feature than shape in the datasets used. But the insight is that although both features are informative, classical training approaches result in networks that over-fit on texture information and ignore shape information. Table 2 of the discussed paper finds that joint training with "Stylized ImageNet" (IN images after a style transfer) and classic IN followed by a fine-tuning pass on IN results in improved classification accuracy over IN training alone.
- skgoa 7y agoWhich proves exactly what the top level comment claimed: the distribution of examples in the dataset was incorrect for the learning task.
- yarg 7y agoNot necessarily. Textures are high frequency low abstraction data forms, shapes are low frequency high abstraction data forms. The former are easier to learn for multiple reasons: The nature of higher frequency data means that you have more samples. (There are far more furry textures in a picture of a cat than there are toes, paws, legs etc.) Low abstraction data forms are also learnt in earlier layers of the network - more abstract forms are resolved through the understanding of how less abstract forms group together, making the low abstraction forms a prerequisite to more in depth learning.
- anbop 7y agoOne thing that’s interesting about textures is that humans have an incredible sensitivity to it by touch. You can detect the difference between a flat surface and a surface with angstrom-magnitude features.
- danharaj 7y agoThat sounds cool! Do you have a reference handy?
- ammar2 7y ago"The participants probed the surfaces with the index finger of their preferred hand in a designated direction (perpendicular to the grooves) for as long as they wished and at loads and speeds that they established themselves" "...it is observed that while the minimum pattern that could be distinguished from the unwrinkled reference surfaces had a wavelength of 760 nm, the amplitude of this pattern was only 13 nm. This shows unambiguously that the human finger, with its coarse fingerprint structure in the sub-millimeter range, is capable of dynamically detecting surface structures many orders of magnitude smaller and indicates that nanotechnology may well have a role to play in haptics and tactile perception." [1] Feeling Small: Exploring the Tactile Perception Limits https://www.nature.com/articles/srep02617 https://www.nature.com/articles/srep02617
- cameldrv 7y agoThere's some truth to this, but it's not the whole story. CNNs have limitations in terms of what they can represent and what they can learn. Counting is an example of this. If I create a class of monsters with 10 eyes, and another that is otherwise identical, but with 11 eyes, these will be difficult classes to tell apart. When it comes to "shape", part of the problem is definitional. In the MNIST example, the network will fairly easily learn features that are nodes, like where two lines intersect, but it has trouble with paths, like "an unbroken line that smoothly intersects itself." CNNs have trouble distinguishing things like a spiral from a set of concentric circles because locally they look the same.
- gjm11 7y ago> CNNs have trouble distinguishing things like a spiral from a set of concentric circles because locally they look the same. So does the human visual system; see e.g. https://en.wikipedia.org/wiki/Fraser_spiral_illusion https://en.wikipedia.org/wiki/Fraser_spiral_illusion .
- petters 7y agoIt seems to me that the number of eyes problem should be solvable. Just convolve with an eye detector and threshold appropriately. Then feed everything to a single neuron in the next layer and threshold at 10.5. No? The spiral vs. circles I agree with.
- mkl 7y agoSolvable with human guidance, yes, but would any existing machine learning system come up with a successful strategy like this by itself? Most deep dream style images suggest the networks don't even realise that dogs have two eyes.
- hammock 7y agoWhat would happen if we fed it SVGs instead of JPGs? (Shape files)
- joe_the_user 7y agoTexture is a more informative feature than shape in the datasets used. TL;DR; Logical reasoning isn't based on a "prepondernace of informative." Human-like image-recognition seems, to me, to quire both sort-of statistical reasoning, which depends on the preponderance of information and logical reasoning, which doesn't (not directly). That's seems sort-of true but "Informative" has a number of different implication. It seems pretty inevitable that a huge image dataset is going to carry the most data in the form of textures. The thing, a human being can (often) extract considerable meaning from an image of a stick figure or a black and white outline of a big cat, neither of which involves that much data. Effectively, human image recognition is able to both use texture-qualities as a rough guestimate for the source of an image but also look at the overall image structure and see "differences that make a difference". I mean, I can look at a puzzle piece and determine it probably goes in particular area but that's only one facility, "knowing" what an image "is", involves more.
- jonplackett 7y agoIf this is true does this mean AIs could be WAAAY better than they currently are if only we had some datasets with shape in as well as texture?
- sdenton4 7y agoThe key part is the 'bag of features' discussion. It's SUPER interesting work, demonstrating that classification works just as well if you cut up the image into square chunks and scramble them - the convolutional activations end up roughly similar, but all sense of shape is lost (as is any human recognition of the scrambled objects). I think MNIST is too simple to have much bearing on the problem. MNIST shapes are two-dimensional; actual object classification requires 3D shapes (and all possible projections thereof), which is a much harder problem. Since this is just a message board discussion, my wild conjecture would be that the model has no idea about the existence of the third dimension, or basic physical concepts like lighting and distance. And without that larger world model in which to situate things, texture is maybe the easiest thing to cling to.
- joe_the_user 7y agoThe "bag of features" work seems plausible but I recall from a reddit discussion that "just as well" might be stretching it. I recall it worked OK for a fairly small set but not for a larger set. Still, some references would be nice.
- JJMcJ 7y agoThe first big win for neural nets, character recognition - that's shapes also.
- pmontra 7y agoI never saw a plane made of lego bricks but I recognized it instantaneously when I saw it for the first time. This hints about a different understanding process in deep learning and human brains. This is one of the points of the article. Actually I don't think humans vision is based only on shapes or textures. There are many novel shapes of planes and we know that's a plane no matter what. We use understanding, with our knowledge of the world, not only the information coming from the eyes. I think it is much larger than what is encoded in a deep learning neural network.
- vannevar 7y agoYes, you have only to look at the images generated by GANs to see that it's not inherent in the algorithm: images generated by GANs are not just a patchwork of textures, they have coherent shapes. They 'needed' shape in order to fool the discriminator, which in turn developed a sensitivity to shape in order to identify generated images (which probably do rely on texture early on) as fakes.