4 ms·

I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar

I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar images differently depending on whether it is an image of an object it recognizes or not - when evaluating the image with an object other qualities of the object (that are not necessarily visible in the image) will be taken into account.

maldusiecle 10y ago
Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.
- posterboy 10y ago
  We listen to foreign music a lot and still find pleasure in the voices. Although I'm mostly speaking from my experience with English before I had learned it, which is yet rather close to German, so YMMV >No True Scotsman thinks that subject matter is irrelevant fixed that for you
  - maldusiecle 10y ago
    No idea what the No True Scotsman fallacy has to do with this; it involves dismissing an example, but no example was offered. You're welcome to offer one, if you'd like. We listen to foreign music and find pleasure in the voices because a voice itself is expressive, independent of language: a cry, a laugh, an imprecation. The same is true of light, shape, etc., but artists who work with these qualities independent of their reference to objects tend to choose media other than photography, for obvious reasons.
    - posterboy 10y ago
      I was just pattern matching.
- randcraw 10y ago
  You have to wonder, though. Is it impossible that there's a "music theory" for images/paintings/art that explains the mechanics of what makes them more compelling vs less compelling? I suspect there is, at least to some degree. Obviously images convey much more information than music, so any theory that doesn't encompass the semantics of the subject will miss most of the signal. But is there a theory for the presentation and composition of the subject? To some degree, I'm confident there is. Some of the methods used to debug the deep learning of images already do a fair job of showing the locus of focus in the image where the DNN found maximum information. I can see such a technique discovering many of the techniques used by artists and photographers to direct the observer's eye or juxtapose objects that conflict.
  - d12345m 10y ago
    Perhaps it's not quite analogous to music theory, but what you're describing in the first paragraph would be referred to as the formal elements of art or simply the elements of art. Analysis of these elements (form, line, space, color, and texture) is usually a part of the sort of art criticism you'd find in academic studio art, art history, or even just the New York Times art section. The visual design field has a similar, extended set of elements for describing the formal elements of a design piece. In both art and design, works are usually considered effective if they use the formal elements of art/design to support what you refer to as the semantics of the subject. That's a broad generalization, but you see it in practice a lot, so it seems like a fair thing to say. Academic art history is starting to feel the influence of machine learning and computer vision precisely because computers can be trained to recognize the formal elements of art and associate their use with movements and historical periods. There are way more detailed articles than this one, but this will get you started if you're interested in this sort of thing: https://www.technologyreview.com/s/537366/the-machine-vision-algorithm-beating-art-historians-at-their-own-game/ https://www.technologyreview.com/s/537366/the-machine-vision...
    - posterboy 10y ago
      I heard the term Gestalt Laws, from a German loan word, used to describe this. I don't the relation of this to your notion. http://www.dict.cc/?s=gestalt http://www.dict.cc/?s=gestalt https://en.wikipedia.org/wiki/Gestalt_psychology https://en.wikipedia.org/wiki/Gestalt_psychology
leblancfg 10y ago
I agree. Although one could imagine that concepts conveyed in a photograph could be extracted and abstracted as vectors -- just like word2vec and its successors. Of course, there is a long way to go before we hit "human understanding" parity, but I think ideas from [1], [2] and [3] could be extrapolated in doing just that. [1] Deep Visual-Semantic Alignments for Generating Image Descriptions - cs.stanford.edu/people/karpathy/cvpr2015.pdf [2] Deep Learning for Content-Based Image Retrieval - www.research.larc.smu.edu.sg/mlg/papers/MM14-fp336-hoi.pdf [3] Deep Learning for Content-Based Image Retrieval - www.cs.rutgers.edu/~elgammal/pub/MTA_2014_Saleh.pdf

maldusiecle 10y ago

Yeah, exactly. No good critic of photography thinks that subject matter is irrelevant, that you can understand pictures as if they were abstract compositions of light and color. You'd might as well try to read a poem in an unknown language. This algorithm might learn to identify certain cliches, but it'll never learn what makes a picture powerful.

posterboy 10y ago
We listen to foreign music a lot and still find pleasure in the voices. Although I'm mostly speaking from my experience with English before I had learned it, which is yet rather close to German, so YMMV >No True Scotsman thinks that subject matter is irrelevant fixed that for you
- maldusiecle 10y ago
  No idea what the No True Scotsman fallacy has to do with this; it involves dismissing an example, but no example was offered. You're welcome to offer one, if you'd like. We listen to foreign music and find pleasure in the voices because a voice itself is expressive, independent of language: a cry, a laugh, an imprecation. The same is true of light, shape, etc., but artists who work with these qualities independent of their reference to objects tend to choose media other than photography, for obvious reasons.
  - posterboy 10y ago
    I was just pattern matching.
randcraw 10y ago
You have to wonder, though. Is it impossible that there's a "music theory" for images/paintings/art that explains the mechanics of what makes them more compelling vs less compelling? I suspect there is, at least to some degree. Obviously images convey much more information than music, so any theory that doesn't encompass the semantics of the subject will miss most of the signal. But is there a theory for the presentation and composition of the subject? To some degree, I'm confident there is. Some of the methods used to debug the deep learning of images already do a fair job of showing the locus of focus in the image where the DNN found maximum information. I can see such a technique discovering many of the techniques used by artists and photographers to direct the observer's eye or juxtapose objects that conflict.
- d12345m 10y ago
  Perhaps it's not quite analogous to music theory, but what you're describing in the first paragraph would be referred to as the formal elements of art or simply the elements of art. Analysis of these elements (form, line, space, color, and texture) is usually a part of the sort of art criticism you'd find in academic studio art, art history, or even just the New York Times art section. The visual design field has a similar, extended set of elements for describing the formal elements of a design piece. In both art and design, works are usually considered effective if they use the formal elements of art/design to support what you refer to as the semantics of the subject. That's a broad generalization, but you see it in practice a lot, so it seems like a fair thing to say. Academic art history is starting to feel the influence of machine learning and computer vision precisely because computers can be trained to recognize the formal elements of art and associate their use with movements and historical periods. There are way more detailed articles than this one, but this will get you started if you're interested in this sort of thing: https://www.technologyreview.com/s/537366/the-machine-vision-algorithm-beating-art-historians-at-their-own-game/ https://www.technologyreview.com/s/537366/the-machine-vision...
  - posterboy 10y ago
    I heard the term Gestalt Laws, from a German loan word, used to describe this. I don't the relation of this to your notion. http://www.dict.cc/?s=gestalt http://www.dict.cc/?s=gestalt https://en.wikipedia.org/wiki/Gestalt_psychology https://en.wikipedia.org/wiki/Gestalt_psychology

leblancfg 10y ago

I agree. Although one could imagine that concepts conveyed in a photograph could be extracted and abstracted as vectors -- just like word2vec and its successors. Of course, there is a long way to go before we hit "human understanding" parity, but I think ideas from [1], [2] and [3] could be extrapolated in doing just that. [1] Deep Visual-Semantic Alignments for Generating Image Descriptions - cs.stanford.edu/people/karpathy/cvpr2015.pdf [2] Deep Learning for Content-Based Image Retrieval - www.research.larc.smu.edu.sg/mlg/papers/MM14-fp336-hoi.pdf [3] Deep Learning for Content-Based Image Retrieval - www.cs.rutgers.edu/~elgammal/pub/MTA_2014_Saleh.pdf