5 ms·

As you can see from the output, the model can simplify the shape, alter the geometry, put a texture on an object (projecting a texture like UV mapping) without

As you can see from the output, the model can simplify the shape, alter the geometry, put a texture on an object (projecting a texture like UV mapping) without being explicitly programmed to do so. It is inherently different from Photoshop filters that apply a fixed transformation on the pixels. The fundamental difference is that the model is able to abstract away from the pixel representation of the image and it can decorelate content and style. Also, it is not necessarily more computationally intensive in the generation / forward phase.

TheOtherHobbes 9y ago
It still looks like Poor Photoshop Art Imitation++ though. Outside of the developer community, no one cares how this works. They only care about whether or not they connect with the result. And a good historical hint is that no one uses those old Photoshop filters now, because they're uniformly cheesy and terrible. Artistically, this is similar. The Deep Dream output was an interesting novelty, for a while, but to most people this style transfer process is going to be Just Another Photo Filter Effect. Technically, it's not abstracting shape and texture in an intelligent way. It's applying a mechanical effect in a mechanical way with absolutely no insight into texture or spatial geometry. Even in abstract art, shapes mean something. They're not just a set of coordinates. This process doesn't understand that meaning - and generally, developers who think "art" means "I made an image with my computer" don't understand it either. Sometimes this process gets lucky and something passable falls out, but mostly the results are mediocre.
- reiinakano 9y ago
  Dunno about you, but when I first saw some of the results, I was amazed. Here are a few more examples: https://github.com/jcjohnson/fast-neural-style https://github.com/jcjohnson/fast-neural-style https://github.com/lengstrom/fast-style-transfer https://github.com/lengstrom/fast-style-transfer Admittedly, though, I haven't seen the Photoshop filters you speak of. Could you link to some of them that show these same effects?
  - notahacker 9y ago
    I think there's something of a human selection effect with the more impressive outputs they've opted to show; much as you can throw a far dumber algorithm at far less selective samples of Shakespeare and end up with the non-incoherent examples seeming wonderfully poetic Take the example of the woman's face and Munch's The Scream. Ask an art student to pain a woman in the style of The Scream and you'll probably get her with a simplified, harrowed expression, surrounded by swirly waves and skies. Run this through the algorithm and you'll get a photograph of a woman's face with brushwork superimposed over it including a choppy, vibrant orange forehead instead of a choppy, vibrant orange sky. It's not particularly visually interesting and is obviously the work of a dumb algorithm[1] On the other hand, I combined the woman's face with Picabia's Udnie and the results were very pleasing, especially the way the contrast of black and white and sharpened edges happened to emphasise the model's cheekbone structure and eyes, and it looks like something a human might draw and sell as an original semi-abstracted portrait (albeit something an art teacher setting the "draw her in the style of Picabia" assignment would probably still frown at and say they'd missed the point of the exercise in producing representative art and not really grasped Picabia's massing or shading either) [1]And I'm sure it's not the first time I've critiqued computer generated art based on combining Munch's the Scream because it treats the top of the image as vivid orange sky regardless of what it actually is...
- JacobiX 9y ago
  I mostly agree that the results are not always good. As someone who read the original style transfer paper, and implemented it, I disagree that it is applying a mechanical effect. Maybe I'm biased but the algorithm use a pre-trained CNN on object recognition. The CNN develops a representation of the image that is more and more abstract along the layers of the network. You can visualize the information in each layer by reconstructing the image only from the feature maps. And when you do this you will see that the network, learns the objects and textures of the image compared to its pixel values (pixels -> edges -> regions -> objects, etc).

TheOtherHobbes 9y ago

It still looks like Poor Photoshop Art Imitation++ though. Outside of the developer community, no one cares how this works. They only care about whether or not they connect with the result. And a good historical hint is that no one uses those old Photoshop filters now, because they're uniformly cheesy and terrible. Artistically, this is similar. The Deep Dream output was an interesting novelty, for a while, but to most people this style transfer process is going to be Just Another Photo Filter Effect. Technically, it's not abstracting shape and texture in an intelligent way. It's applying a mechanical effect in a mechanical way with absolutely no insight into texture or spatial geometry. Even in abstract art, shapes mean something. They're not just a set of coordinates. This process doesn't understand that meaning - and generally, developers who think "art" means "I made an image with my computer" don't understand it either. Sometimes this process gets lucky and something passable falls out, but mostly the results are mediocre.

reiinakano 9y ago
Dunno about you, but when I first saw some of the results, I was amazed. Here are a few more examples: https://github.com/jcjohnson/fast-neural-style https://github.com/jcjohnson/fast-neural-style https://github.com/lengstrom/fast-style-transfer https://github.com/lengstrom/fast-style-transfer Admittedly, though, I haven't seen the Photoshop filters you speak of. Could you link to some of them that show these same effects?
- notahacker 9y ago
  I think there's something of a human selection effect with the more impressive outputs they've opted to show; much as you can throw a far dumber algorithm at far less selective samples of Shakespeare and end up with the non-incoherent examples seeming wonderfully poetic Take the example of the woman's face and Munch's The Scream. Ask an art student to pain a woman in the style of The Scream and you'll probably get her with a simplified, harrowed expression, surrounded by swirly waves and skies. Run this through the algorithm and you'll get a photograph of a woman's face with brushwork superimposed over it including a choppy, vibrant orange forehead instead of a choppy, vibrant orange sky. It's not particularly visually interesting and is obviously the work of a dumb algorithm[1] On the other hand, I combined the woman's face with Picabia's Udnie and the results were very pleasing, especially the way the contrast of black and white and sharpened edges happened to emphasise the model's cheekbone structure and eyes, and it looks like something a human might draw and sell as an original semi-abstracted portrait (albeit something an art teacher setting the "draw her in the style of Picabia" assignment would probably still frown at and say they'd missed the point of the exercise in producing representative art and not really grasped Picabia's massing or shading either) [1]And I'm sure it's not the first time I've critiqued computer generated art based on combining Munch's the Scream because it treats the top of the image as vivid orange sky regardless of what it actually is...
JacobiX 9y ago
I mostly agree that the results are not always good. As someone who read the original style transfer paper, and implemented it, I disagree that it is applying a mechanical effect. Maybe I'm biased but the algorithm use a pre-trained CNN on object recognition. The CNN develops a representation of the image that is more and more abstract along the layers of the network. You can visualize the information in each layer by reconstructing the image only from the feature maps. And when you do this you will see that the network, learns the objects and textures of the image compared to its pixel values (pixels -> edges -> regions -> objects, etc).