4 ms·

Implementation of Neural Style Transfer in Python and Keras

62 points by fchollet 10y ago

jbmorgado 10y ago
Any idea on the algorithm that deepart.io uses compared to this one? The image from deepart.io looks nicer, although it could just be due to brute force (deepart.io example has 1000 interactions, while the one from this project only shows 50).
- andreyk 10y ago
  They say they use a variation on the neural art transfer algorithm: "We apply an algorithm developed by Leon Gatys, Alexander Ecker and Matthias Bethge. " https://deepart.io/page/about/ https://deepart.io/page/about/ https://arxiv.org/abs/1508.06576 https://arxiv.org/abs/1508.06576 There are various tweaks for how to do it and some hacks to make it faster, so they probably iterated to get better results.
- titu1994 10y ago
  Hi, I'm the developer of the script. The algorithm used by Deepart.io differs in two crucial ways : It uses Markov Random Field Regularization from the CNNMRF paper, as well as Image Analogy loss from the Image Analogies" paper. The output is far more precise, although it requires far more time to process the image, as well as requires a 1000 iterations. Of course on powerful GPU's this is not a big problem. However on a desktop gpu or laptop gpu for home use, it is simply not worth it to use several hours to develop a single image. On the plus side, there is a second script called INetwork.py which uses several improvements from a recent paper "Improving the Neural Algorithm of Artistic Style" which takes slightly more time, but produces good results in under 100 iterations and far less time than with MRF loss.
dswalter 10y ago
Excellent writeup. The post makes use of the OP's deep learning library Keras, which is utterly fantastic. If you want to learn more about deep learning, I'd recommend getting started with Keras on top of Theano or Tensorflow; the API is elegant and keeps pretty up-to-date on modern tweaks to structure/initializations, etc.
- jbmorgado 10y ago
  Also check MxNet. I don't know enough of DNN to check if it's as powerful as Keras to build custom nets, but it's faster training the network and - for me - easier to install with CUDA and CNN support and has the big plus of being usable in R besides Python, Julia and Scala.
rco8786 10y ago
Neat. What are some practical uses for this?
- zardo 10y ago
  A good example from SIGGRAPH this year: https://youtu.be/urf-AAIwNYk?t=176 https://youtu.be/urf-AAIwNYk?t=176

62 points by fchollet 10y ago

Any idea on the algorithm that deepart.io uses compared to this one? The image from deepart.io looks nicer, although it could just be due to brute force (deepart.io example has 1000 interactions, while the one from this project only shows 50).

andreyk 10y ago
They say they use a variation on the neural art transfer algorithm: "We apply an algorithm developed by Leon Gatys, Alexander Ecker and Matthias Bethge. " https://deepart.io/page/about/ https://deepart.io/page/about/ https://arxiv.org/abs/1508.06576 https://arxiv.org/abs/1508.06576 There are various tweaks for how to do it and some hacks to make it faster, so they probably iterated to get better results.
titu1994 10y ago
Hi, I'm the developer of the script. The algorithm used by Deepart.io differs in two crucial ways : It uses Markov Random Field Regularization from the CNNMRF paper, as well as Image Analogy loss from the Image Analogies" paper. The output is far more precise, although it requires far more time to process the image, as well as requires a 1000 iterations. Of course on powerful GPU's this is not a big problem. However on a desktop gpu or laptop gpu for home use, it is simply not worth it to use several hours to develop a single image. On the plus side, there is a second script called INetwork.py which uses several improvements from a recent paper "Improving the Neural Algorithm of Artistic Style" which takes slightly more time, but produces good results in under 100 iterations and far less time than with MRF loss.

dswalter 10y ago

Excellent writeup. The post makes use of the OP's deep learning library Keras, which is utterly fantastic. If you want to learn more about deep learning, I'd recommend getting started with Keras on top of Theano or Tensorflow; the API is elegant and keeps pretty up-to-date on modern tweaks to structure/initializations, etc.

jbmorgado 10y ago
Also check MxNet. I don't know enough of DNN to check if it's as powerful as Keras to build custom nets, but it's faster training the network and - for me - easier to install with CUDA and CNN support and has the big plus of being usable in R besides Python, Julia and Scala.

rco8786 10y ago

Neat. What are some practical uses for this?

zardo 10y ago
A good example from SIGGRAPH this year: https://youtu.be/urf-AAIwNYk?t=176 https://youtu.be/urf-AAIwNYk?t=176