13 ms·
Show HN: Paint yourself in the style of classic art, in your browser
- TekMol 9y agoCool. I'm getting some great results. Would be awesome if you could train your own model in the browser too. By the way: How does a NN learn from a single image? I understand how a NN can learn to classify images. But how can it learn a style from a single image?
- reiinakano 9y agoFast Neural Style Transfer works like this: Each style image is actually trained on a neural network outside of the browser (using Python on Tensorflow) for around 6+ hours with a GPU. The output is a neural network that can take any content image and outputs it in the style it was trained on. The browser does this: 1. Downloads the model (6.6MB~) for the particular style. 2. Does one pass through the content image and outputs the styled image. No training is done in the browser whatsoever.
- TekMol 9y agoYes, I am aware of that. But I would like to know how the training works. How does a NN learn the style of an image?
- reiinakano 9y agoAhh I see, these are some resources that should help you. The original Neural Style Transfer paper should help you understand the loss functions involved: https://arxiv.org/abs/1508.06576 https://arxiv.org/abs/1508.06576 The paper introducing Real-Time Style Transfer is basically the algorithm used here: https://arxiv.org/abs/1603.08155 https://arxiv.org/abs/1603.08155
- SamBam 9y agoI think you can consider one single image as a large corpus of data, in the same was that you could train a simple Markov model from a large paragraph of text. It doesn't matter to the model whether the original text was 500 words in one piece of text, or five 100-word texts. It's mostly looking at local regions and seeing what guesses it can make about the next region given one region. (If I understand correctly).
- deleted 9y ago[deleted]
- reiinakano 9y agoDemo website: https://reiinakano.github.io/fast-style-transfer-deeplearnjs https://reiinakano.github.io/fast-style-transfer-deeplearnjs Deeplearn.JS library doesn't work on mobile yet, sorry to folks on the go!
- deleted 9y ago[deleted]
- Beltiras 9y agoIs the aspect ratio important? I'm choosing a roughly square image and it only calculates half an image.
- reiinakano 9y agoAspect ratio shouldn't matter, the output should still be the same size as the Content image. Mind sharing the pic?
- Beltiras 9y agohttps://imgur.com/a/acanW https://imgur.com/a/acanW
- reiinakano 9y agoI was able to stylize it fine. Maybe refresh and try again with a smaller size?
- iaml 9y agoI'm getting 404 on js bundle so the page won't load.
- reiinakano 9y agoFixed now. Sorry!
- iamaffan 9y agoGreat one.
- sabeltann 9y agoUncaught ReferenceError: dialogPolyfill is not defined at HTMLDocument.<anonymous> ((index):22672) at (index):24
- reiinakano 9y agoThis doesn't happen when I visit the site on Chrome on Linux. Do you mind sharing your system + browser?
- sabeltann 9y agoSure! Running Chrome (Version 61.0.3163.100 (Official Build) (64-bit)) on windows 7. Let me know if you need anything else.
- davecardwell 9y agoSame with Chrome/63.0.3223.8 on Intel Mac OS X 10_11_6 It’s the call to dialogPolyfill.registerDialog(dialog);
- reiinakano 9y agoHi, thanks for the report, but I'm not really a JS developer and have no idea how to work around this. Would you happen to know if there are any fixes I should look into?
- reiinakano 9y agoOkay, looked a bit into this and this piece of code is only run if an unsupported browser is detected, which is one of three cases, Mobile or Safari or a non-WebGL enabled browser. I guess if you're not on mobile or Safari, then your browser doesn't support WebGL and you wouldn't be able to run it either. Sorry!
- davedx 9y agoWeirdly, something about the generated images (colour range?) breaks Twitter's profile photo update. I actually took screenshots of the original taken with my webcam and the deep learning style transferred version, the former worked for my Twitter profile photo, and the latter resulted in an error. I tried different resolutions too, and it always broke. Shame, I really like the effect!
- reiinakano 9y agoYou took screenshots and it still broke? That's.. very odd, haha. I will look into it and see if it outputs out-of-range colors. EDIT: My bad, there are indeed out of range colors! I will update here when fixed.
- reiinakano 9y agoUpdated! There should no longer be a problem with Twitter. The way I saved the styled photo was Right click > Save image in Chrome. Thanks for reporting!
- everdev 9y agoWhat size images can it generate? I've used other libraries that can't go past 800x600.
- reiinakano 9y agoTheoretically, for this algorithm, there is no size limit. Practically, if you select something too high, your browser will die for lack of memory or you will grow old waiting for it to finish. My crappy 4GB RAM laptop can only handle around 300 x 500. Try setting the size to maximum, and if your computer can handle it, I'd be glad to guide you in using bigger images.
- reacweb 9y agoError: Failed to link vertex and fragment shaders. I am on windows 7 using firefox 55.0.3
- reiinakano 9y agoDo you mind trying Chrome? Deeplearn.JS is a new library and there are bound to be weird bugs like this I have no hope of debugging. I believe the most supported version is desktop Chrome.
- fiatjaf 9y agoHow do I input an image? There's no option to do that on Firefox 58.
- reiinakano 9y ago"Upload from file" option not appearing?
- fiatjaf 9y agoNo controls at all were appearing. Now, from a different computer --- but still at Firefox 58 -- everything is working fine (maybe you've fixed it in the meantime, maybe not).
- code-is-code 9y agoLooks like it does not work on mobile chrome
- reiinakano 9y agoDeeplearnJS does not work on mobile at all unfortunately. It's a new library, they'll get there. :)
- code-is-code 9y agoThat makes sense. An error-message would have been nice.
- rsoto 9y agoHey, it looks very cool. However, it's hanging my browser. Haven't looked at the source, but maybe this should be implemented with web workers to increase perceived performance. Anyway, great work. Congrats!
- reiinakano 9y agoThings are currently quite inefficient as there are some non-optimized (non-WebGL) calls being made. DeeplearnJS is a new library but once optimized functions are available, I'll update the site. :)
- chias 9y agoThe demo site doesn't appear to work on my end -- it gets a 404 attempting to load "bundle.js"
- supermdguy 9y agoYeah, same for me. I added an issue: https://github.com/reiinakano/fast-style-transfer-deeplearnjs/issues/5 https://github.com/reiinakano/fast-style-transfer-deeplearnj....
- chias 9y agoIt appears to be working again (for me)
- supermdguy 9y agoYep, it's working now.
- etrevino 9y agoI don't even get that far. It just don't load anything.
- reiinakano 9y agoWorks now. Sorry guys. I tried to hotfix something into production and screwed things up. This is a lesson I will not forget... Lol.
- BugsJustFindMe 9y agoDespite being able to fetch https://raw.githubusercontent.com/reiinakano/fast-style-transfer-deeplearnjs/gh-pages/bundle.js https://raw.githubusercontent.com/reiinakano/fast-style-tran... I'm still getting 404s for https://reiinakano.github.io/fast-style-transfer-deeplearnjs/bundle.js https://reiinakano.github.io/fast-style-transfer-deeplearnjs... (but I can get https://reiinakano.github.io/fast-style-transfer-deeplearnjs/bundle.js?foo https://reiinakano.github.io/fast-style-transfer-deeplearnjs...) Maybe throw ?<random> on your linked urls? Github's cdn appears to be in terrible shape. ¯\_(ツ)_/¯
- deleted 9y ago[deleted]
- sergiotapia 9y agoDemo doesn't load anythin
- reiinakano 9y agoFixed now. Sorry!
- deleted 9y ago[deleted]
- earlybike 9y agotl;dr: Impressive Prisma clone with six filters but running all calculations locally in the browser. Compared to Prisma it feels slightly slower or as slow, one filter almost froze my MBP 4@2.7.
- reiinakano 9y agoIt is currently VERY slow since there are a lot of non-optimized (non-GPU) calls being made. Once those are implemented in deeplearn.JS, things should run much faster. In fact, one of the creators of deeplearn.JS says it might be feasible to do this realtime in a webcam (although not at 60FPS). This was a super quick PoC demo I hacked together in a week on a very new library. :)
- make3 9y agoyou should likely put a warning message for mobile users to let them know the website doesn't work on mobile
- samstave 9y agoMobile not supported - please post some screens of results..
- reiinakano 9y agoYou can see one result in the repository README! :)
- hnlmorg 9y agoI don't know if it's my machine or whether my expectations are too high, but every generated image looks like the same blurry version of the original but with a different colour pallet. I certainly don't get anything as beautiful as the output examples. This happens regardless of the source content I use (any of the examples or even my own uploaded files) nor any of the reference styles. User agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36 (Chromium running on ArchLinux AMD64)
- hnlmorg 9y agoLooks like I'm getting the same results with Firefox on the same host PC as well. Screenshot showing the same input values used as the one in your example: https://armyofcrabs.com/rand/Screenshot_20171002_170417.png https://armyofcrabs.com/rand/Screenshot_20171002_170417.png
- reiinakano 9y agoThese definitely look wrong. Unfortunately, it's super hard to debug since I can't reproduce. :( My only advice I guess is to try it on a different machine when you have one available. Sorry, if this is a problem with the library I used, I can't really do anything to fix it. :/
- hnlmorg 9y agoNot sure if this helps but I've tried the project you based this off[1] and that produces images correctly (or at least that look more like my expectations). (I say "based", I'm not sure just how much of their code - if any - you used. But you did compliment them earlier in this discussion) [1] https://tenso.rs/demos/fast-neural-style/ https://tenso.rs/demos/fast-neural-style/
- reiinakano 9y agoDidn't really use any of their code. I reimplemented their model in a new library called deeplearn.JS. If the issue is indeed with the library, there's little I can do. I will still try to find what the issue is but can't make promises. Sorry for the letdown!
- Belphemur 9y agoAm I the only one that get "the stuff of nightmares" kind of results ? Ubuntu 17.04 FF 57 & Chrome 59 with face and Udnie: https://screenshots.firefox.com/4bMiETLnl1yx8NOx/reiinakano.github.io https://screenshots.firefox.com/4bMiETLnl1yx8NOx/reiinakano....
- dabber 9y agoNope, same results on Ubuntu 16.04, FF 57 Edit: Looks really cool though! Maybe I can find a windows machine to give it a try on.
- whataretensors 9y agoI've seen similar results in my open source work when trying to deploy models using batch normalization. I wonder if the author is using this operation.
- oever 9y agoI get the same nightmare results on Linux in Firefox FF 59 and Chromium 58.
- reiinakano 9y agoTo everyone facing this issue, please leave a comment here with your browser and system specs: https://github.com/reiinakano/fast-style-transfer-deeplearnjs/issues/7 https://github.com/reiinakano/fast-style-transfer-deeplearnj...
- logane 9y agoCool work! We made this demo using the same style transfer network in TensorFire: https://tenso.rs/demos/fast-neural-style/ https://tenso.rs/demos/fast-neural-style/! Your size slider was a nice touch :)
- reiinakano 9y agoYou're lengstrom from Github! I must let you know your models are exactly what I used! I don't have a beefy GPU so I was forced to use what you already had, haha. For everyone reading, these guys coded up the neural network I used, all I did was port it to a different library. :) Eagerly waiting for the Tensorfire API!
- selljamhere 9y agoAny news on the fast-neural-style release date?
- gwbas1c 9y agoMight want to try some basic HTTP caching settings for the model. This way the browser doesn't need to re-download everything every time. This can probably be configured in the server instead of in the application.
- wrigby 9y agoI think most CDNs will add reasonable cache-control headers as well - IIRC Cloudflare does this with sane defaults out of the box.
- reiinakano 9y agoStupid question: Is CloudFlare free for CDN purposes? Right now I'm serving everything from Github Pages. I don't expect there to be much traffic after this initial HN spike dies down, so if it isn't free, it won't be worth it for me to pay for it.
- wrigby 9y agoYep[1]! I used it for my girlfriend's small business site (Wordpress) and was very happy with it. I think CSS, JS, and images are cached - HTML is not. 1: https://www.cloudflare.com/plans/ https://www.cloudflare.com/plans/
- laythea 9y agoI get "Error: Unable to create WebGLTexture." every time I try and run a sample. I got a 780ti, so its no graphics hardware problem...?
- TheRealPomax 9y agoyou probably want to file that on the issue tracker, not here.
- laythea 9y agoWhy not here? - if someone is going to put something "in my face" that does not work (for me), I do not feel it inappropriate to respond in the same forum.
- michrassena 9y agoWhen I see the style transfers, I'm reminded of the early days of Photoshop, and whatever else was a commonplace photo editing application of the mid-90s (like Fractal Painter). It was fun to play with the idea of oil painting, or watercolor, or to simulate paper textures. How is style transfer an improvement? Undoubtedly, it's more sophisticated and requires more computational resources. Can it be used for artistic purposes, or is it only skin deep like those old filters? It looks to me as if this is pure imitation, your images of a night with friends at the street cafe, or stacking bails of hay can look like a Van Gogh when you upload it to Instagram. Am I missing something?
- jordanwallwork 9y agoWhat do you mean? It is what it is. You can use it for artistic purposes if you can think of some creative way of using it, but if you're just uploading images and spitting out altered versions I don't suppose many people would consider that to be particularly artistic
- math0ne 9y agoI don't know if the un-repurposed output is artistic, but surely creating these algorithms is.
- themcaffee 9y agoSure, for example https://imgur.com/a/HUN9G https://imgur.com/a/HUN9G. Just by playing around with drawing different textures, I was able to get the above with the wave model. I just see it as another digital tool like you do, but with creative use I could see some pretty cool applications in modern art.
- h2rra 9y agoWow, that is actually super cool!
- michrassena 9y agoAfter seeing this, and thinking through what I wrote yesterday, I have a bit of a different take. I don't like the style transfers of famous art-piece, to user-provided image. A skin-deep transfer of the look of Van Gogh, or Katsushika Hokusai, or Renoir, or whatever doesn't make art (in my opinion). I realize these tools are just a proofs-of-concept, which is something I forgot yesterday. What I'd like to be able to do is style transfer between arbitrary images, whether it's my own work or that of others. That would open up the creative possibilities.
- mercurysmessage 9y agoThe chat program on Line, Rina, does this as well if you send a picture of a person to her.
- lerie82 9y agoI don't know if it's loading or what but it's just blank on my phone's chrome browser.
- reiinakano 9y agoDoesn't support mobile :) Please use desktop (Chrome suggested)
- wowtip 9y agoIs it using the same tech as android/ios app Deep Art Effects? https://www.deeparteffects.com/ https://www.deeparteffects.com/ Results look very similar
- reiinakano 9y agoYes, most likely. At most they probably tweaked things a little bit for some minor improvements, but the underlying idea and algorithm is very likely the same. Can't confirm this, though. :)