10 ms·

I was messing around with img2img and I created this monstrosity

146 points by sanroot99 4y ago

pushedx 4y ago
The random face style each frame is oddly captivating
- panzerklein 4y ago
  It's like a scramble suit from Scanner Darkly
hericium 4y ago
Those morphing faces gave me a disturbing flashback of looking into a mirror after LSD macrodose.
- testernews 4y ago
  A scanner darkly
  - aliqot 4y ago
    Waking Life
  - Maken 4y ago
    Don't compare. This looks 100 time better than that movie.
- smitty1110 4y ago
  The thing that really triggered an Uncanny Valley reaction for me was when animal faces appeared on the back of his head in some sections. Very unsettling.
- 5560675260 4y ago
  > macrodose Is that a very large dose or just a regular one?
mynegation 4y ago
In the next five years all high cost productions are going to be replaced by a neural style transfer from the child scribbles. I am only half-joking.
- ryandrake 4y ago
  The next five years is going to be a wild ride for all of visual media. Studios need to buckle up.
- idiocrat 4y ago
  It will be a very boring over-saturated world, where everything is possible and everything is superficial.
  - Sabinus 4y ago
    Check out a lot of children's shows nowadays, just soulless 3D animation.
    - dorkwood 4y ago
      I feel like this is going to result in a generation of adults with absolutely awful taste. Imagine if your favourite childhood show was, like, a poorly animated spiderman-driving-a-car video on YouTube.
      - Barrin92 4y ago
        every generation of adults has absolutely awful taste in general. There's a reason the longest running shows on TV are telenovelas. Or my personal favorite, the readers vs board modern library top 100 books (https://i.warosu.org/data/lit/img/0048/55/1399297062273.png https://i.warosu.org/data/lit/img/0048/55/1399297062273.png) People who grew up and want more out of culture will do what they always have done, which is look for alternative media.
      - viraptor 4y ago
        A lot of my childhood shows were poorly animated (https://g.denik.cz/35/bd/jc-rumcajs0617_denik-320-16x9@2x.jpg https://g.denik.cz/35/bd/jc-rumcajs0617_denik-320-16x9@2x.jp...) or shot in stop motion with really simple elements. (http://fototeka.fn.org.pl/public/cache3/1-F-3092-5-1291x629.jpg http://fototeka.fn.org.pl/public/cache3/1-F-3092-5-1291x629....) Everyone who watched those is doing fine. Kids shows don't need to be art pieces - they're simple stories.
superchroma 4y ago
Not even close to right or stable, falls apart in the tighter shot on geralt when it no longer recognizes him as human, and AI of course has no idea about 3D context so can never do cell shading in the way it is being trained. This was honestly a great waste of compute power. To the downvoters, do what you want, but it doesn't change the apparent reality of what we're seeing. AI image generation is an exciting and growing technology but this isn't a remarkable or successful application of it in any measurable dimension. The system lacks temporal and spatial context and that is going to be a hard blocker for this application.
- charcircuit 4y ago
  I agree. Some of these frames look live action instead of anime.
  - sanroot99 4y ago
    Yeh it's seems more cell shaded then a anime
  - zaroth 4y ago
    They should have included the input frames side by side - it’s source images is live action video frames.
- hipshaker 4y ago
  True, wonkiness level is high. Still impressive compared to not having this emerging tech a year ago. It will get better of course. But wate of computer power? No, not at all. You are missing the novel creative aspect of this. This level of wonkiness is absolutely perfect for music video producers, and I am betting it will not take long before we see videos in this “style”. Some time ago there was a short trend of data moshing videos. They were certainly trippy and stood out, albeit short lived. Digital artefacts as a way of creative expression can work great, and will too in this case.
mrtree 4y ago
knrz 4y ago
The redditor was using img2img to do style transfer frame by frame, which is why we have the jumping faces all the time (“instability”). This isn’t a limitation of neural nets, as early as 2018 we’ve had stable style transfer for videos, see https://medium.com/element-ai-research-lab/stabilizing-neural-style-transfer-for-video-62675e203e42 https://medium.com/element-ai-research-lab/stabilizing-neura...
socialismisok 4y ago
Ghibli style is a real stretch. Even discounting "style" means more than just the colors on the frame (what shot compositions, what's in frame, where does the camera linger, etc) the redditor didn't really succeed in capturing even one frame that had the visual style of a Ghibli frame. Praying stuff like this feels... Not crass, exactly, but like... Unrefined and ersatz in a way. It's like buying a mall katana and displaying it like it's something of cultural value.
- sanroot99 4y ago
  I agree ,but it's a fun experiment to take image generator to it's limit
6510 4y ago
The future will need some bug fixes.
dirtyid 4y ago
If I didn't know context, looks almost stylistically intentional. Crazy to think a few years ago this would have taken untold tens of thousands of man hours to plan and animate, would have probably gone viral and won a bunch of awards. Now it's just an interesting accident from fucking around.
- forgotpwd16 4y ago
  >would have taken untold tens of thousands of man hours to plan and animate The "tens of thousands" is an overestimation for 1min. And my guess is it took substantial time to plan and animate the original Witcher sequence as well. Also modern anime utilize similar conversions (such as sourcing photographs) and 3d software.
davidkuennen 4y ago
Crazy. Just imagine what you could do with it. You could basically just do some amateur scenes in your garden and have SD make an anime masterpiece of it.
jesperlang 4y ago
Very dream like, trippy results! Reminds me alot of Waking Life movie
mellosouls 4y ago
Original: https://youtube.com/watch?v=biZwR6CzVzs https://youtube.com/watch?v=biZwR6CzVzs
aaron695 4y ago

146 points by sanroot99 4y ago

pushedx 4y ago

The random face style each frame is oddly captivating

panzerklein 4y ago
It's like a scramble suit from Scanner Darkly

hericium 4y ago

Those morphing faces gave me a disturbing flashback of looking into a mirror after LSD macrodose.

testernews 4y ago
A scanner darkly
- aliqot 4y ago
  Waking Life
- Maken 4y ago
  Don't compare. This looks 100 time better than that movie.
smitty1110 4y ago
The thing that really triggered an Uncanny Valley reaction for me was when animal faces appeared on the back of his head in some sections. Very unsettling.
5560675260 4y ago
> macrodose Is that a very large dose or just a regular one?

mynegation 4y ago

In the next five years all high cost productions are going to be replaced by a neural style transfer from the child scribbles. I am only half-joking.

ryandrake 4y ago
The next five years is going to be a wild ride for all of visual media. Studios need to buckle up.
idiocrat 4y ago
It will be a very boring over-saturated world, where everything is possible and everything is superficial.
- Sabinus 4y ago
  Check out a lot of children's shows nowadays, just soulless 3D animation.
  - dorkwood 4y ago
    I feel like this is going to result in a generation of adults with absolutely awful taste. Imagine if your favourite childhood show was, like, a poorly animated spiderman-driving-a-car video on YouTube.
    - Barrin92 4y ago
      every generation of adults has absolutely awful taste in general. There's a reason the longest running shows on TV are telenovelas. Or my personal favorite, the readers vs board modern library top 100 books (https://i.warosu.org/data/lit/img/0048/55/1399297062273.png https://i.warosu.org/data/lit/img/0048/55/1399297062273.png) People who grew up and want more out of culture will do what they always have done, which is look for alternative media.
    - viraptor 4y ago
      A lot of my childhood shows were poorly animated (https://g.denik.cz/35/bd/jc-rumcajs0617_denik-320-16x9@2x.jpg https://g.denik.cz/35/bd/jc-rumcajs0617_denik-320-16x9@2x.jp...) or shot in stop motion with really simple elements. (http://fototeka.fn.org.pl/public/cache3/1-F-3092-5-1291x629.jpg http://fototeka.fn.org.pl/public/cache3/1-F-3092-5-1291x629....) Everyone who watched those is doing fine. Kids shows don't need to be art pieces - they're simple stories.

superchroma 4y ago

Not even close to right or stable, falls apart in the tighter shot on geralt when it no longer recognizes him as human, and AI of course has no idea about 3D context so can never do cell shading in the way it is being trained. This was honestly a great waste of compute power. To the downvoters, do what you want, but it doesn't change the apparent reality of what we're seeing. AI image generation is an exciting and growing technology but this isn't a remarkable or successful application of it in any measurable dimension. The system lacks temporal and spatial context and that is going to be a hard blocker for this application.

charcircuit 4y ago
I agree. Some of these frames look live action instead of anime.
- sanroot99 4y ago
  Yeh it's seems more cell shaded then a anime
- zaroth 4y ago
  They should have included the input frames side by side - it’s source images is live action video frames.
hipshaker 4y ago
True, wonkiness level is high. Still impressive compared to not having this emerging tech a year ago. It will get better of course. But wate of computer power? No, not at all. You are missing the novel creative aspect of this. This level of wonkiness is absolutely perfect for music video producers, and I am betting it will not take long before we see videos in this “style”. Some time ago there was a short trend of data moshing videos. They were certainly trippy and stood out, albeit short lived. Digital artefacts as a way of creative expression can work great, and will too in this case.

mrtree 4y ago

knrz 4y ago

The redditor was using img2img to do style transfer frame by frame, which is why we have the jumping faces all the time (“instability”). This isn’t a limitation of neural nets, as early as 2018 we’ve had stable style transfer for videos, see https://medium.com/element-ai-research-lab/stabilizing-neural-style-transfer-for-video-62675e203e42 https://medium.com/element-ai-research-lab/stabilizing-neura...

socialismisok 4y ago

Ghibli style is a real stretch. Even discounting "style" means more than just the colors on the frame (what shot compositions, what's in frame, where does the camera linger, etc) the redditor didn't really succeed in capturing even one frame that had the visual style of a Ghibli frame. Praying stuff like this feels... Not crass, exactly, but like... Unrefined and ersatz in a way. It's like buying a mall katana and displaying it like it's something of cultural value.

sanroot99 4y ago
I agree ,but it's a fun experiment to take image generator to it's limit

6510 4y ago

The future will need some bug fixes.

dirtyid 4y ago

If I didn't know context, looks almost stylistically intentional. Crazy to think a few years ago this would have taken untold tens of thousands of man hours to plan and animate, would have probably gone viral and won a bunch of awards. Now it's just an interesting accident from fucking around.

forgotpwd16 4y ago
>would have taken untold tens of thousands of man hours to plan and animate The "tens of thousands" is an overestimation for 1min. And my guess is it took substantial time to plan and animate the original Witcher sequence as well. Also modern anime utilize similar conversions (such as sourcing photographs) and 3d software.

davidkuennen 4y ago

Crazy. Just imagine what you could do with it. You could basically just do some amateur scenes in your garden and have SD make an anime masterpiece of it.

jesperlang 4y ago

Very dream like, trippy results! Reminds me alot of Waking Life movie

mellosouls 4y ago

Original: https://youtube.com/watch?v=biZwR6CzVzs https://youtube.com/watch?v=biZwR6CzVzs

aaron695 4y ago