4 ms·

> I've seen animation created with diffusion models, that would have taken a traditional animator a year to animate by hand, now doable by a single person, with

by creata 4y ago

> I've seen animation created with diffusion models, that would have taken a traditional animator a year to animate by hand, now doable by a single person, within a week. Do you have a link? That's pretty hard to believe.

kyleyeats 4y ago
This was posted yesterday and I think it qualifies: https://www.reddit.com/r/StableDiffusion/comments/yjhkjv/tardigrade_dreams_a_stable_diffusion_animation/ https://www.reddit.com/r/StableDiffusion/comments/yjhkjv/tar...
anonylizard 4y ago
https://www.bilibili.com/video/BV1B14y1575G/ https://www.bilibili.com/video/BV1B14y1575G/ It uses diffusion to post-process a 3d video frame-by-frame. You can see the massive improvements in facial expressions, from bug-eye alien face to something quite pleasant. 3d animation is great at allow for complex movements and dynamic camera angles, but horrible at facial expressions. That's why Pixar had to go with the exaggerated proportions 'cartoons' look, to ensure the faces are large enough to convey emotions, even in 3d. 2d animation is superb at facial expressions, but can't do complex movements. Drawing a static face is easy, drawing a rotating character is very hard, drawing a rotating character, while the camera itself is rotating, is brain meltingly hard, so almost never done. Diffusion based workflows will have none of the downsides, and all of the upsides, compared to either approach.
- creata 4y ago
  Very pretty! > Diffusion based workflows will have none of the downsides, and all of the upsides, compared to either approach. But isn't this true in the same way as tracing over 3D has the "upsides" of both approaches?
  - anonylizard 4y ago
    Not exactly, tracing over the 3d is called 'rotoscoping', and has been used in animation for a few years now, basically hand-drawing animation over a 3d video. There are two problems with rotoscoping. 1. The cost, its not much cheaper than just drawing it by hand. 2. The effects don't compare to diffusion. If you pay close attention, you'll find that diffusion not only added facial expressions, it even added SHADING and crevasses to the dresses. That stuff is extremely difficult even for a human to do repeatedly. Indeed, diffusion is superior to 2d animation, even in raw quality. Because diffusion will soon be able to produce illustration level pictures for each individual frame. Whereas 2d animation had to rely on detail-compressed simple artstyles AND keyframe-inserts to cut down on costs. Soon every frame will be a keyframe, and it can execute artstyles previously impossible to animate.

by creata 4y ago

kyleyeats 4y ago

This was posted yesterday and I think it qualifies: https://www.reddit.com/r/StableDiffusion/comments/yjhkjv/tardigrade_dreams_a_stable_diffusion_animation/ https://www.reddit.com/r/StableDiffusion/comments/yjhkjv/tar...

anonylizard 4y ago

https://www.bilibili.com/video/BV1B14y1575G/ https://www.bilibili.com/video/BV1B14y1575G/ It uses diffusion to post-process a 3d video frame-by-frame. You can see the massive improvements in facial expressions, from bug-eye alien face to something quite pleasant. 3d animation is great at allow for complex movements and dynamic camera angles, but horrible at facial expressions. That's why Pixar had to go with the exaggerated proportions 'cartoons' look, to ensure the faces are large enough to convey emotions, even in 3d. 2d animation is superb at facial expressions, but can't do complex movements. Drawing a static face is easy, drawing a rotating character is very hard, drawing a rotating character, while the camera itself is rotating, is brain meltingly hard, so almost never done. Diffusion based workflows will have none of the downsides, and all of the upsides, compared to either approach.

creata 4y ago
Very pretty! > Diffusion based workflows will have none of the downsides, and all of the upsides, compared to either approach. But isn't this true in the same way as tracing over 3D has the "upsides" of both approaches?
- anonylizard 4y ago
  Not exactly, tracing over the 3d is called 'rotoscoping', and has been used in animation for a few years now, basically hand-drawing animation over a 3d video. There are two problems with rotoscoping. 1. The cost, its not much cheaper than just drawing it by hand. 2. The effects don't compare to diffusion. If you pay close attention, you'll find that diffusion not only added facial expressions, it even added SHADING and crevasses to the dresses. That stuff is extremely difficult even for a human to do repeatedly. Indeed, diffusion is superior to 2d animation, even in raw quality. Because diffusion will soon be able to produce illustration level pictures for each individual frame. Whereas 2d animation had to rely on detail-compressed simple artstyles AND keyframe-inserts to cut down on costs. Soon every frame will be a keyframe, and it can execute artstyles previously impossible to animate.