6 ms·

How far are we from automating the job of animators with tech like what makes DeepFake videos?

yodon 5y ago
Animation is about acting - sometimes comedy, always emotions. Like acting it is hard to specify the desired end state and harder still to identify the decisions and actions required to achieve it. "She needs to look more nurturing" or "that ocean wave needs to look more aggressive (but not malicious)" are legitimate things a Director might say to an animator. Machine Learning can already make an image walk. The trick is making it walk in a way that the audience will connect with emotionally in that moment.
- blueblimp 5y ago
  Is this true of in-between animation (which is what's being discussed in the linked article)? I would think this doesn't require much creativity, since you're interpolating between keyframes, and might be a reasonable target for ML methods.
numpad0 5y ago
Decades, perhaps. I don’t think there have even been an ML model that generate vector drawings, only patchworks of existing bitmaps from training data.
SilverRed 5y ago
IMO AI is going to be like code autocomplete is today. A real animator will draw in keyframes and then the AI will try to interpolate correctly. The human then watches the interpolation and assesses the result. If anything looks off, they then go and draw in manual frames until the AI gets it right. Then later we may see the AI assist in drawing the individual frames so the artist draws an image and uses the shading AI to attempt to shade the image. Again, with manual work to correct where it messes up.
newfriend 5y ago
There are already some impressive models that can render animations at higher framerates, for example DAIN[0] and RIFE[1]. DAIN video: https://www.youtube.com/watch?v=q2i6FXVjNT0 https://www.youtube.com/watch?v=q2i6FXVjNT0 RIFE video: https://www.youtube.com/watch?v=lqtqmP46LaA https://www.youtube.com/watch?v=lqtqmP46LaA [0] https://github.com/baowenbo/DAIN https://github.com/baowenbo/DAIN [1] https://github.com/hzwer/arXiv2020-RIFE https://github.com/hzwer/arXiv2020-RIFE
- majewsky 5y ago
  And here's the obligatory rebuttal against such tweening models from an animator: https://www.youtube.com/watch?v=_KRb_qV9P4g https://www.youtube.com/watch?v=_KRb_qV9P4g
chansiky 5y ago
I forget where I saw it but there was a documentary or a short that showed a series on Netflix using AI to fill in-between frames, allowing animators to draw less frames. Of course it didn't seem to change the workload of the animators, they just improved the quality of each keyframe - so another case of "robots are taking our jobs, but not exactly".

Animation is about acting - sometimes comedy, always emotions. Like acting it is hard to specify the desired end state and harder still to identify the decisions and actions required to achieve it. "She needs to look more nurturing" or "that ocean wave needs to look more aggressive (but not malicious)" are legitimate things a Director might say to an animator. Machine Learning can already make an image walk. The trick is making it walk in a way that the audience will connect with emotionally in that moment.

blueblimp 5y ago
Is this true of in-between animation (which is what's being discussed in the linked article)? I would think this doesn't require much creativity, since you're interpolating between keyframes, and might be a reasonable target for ML methods.

numpad0 5y ago

Decades, perhaps. I don’t think there have even been an ML model that generate vector drawings, only patchworks of existing bitmaps from training data.

SilverRed 5y ago

IMO AI is going to be like code autocomplete is today. A real animator will draw in keyframes and then the AI will try to interpolate correctly. The human then watches the interpolation and assesses the result. If anything looks off, they then go and draw in manual frames until the AI gets it right. Then later we may see the AI assist in drawing the individual frames so the artist draws an image and uses the shading AI to attempt to shade the image. Again, with manual work to correct where it messes up.

newfriend 5y ago

There are already some impressive models that can render animations at higher framerates, for example DAIN[0] and RIFE[1]. DAIN video: https://www.youtube.com/watch?v=q2i6FXVjNT0 https://www.youtube.com/watch?v=q2i6FXVjNT0 RIFE video: https://www.youtube.com/watch?v=lqtqmP46LaA https://www.youtube.com/watch?v=lqtqmP46LaA [0] https://github.com/baowenbo/DAIN https://github.com/baowenbo/DAIN [1] https://github.com/hzwer/arXiv2020-RIFE https://github.com/hzwer/arXiv2020-RIFE

majewsky 5y ago
And here's the obligatory rebuttal against such tweening models from an animator: https://www.youtube.com/watch?v=_KRb_qV9P4g https://www.youtube.com/watch?v=_KRb_qV9P4g

chansiky 5y ago

I forget where I saw it but there was a documentary or a short that showed a series on Netflix using AI to fill in-between frames, allowing animators to draw less frames. Of course it didn't seem to change the workload of the animators, they just improved the quality of each keyframe - so another case of "robots are taking our jobs, but not exactly".