3 ms·
I think that's wrong: the whole point of GANs is that they're quite intelligent and good at faking outputs. I've seen interpolated/in-betweened videos (mostly b
by gwern 7y ago
I think that's wrong: the whole point of GANs is that they're quite intelligent and good at faking outputs. I've seen interpolated/in-betweened videos (mostly but not entirely live-action), and it looks realistic to me.
The reason I'm somewhat skeptical is that just because something looks realistic doesn't mean that it what is intended. It's a version of the 'zoom in, enhance, enhance' problem. It's like the _Hobbit_ problem: a GAN could perfectly well fake a 60FPS version of a 30FPS version of the _Hobbit_ such that you couldn't tell that it wasn't the actual 60FPS version that Peter Jackson shot... but the problem is that it's 60FPS and that just feels wrong for cinema. Animators, anime included, use the limitations of framerate and deliberate switches between animating 'on twos' etc, with reductions in framerates being deliberately done for action segments and sakuga and other reasons. An anime isn't simply a film which was unavoidably shot with a too-low framerate.
(This is less true of superresolution: in most cases, if an anime studio could have afforded to animate at a higher resolution originally, they would have; and you're not compromising any 'artistic vision' if you use a GAN to do a good upscaling job instead of a lousy bilinear upscale built into your video player.)
- HeWhoLurksLate 7y agoWhat's the reason that we shouldn't have 60 FPS in movies?
- taneq 7y agoWe're so used to 24fps movies that it's become a subconscious cue for identifying 'realistic' film. Higher frame rates like like video games because all of the high frame rate CGI we see is in games. (IMO this is just something we have to push through. I hate the low frame rate of movies.)
- ionionionioa 7y ago>interpolated/in-betweened That's the problem: no matter how smart your algorithm is, you cannot make animation look smooth by only adding frames. Not even human animators could do that. The framerate of animation is irrelevant. What matters is the number of drawings per second, not the number of frames. An intelligent system would interpolate between drawings, which would often require modifying or deleting frames from the source. I'm not some purist claiming that this is an evil technology. It just plain doesn't apply to animation, except for pans or the rare scene animated at a full 24 FPS.
- gwern 7y agoI'm not following. (If it doesn't apply at all, how is anyone doing it...?) Of course you can identify drawings per second, much the same way a monitor can display a 24FPS video at 120hz without needing to be an 'intelligent system': you increase or decrease the number of duplicates as necessary. You in-between pairs of different frames, replacing all the identical ones which are simply displaying the same drawing.