3 ms·

I think THE problem in computer vision. And it's really an AI problem. May be video prediction. Basically output the next N frames, given an input sample. It's

I think THE problem in computer vision. And it's really an AI problem. May be video prediction. Basically output the next N frames, given an input sample. It's trivial to estimate the error as you just split the video in two equal pieces. And try to guess the second half. And it's so difficult because it's not mere rgb probability. It requires scene understanding. And the physical trajectories of the bodies in motion! I had the idea that you could take a really basic video game such as pong, snake, space invaders or tetris. And not only predict next frames. But also infer physical quantities from laws of motion. Being able to apply it to a bullet hell style shmup would very very interesting. As some projectiles use sine waves. And others follow rng random walk paths ;)

ksaj 7y ago
"Basically output the next N frames, given an input sample. It's trivial to estimate the error as you just split the video in two equal pieces. And try to guess the second half." I actually did an experiment with something very close to this in the 90's, using simple morphing. For example, a video of a scene panning, but it was actually still photos with morphing between them to create the animation as more of a smooth pan instead of the seizure-porn strobe effect of flipping pictures. It didn't work as well as I thought it would, but I think what came out of it was possibly a form of art. Everything looked almost normal, but there was a really frenetic halo that accompanied anything that moved on the screen - especially if it moved in any direction where parallax plays a part in understanding what went where. The wriggley halo artifacts looked cool, but maybe only for a rock video. There was none of the smoothness I expected or hoped for.