4 ms·
It's a shame that Yann didn't explicitly say what the misunderstanding was or clear it up. I think he's claiming that the misunderstanding is that SORA underst
by fancyfredbot 3y ago
It's a shame that Yann didn't explicitly say what the misunderstanding was or clear it up.
I think he's claiming that the misunderstanding is that SORA understands the physical world.
He goes on to say that generating the next frame conditional on some action is much harder than generating an entire plausible video. This just doesn't make sense to me. Every frame in a plausible video clearly needs to be conditioned on the actions shown in previous frames.
The tweet is mostly incoherent and I'm left assuming he's upset about SORA and wants to say his ideas are/were better but hasn't managed to express why in a meaningful way.
- makomk 3y agoAs I understand it, diffusion-based video generation models simply are not casual in this way. They work by modifying the previous frames in the video to be consistent with future frames just as much as they do later frames to be consistent with earlier ones. That's why Yann LeCun can argue that they do not have to be able to generate plausible continuations of a real video, just generate some arbitrary sample from the space of plausible-looking videos, and that the latter does not imply the ability to do the former. It's also why it's not possible to just generate videos of arbitrary length and lots of VRAM is required to create even a relatively short clip.
- fancyfredbot 3y agoThank you. That would make sense. Perhaps Yann's target audience are supposed to know this already, but your explanation actually cleared up a misunderstanding for me.