3 ms·

Well, it's a good thing I didn't say "fundamentally unable to learn this"! I said that learning visual reasoning from video is probably not enough: if you clai

• dinfinity 2y ago
  > I said that learning visual reasoning from video is probably not enough Fair enough; you did indeed say that. > if you claim it is enough, you have to reconcile that with failures in Sora, Veo 2, etc. This is flawed reasoning, though. The current state of video generating AI and the completeness of the training set does not reliably prove that the network used to perform the generation is incapable of physical modeling and/or object permanence. Those things are ultimately (the modeling of) relations between past and present tokens, so the transformer architecture does fit. It might just be a matter of compute/network size (modeling four dimensional physical relations in high resolution is pretty hard, yo). If you look at the scaling results from the early Sora blogs, the natural increase of physical accuracy with more compute is visible: https://openai.com/index/video-generation-models-as-world-simulators/#scaling-transformers-for-video-generation https://openai.com/index/video-generation-models-as-world-si... It also might be a matter of fine-tuning training on (and optimizing for) four dimensional/physical accuracy rather than on "does this generated frame look like the actual frame?"
  • zveyaeyv3sfye 2y ago
    [flagged]