3 ms·

It doesn't understand physics. It just computes next frame based on current one and what it learned before, it's a plausible continuation. In the same way, Ch

by XCSme 3y ago

It doesn't understand physics. It just computes next frame based on current one and what it learned before, it's a plausible continuation. In the same way, ChatGPT struggles with math without code interpreter, Sora won't have accurate physics without a physics engine and rendering 3d objects. Now it's just a "what is the next frame of this 2D image" model plus some textual context.

yberreby 3y ago
> It just computes next frame based on current one and what it learned before, it's a plausible continuation. ... > Now it's just a "what is the next frame of this 2D image" model plus some textual context. This is incorrect. Sora is not an autoregressive model like GPT, but a diffusion transformer. From the technical report[1], it is clear that it predicts the entire sequence of spatiotemporal patches at once. [1]: https://openai.com/research/video-generation-models-as-world-simulators https://openai.com/research/video-generation-models-as-world...
- XCSme 3y ago
  Good link. But, even there it says: > Sora currently exhibits numerous limitations as a simulator. For example, it does not accurately model the physics of many basic interactions, like glass shattering. Other interactions, like eating food, do not always yield correct changes in object states Regardless whether all the frames are generated at once, or one by one, you can see in their examples it's still just pixel based. See the first example with the dog with blue hat, the woman has a blue thing suddenly spawn into her hand because her hand went over another blue area of the image.
  - yberreby 3y ago
    I'm not denying that there are obvious limitations. However, attributing them to being "pixel-based" seems misguided. First off, the model acts in latent space, not directly on pixels. Secondly, there is no fundamental limitation here. The model has already acquired limited-yet-impressive ability to understand movement, texture, social behavior, etc., just from watching videos. I learned to understand reality by interpreting photons and various sensory inputs. Does that make my model of reality fundamentally flawed? In the sense that I only have a partial intuitive understanding of it, yes. But I don't need to know Maxwell's equations to get a sense of what happens when I open the blinds or turn on my phone. I think many of the limitations we are seeing here - poor glass physics, flawed object permanence - will be overcome given enough training data and compute. We will most likely need to incorporate exploration, but we can get really far with astute observation.
    - XCSme 3y ago
      This is an excellent comparison and I agree with you. Unfortunately we are flawed. We do know how physics work intuitively and can somewhat predict them, but not perfectly. We can imagine how a ball will move, but the image is blurry and trajectory only partially correct. This is why we invented math and physics studies, to be able to accurately calculate, predict and reproduce those events. We are far off from creating something as efficient as the human brain. It will take insane amounts of compute power to simply match our basic innacurate brains, imagine how much will be needed to create something that is factually accurate.
      - yberreby 3y ago
        Indeed. But a point that is often omitted from comparisons with organic brains is how much "compute equivalent" we spent through evolution. The brain is not a blank slate; it has clear prior structure that is genetically encoded. You can see this as a form of pretraining through a RL process wherein reward ~= surviving and procreating. If you see things this way, data-efficiency comparisons are more appropriate in the context of learning a new task or piece of information, and foundation models tend to do this quite well. Additionally, most of the energy cost comes from pretraining, but once we have the resulting weights, downstream fine-tuning or inference are comparatively quite cheap. So even if the energy cost is high, it may be worth it if we get powerful generalist models that we can specialize in many different ways. > This is why we invented math and physics studies, to be able to accurately calculate, predict and reproduce those events. We won't do away without those, but an intuitive understanding of the world can go a long way towards knowing when and how to use precise quantitative methods.
    - sydd 3y ago
      Actually your comment gives me hope that we will never have AI singularity, since how the brain works is flawed, and were trying to copy it. Heck a super AI might not even be possible, what if we're peak intelligence with our millions of years of evolution? Just adding compute speed will not help much -- say the goal of an intelligence is to win a war. If you're tasked with it then it doesn't matter if you have a month or a decade (assume that time is.frozen while you do your research), its a too complex problem and simply cannot be solved, and the same goes for an AI. Or it will be like with chess solvers, machines will be more intelligent than us simply because they can load much more context to solve a problem than us in their "working memory"
famouswaffles 3y ago
GPT-4 doesn't "struggle with math". It does fine. Most humans aren't any better. Sora is not autoregressive anyway but there's nothing "just" and next frame/token prediction.
- 8n4vidtmkvmk 3y ago
  It absolutely struggles with math. It's not solving anything. It sometimes gets the answer right only because it's seen the question before. It's rote memorization at best.
  - famouswaffles 3y ago
    No it doesn't. I know because I've actually used the thing and you clearly haven't. And if Terence Tao finds some use for GPT-4 as well as Khan Academy employing it as a Math tutor then I don't think I have some wild opinion either. Now Math isn't just Arithmetic but do you know easy it is to go out of training for say Arithmetic ?
    - peebeebee 3y ago
      Yesterday, it failed to give me the correct answer to 4 + 2 / 2. It said 3...
      - isaacfrond 3y ago
        Just tried in chatGpt-4. It gives the correct output (5), along with a short explanations of the order of operations (which you probably need to know, if you're asking the question).
        xanderlewis 3y ago
        We shouldn't necessarily regard 5 as the correct output. Sure, almost all of us choose to make division higher precedence than addition, but there's no reason that has to be the case. I think a truly intelligent system would reply with 5 (which follows the usual convention, and would therefore mimic the standard human response), but immediately ask if perhaps you had intended a different order of operations (or even other meanings for the symbols), and suggest other possibilities and mention the fact that your question could be considered not well-defined...which is basically what it did.
deleted 3y ago
[deleted]

by XCSme 3y ago

yberreby 3y ago

> It just computes next frame based on current one and what it learned before, it's a plausible continuation. ... > Now it's just a "what is the next frame of this 2D image" model plus some textual context. This is incorrect. Sora is not an autoregressive model like GPT, but a diffusion transformer. From the technical report[1], it is clear that it predicts the entire sequence of spatiotemporal patches at once. [1]: https://openai.com/research/video-generation-models-as-world-simulators https://openai.com/research/video-generation-models-as-world...

XCSme 3y ago
Good link. But, even there it says: > Sora currently exhibits numerous limitations as a simulator. For example, it does not accurately model the physics of many basic interactions, like glass shattering. Other interactions, like eating food, do not always yield correct changes in object states Regardless whether all the frames are generated at once, or one by one, you can see in their examples it's still just pixel based. See the first example with the dog with blue hat, the woman has a blue thing suddenly spawn into her hand because her hand went over another blue area of the image.
- yberreby 3y ago
  I'm not denying that there are obvious limitations. However, attributing them to being "pixel-based" seems misguided. First off, the model acts in latent space, not directly on pixels. Secondly, there is no fundamental limitation here. The model has already acquired limited-yet-impressive ability to understand movement, texture, social behavior, etc., just from watching videos. I learned to understand reality by interpreting photons and various sensory inputs. Does that make my model of reality fundamentally flawed? In the sense that I only have a partial intuitive understanding of it, yes. But I don't need to know Maxwell's equations to get a sense of what happens when I open the blinds or turn on my phone. I think many of the limitations we are seeing here - poor glass physics, flawed object permanence - will be overcome given enough training data and compute. We will most likely need to incorporate exploration, but we can get really far with astute observation.
  - XCSme 3y ago
    This is an excellent comparison and I agree with you. Unfortunately we are flawed. We do know how physics work intuitively and can somewhat predict them, but not perfectly. We can imagine how a ball will move, but the image is blurry and trajectory only partially correct. This is why we invented math and physics studies, to be able to accurately calculate, predict and reproduce those events. We are far off from creating something as efficient as the human brain. It will take insane amounts of compute power to simply match our basic innacurate brains, imagine how much will be needed to create something that is factually accurate.
    - yberreby 3y ago
      Indeed. But a point that is often omitted from comparisons with organic brains is how much "compute equivalent" we spent through evolution. The brain is not a blank slate; it has clear prior structure that is genetically encoded. You can see this as a form of pretraining through a RL process wherein reward ~= surviving and procreating. If you see things this way, data-efficiency comparisons are more appropriate in the context of learning a new task or piece of information, and foundation models tend to do this quite well. Additionally, most of the energy cost comes from pretraining, but once we have the resulting weights, downstream fine-tuning or inference are comparatively quite cheap. So even if the energy cost is high, it may be worth it if we get powerful generalist models that we can specialize in many different ways. > This is why we invented math and physics studies, to be able to accurately calculate, predict and reproduce those events. We won't do away without those, but an intuitive understanding of the world can go a long way towards knowing when and how to use precise quantitative methods.
  - sydd 3y ago
    Actually your comment gives me hope that we will never have AI singularity, since how the brain works is flawed, and were trying to copy it. Heck a super AI might not even be possible, what if we're peak intelligence with our millions of years of evolution? Just adding compute speed will not help much -- say the goal of an intelligence is to win a war. If you're tasked with it then it doesn't matter if you have a month or a decade (assume that time is.frozen while you do your research), its a too complex problem and simply cannot be solved, and the same goes for an AI. Or it will be like with chess solvers, machines will be more intelligent than us simply because they can load much more context to solve a problem than us in their "working memory"

famouswaffles 3y ago

GPT-4 doesn't "struggle with math". It does fine. Most humans aren't any better. Sora is not autoregressive anyway but there's nothing "just" and next frame/token prediction.

8n4vidtmkvmk 3y ago
It absolutely struggles with math. It's not solving anything. It sometimes gets the answer right only because it's seen the question before. It's rote memorization at best.
- famouswaffles 3y ago
  No it doesn't. I know because I've actually used the thing and you clearly haven't. And if Terence Tao finds some use for GPT-4 as well as Khan Academy employing it as a Math tutor then I don't think I have some wild opinion either. Now Math isn't just Arithmetic but do you know easy it is to go out of training for say Arithmetic ?
  - peebeebee 3y ago
    Yesterday, it failed to give me the correct answer to 4 + 2 / 2. It said 3...
    - isaacfrond 3y ago
      Just tried in chatGpt-4. It gives the correct output (5), along with a short explanations of the order of operations (which you probably need to know, if you're asking the question).
      - xanderlewis 3y ago
        We shouldn't necessarily regard 5 as the correct output. Sure, almost all of us choose to make division higher precedence than addition, but there's no reason that has to be the case. I think a truly intelligent system would reply with 5 (which follows the usual convention, and would therefore mimic the standard human response), but immediately ask if perhaps you had intended a different order of operations (or even other meanings for the symbols), and suggest other possibilities and mention the fact that your question could be considered not well-defined...which is basically what it did.

deleted 3y ago

[deleted]