4 ms·
>There is no such difference, we evaluate that based on their output. We see these massive model make silly errors that nobody who understands it would make, th
by ninetyninenine 3y ago
>There is no such difference, we evaluate that based on their output. We see these massive model make silly errors that nobody who understands it would make, thus we say the model doesn't understand. We do that for humans as well.
Two things. We also see the model make things that are correct. In fact the mistakes are a minority in comparison to what it got correct. That is in itself an indicator of understanding to a degree.
The other thing is, if a human tried to reproduce that output according to the same prompt, the human would likely not generate something photorealistic and the thing a human comes up with will be flawed, ugly disproportionate wrong and an artistic travesty. Does this mean a human doesn't understand reality? No.
Because the human generates worse output visually than an LLM we cannot say the human doesn't understand reality.
Additionally the majority of the generated media is correct. Therefore it can be said that the LLM understands the majority of the task it was instructed to achieve.
Sora understands the shape of the dog. That is in itself remarkable. I'm sure with enough data sora can understand the world completely and to a far greater degree than us.
I would say it's uncharitable to say sora doesn't understand physics when it gets physics wrong, and that for the things it gets right it's only heuristics.
- beardedwizard 3y agoHow can it possibly understand physics when the training data does not teach it or contain the laws of physics?
- ninetyninenine 3y agoVideo data contains physics. Objects in motion obey the laws of physics. Sora understand physics the same way you understand it.
- beardedwizard 3y agoI understand physics because science has performed a series of measurable experiments over 100s of years resulting in concrete mathematical formulas and theories that explain the laws of physics so that they can be reproduced by machines. Sora has zero of this knowledge. This is very much allegory of the cave[1]. If Sora sees a series of images that contain impossible physics, for example MC Escher paintings, what will happen? 1: https://en.wikipedia.org/wiki/Allegory_of_the_cave https://en.wikipedia.org/wiki/Allegory_of_the_cave
- ninetyninenine 3y agoSora understands basic motion without mathematics in the same way people typically understand physics. A person with no knowledge about mathematical formulas will still be able to recognize impossible MC Escher paintings. With enough data, Sora will be able to generate both impossible and possible physics and know the difference. We can already see in the video that it has a rough understanding of it.