6 ms·

Not a chance. There are fundamental limitations with what are in the end all essentially neural nets; there is no understanding, only prediction. Prediction a

Not a chance. There are fundamental limitations with what are in the end all essentially neural nets; there is no understanding, only prediction. Prediction alone is not enough to emulate reality, which is why for example genuinely self-driving cars have not, and will not, emerge. A fundamental advance in AI technology will be required for that, something which leads to genuine intelligence, and we are no closer to that than ever we were.

jiggawatts 2y ago
For simulations like games, it's a trivial matter to feed the neural game engine pixel-perfect metadata. Instead of rendering the final shaded and textured pixels, the engine would output just the material IDs, motion vectors, and similar "meta" data that would normally be the inputs into a real-time shader. The AI can use this as inputs to render a photorealistic output. It can be trained using offline-rendered "ground-truth" raytraced scenes. Potentially, video labelled in a similar way could be used to give it a flair of realism. This is already what NVIDIA DLSS and similar AI upscaling tech uses. The obvious next step is not just to upscale rendered scenes, but to do the rendering itself.
fancyfredbot 2y ago
Looking at the examples of 2600 games in the paper I'm not sure you can tell that they are just predictions. Have you considered how you'd tell the difference between a prediction and understanding in practice?
francoisfleuret 2y ago
"there is no understanding, only prediction" I have no idea what this means.
- tourmalinetaco 2y ago
  The MLM has no idea what it’s making, where you are in the map, what you left behind, and what you picked up. It can accurately predict what comes next, but if you pick up an item and do a 360° turn the item will be back and you can repeat the process.
- GaggiX 2y ago
  When a human does it, it's understanding, when an AI does it, it's prediction, I thinks it's very clear /s
  - therouwboat 2y ago
    Does what? In normal game world things tend to stay where they are without player having to do anything.
    - GaggiX 2y ago
      We are talking about neural networks in general, not this one or that one, if you train a bad model or the model is untrained it would not indeed understand much or anything.
- nonrandomstring 2y ago
  > > "there is no understanding, only prediction" > I have no idea what this means. You can throw a ball up in the air and predict that it will fall again and bounce. You have no understanding of mass, gravity, acceleration, momentum, impulse, elasticity... You can press a button that makes an Uber car appear in reality and take you home. You have no understanding of apps, operating systems, radio, internet, roads, wheels, internal combustion engines, driving, GPS, maps... This confusion of understanding and prediction affects a lot of people who use technology in a "machine-like" way, purely instrumental and utilitarian... "how does this get me what I want immediately?" You can take any complex reality and deflate it, abstract it, reduce it down to a mere set of predictions that preserve all the utility for a narrow task (in this case visual facsimile) but strip away all depth of meaning. The models, of both the system and the internal working model of the user are flattened. In this sense "AI" is probably the greatest assault on actual knowledge since the book burning under totalitarian regimes of the mid 20th century.
  - GaggiX 2y ago
    What if the model actually understands that the ball will fall and bounce because of mass, gravity, acceleration, momentum, impulse, elasticity? I mean you can just ask ChatGPT and Claude, I guess you would answer that in this case it's just prediction, but if they were human then it would be understanding.
    - nonrandomstring 2y ago
      > I guess you would answer that in this case it's just prediction, No I would answer that it is indeed understanding, to upend your "guess" (prediction) and so prove that while you think you can "predict" the next answer you lack understanding of what the argument is really about :)
      - GaggiX 2y ago
        I think I understand the topic quite well, since you deliberately deviate from answering the question. You made a practical example that doesn't really work in practice.
killerstorm 2y ago
That's bs. You have no understanding of understanding. Hooke's law was pure curve-fitting. Hooke definitely did not understand the "why". And yet we don't consider that bad physics. Newton's laws can be derived from curve fitting. How is that different from "understanding"?
- madaxe_again 2y ago
  Einstein couldn’t even explain why general relativity occurred. Sure, spacetime is curved by mass, but why? What a loser.
  - killerstorm 2y ago
    It's very illustrative to look into the history of discovery of laws of motion, as it's quite well documented. People have an intuitive understanding of motion - we see it literally every day, we throw objects, etc. And yet it took literally thousands of years since discovery of mathematics (geometry, etc.) to formulate a concept of force, momentum, etc. Ancient Greek mathematicians could do integration, so they were not lacking mathematical sophistication. And yet their understanding of motion was so primitive: Aristotle, an extremely smart man, was muttering something about "violent" and "natural" motion: https://en.wikipedia.org/wiki/Newton%27s_laws_of_motion#Antiquity_and_medieval_background https://en.wikipedia.org/wiki/Newton%27s_laws_of_motion#Anti... People started to understand the conservation of quantity of motion only in 17th century. So we have two possibilities: * everyone until 17th century was dumb af (despite being able to do quite impressive calculations) * scientific discovery is really a heuristic-driven search process where people try various things until they find a good fit I.e. millions of people were somehow failing to understand motion for literally thousands of years until they collected enough assertions about motion that they were able to formulate the rule of conservation, test it, and confirm it fits. And only then it became understanding. You can literally see conservation of momentum on a billiard table: you "violently" hit one ball, it hits other balls and they start to move, but slower, etc. So you really transfer something from one ball to the rest. And yet people could not see it for thousands of years. What this shows is that there's nothing fundamental about understanding: it's just a sense of familiarity, it is a sense that your model fits well. Under the hood it's all prediction and curve fitting. We literally have prediction hardware in our brains: cerebellum has specialized cells which can predict, e.g. motion. So people with damaged cerebellum have impaired movement: they still can move, but their movement are not precise. When do you think we find specialized understanding cells in the human brain?
madaxe_again 2y ago
Yet we have no understanding, only prediction. We can describe a great many things in detail, how they interact - and we can claim to understand things, yet if you recursively ask “why?” everybody, and I mean everybody, will reach a point where they say “I don’t know” or “god”. An incomplete understanding is no understanding at all, and I would argue that we can only predict, and we can certainly emulate reality, otherwise we would not be able to function within it. A toddler can emulate reality, anticipate causality - and they certainly can’t be said to be in possession of a robust grand unified theory.
esafak 2y ago
Ilya Sutskever, among others, believe that better prediction is achieved through understanding, and that models can be learn to understand. As we have. We are not born understanding everything.

jiggawatts 2y ago

For simulations like games, it's a trivial matter to feed the neural game engine pixel-perfect metadata. Instead of rendering the final shaded and textured pixels, the engine would output just the material IDs, motion vectors, and similar "meta" data that would normally be the inputs into a real-time shader. The AI can use this as inputs to render a photorealistic output. It can be trained using offline-rendered "ground-truth" raytraced scenes. Potentially, video labelled in a similar way could be used to give it a flair of realism. This is already what NVIDIA DLSS and similar AI upscaling tech uses. The obvious next step is not just to upscale rendered scenes, but to do the rendering itself.

fancyfredbot 2y ago

Looking at the examples of 2600 games in the paper I'm not sure you can tell that they are just predictions. Have you considered how you'd tell the difference between a prediction and understanding in practice?

francoisfleuret 2y ago

"there is no understanding, only prediction" I have no idea what this means.

tourmalinetaco 2y ago
The MLM has no idea what it’s making, where you are in the map, what you left behind, and what you picked up. It can accurately predict what comes next, but if you pick up an item and do a 360° turn the item will be back and you can repeat the process.
GaggiX 2y ago
When a human does it, it's understanding, when an AI does it, it's prediction, I thinks it's very clear /s
- therouwboat 2y ago
  Does what? In normal game world things tend to stay where they are without player having to do anything.
  - GaggiX 2y ago
    We are talking about neural networks in general, not this one or that one, if you train a bad model or the model is untrained it would not indeed understand much or anything.
nonrandomstring 2y ago
> > "there is no understanding, only prediction" > I have no idea what this means. You can throw a ball up in the air and predict that it will fall again and bounce. You have no understanding of mass, gravity, acceleration, momentum, impulse, elasticity... You can press a button that makes an Uber car appear in reality and take you home. You have no understanding of apps, operating systems, radio, internet, roads, wheels, internal combustion engines, driving, GPS, maps... This confusion of understanding and prediction affects a lot of people who use technology in a "machine-like" way, purely instrumental and utilitarian... "how does this get me what I want immediately?" You can take any complex reality and deflate it, abstract it, reduce it down to a mere set of predictions that preserve all the utility for a narrow task (in this case visual facsimile) but strip away all depth of meaning. The models, of both the system and the internal working model of the user are flattened. In this sense "AI" is probably the greatest assault on actual knowledge since the book burning under totalitarian regimes of the mid 20th century.
- GaggiX 2y ago
  What if the model actually understands that the ball will fall and bounce because of mass, gravity, acceleration, momentum, impulse, elasticity? I mean you can just ask ChatGPT and Claude, I guess you would answer that in this case it's just prediction, but if they were human then it would be understanding.
  - nonrandomstring 2y ago
    > I guess you would answer that in this case it's just prediction, No I would answer that it is indeed understanding, to upend your "guess" (prediction) and so prove that while you think you can "predict" the next answer you lack understanding of what the argument is really about :)
    - GaggiX 2y ago
      I think I understand the topic quite well, since you deliberately deviate from answering the question. You made a practical example that doesn't really work in practice.

killerstorm 2y ago

That's bs. You have no understanding of understanding. Hooke's law was pure curve-fitting. Hooke definitely did not understand the "why". And yet we don't consider that bad physics. Newton's laws can be derived from curve fitting. How is that different from "understanding"?

madaxe_again 2y ago
Einstein couldn’t even explain why general relativity occurred. Sure, spacetime is curved by mass, but why? What a loser.
- killerstorm 2y ago
  It's very illustrative to look into the history of discovery of laws of motion, as it's quite well documented. People have an intuitive understanding of motion - we see it literally every day, we throw objects, etc. And yet it took literally thousands of years since discovery of mathematics (geometry, etc.) to formulate a concept of force, momentum, etc. Ancient Greek mathematicians could do integration, so they were not lacking mathematical sophistication. And yet their understanding of motion was so primitive: Aristotle, an extremely smart man, was muttering something about "violent" and "natural" motion: https://en.wikipedia.org/wiki/Newton%27s_laws_of_motion#Antiquity_and_medieval_background https://en.wikipedia.org/wiki/Newton%27s_laws_of_motion#Anti... People started to understand the conservation of quantity of motion only in 17th century. So we have two possibilities: * everyone until 17th century was dumb af (despite being able to do quite impressive calculations) * scientific discovery is really a heuristic-driven search process where people try various things until they find a good fit I.e. millions of people were somehow failing to understand motion for literally thousands of years until they collected enough assertions about motion that they were able to formulate the rule of conservation, test it, and confirm it fits. And only then it became understanding. You can literally see conservation of momentum on a billiard table: you "violently" hit one ball, it hits other balls and they start to move, but slower, etc. So you really transfer something from one ball to the rest. And yet people could not see it for thousands of years. What this shows is that there's nothing fundamental about understanding: it's just a sense of familiarity, it is a sense that your model fits well. Under the hood it's all prediction and curve fitting. We literally have prediction hardware in our brains: cerebellum has specialized cells which can predict, e.g. motion. So people with damaged cerebellum have impaired movement: they still can move, but their movement are not precise. When do you think we find specialized understanding cells in the human brain?

madaxe_again 2y ago

Yet we have no understanding, only prediction. We can describe a great many things in detail, how they interact - and we can claim to understand things, yet if you recursively ask “why?” everybody, and I mean everybody, will reach a point where they say “I don’t know” or “god”. An incomplete understanding is no understanding at all, and I would argue that we can only predict, and we can certainly emulate reality, otherwise we would not be able to function within it. A toddler can emulate reality, anticipate causality - and they certainly can’t be said to be in possession of a robust grand unified theory.

esafak 2y ago

Ilya Sutskever, among others, believe that better prediction is achieved through understanding, and that models can be learn to understand. As we have. We are not born understanding everything.