2 ms·

For the skeptics, think about how bad the video generation models were last year versus today. Besides optimizations, the biggest difference between those model

by ilaksh 1y ago

For the skeptics, think about how bad the video generation models were last year versus today. Besides optimizations, the biggest difference between those models and todays are more training and clever tweaks. A lot of core concepts are the same or very similar. So I think that the fact that this is so close to a real game engine (if you squint) and combined with one or two other demos that were a little bit more general purpose, is pretty big news. Because it is concrete progress on the path to being able to prompt games into existence the same way we can generate videos, images or HTML/JS etc. It's a step beyond generating games or other code with LLMs. One can even imagine (maybe) with the right training and architecture, you could prompt your productivity tools into existence -- rendered on the fly frame by frame. If you keep going, maybe our whole universe is a simulation in some kind of incredible alien neural network or pseudo-reality decompression system.

samrus 1y ago
I dont mean to insult you but your last paragraph makes me not take you seriously at all. I could feel it in the rest of the post but the idea of an alien simulation just reaks of ayuhasca fueled quack science > A lot of core concepts are the same or very similar. One point about core models is that LLMs mess up long contexts because they are trained on a static system rather (next token prediction) than a dynamic system (a reactive world, like a market or a road), so they dont know that their decisions affect the the world they are observing. This means they fundamentally cant model and extrapolate how actions affect current state. They can memorize and interpolate that as a static system, which is what they do for these videos and game systems, but that is incredibly data inefficient. Thats a core problem in them simulating games rather than understanding what a text says and means. The text wont change as they interpret it. These models will need to be re hypithesized to understand that their actions change their environment and that is not a trivial change thats a few years ago, thats a change at the level of the transformers where it might happen next year or 50 years from now. theres no way to tell
- ilaksh 1y ago
  The game generator is not an LLM.

by ilaksh 1y ago

samrus 1y ago

I dont mean to insult you but your last paragraph makes me not take you seriously at all. I could feel it in the rest of the post but the idea of an alien simulation just reaks of ayuhasca fueled quack science > A lot of core concepts are the same or very similar. One point about core models is that LLMs mess up long contexts because they are trained on a static system rather (next token prediction) than a dynamic system (a reactive world, like a market or a road), so they dont know that their decisions affect the the world they are observing. This means they fundamentally cant model and extrapolate how actions affect current state. They can memorize and interpolate that as a static system, which is what they do for these videos and game systems, but that is incredibly data inefficient. Thats a core problem in them simulating games rather than understanding what a text says and means. The text wont change as they interpret it. These models will need to be re hypithesized to understand that their actions change their environment and that is not a trivial change thats a few years ago, thats a change at the level of the transformers where it might happen next year or 50 years from now. theres no way to tell

ilaksh 1y ago
The game generator is not an LLM.