4 ms·

What is actually of value here? There's no actual game, it's incredibly expensive to compute, the behavior is erratic.. It's cool because it's new - but that wi

by brink 2y ago

What is actually of value here? There's no actual game, it's incredibly expensive to compute, the behavior is erratic.. It's cool because it's new - but that will quickly wear off, and once that's gone, what's left? There's insane amounts of money being spent on this, and for what?

deleted 2y ago
[deleted]
adverbly 2y ago
> What is actually of value here? Noone knows yet. AI technology like this is closer to scientific research than it is to product development. AI is basically new magic, and people are in a "discovery" phase where we are still trying to figure out what is possible. Nothing of value was immediately created when they discovered DNA. Productization came much later when it was combined with other technologies to fit a particular use case.
Menu_Overview 2y ago
Well, what's next? Beyond prototyping, I imagine this is an early step towards more practical agents building their own world model. Better problem solving. Prompt: Here's a blueprint of my new house and a photo of my existing furniture. Show me some interior design options.
ilaksh 2y ago
It's an obviously amazing research development. You just don't like AI. It can be used for training agents, prototyping, video generation, and is quite possibly a glimpse of a whole new type of entertainment or a new way to create video games. What's the point of the massive amount of money spent on video games in general? Or all of the energy spent moving people back and forth to an office? Or expensive meals at restaurants? Or trillions in weaponry? Or television shows or movies?
- nightski 2y ago
  Video games bring billions of real people joy. This is sitting in some lab at Google inaccessible to anyone.
  - lassenordahl 2y ago
    Is your argument that them sharing research progress and demos doesn't benefit anybody purely because we can't immediately play around with them? I feel like sharing early closed-source blog-posts is part of the research process. I'm sure someone in this thread has thought of a use case that the Google team missed. Open/closed source arguments here feel premature IMO.
    - nightski 2y ago
      It's not part of the research process. Being part of the search process would involve a publication and sharing code/data/results/methods. It's not research unless it can be verified by peers. This is just a marketing fluff piece that does not benefit anyone and is ego stroking at best.
      - lassenordahl 2y ago
        Hm yeah - I think you and I just have differing opinions on the research process. I'd be a bit more vague, and define the publication process as something similar to you. I still think things like this are important, and at least give folks a bit of time to ideate on what will be possible in a few years. Of course having the model or architecture on hand would be nice, but I'm not holding that against Google here.
ThouYS 2y ago
same q here. what can I do with this "world model" that I can't do with a game like minecraft or counter strike? asked the same thing a while back, and the answers boiled down to "somehow helps RL agents train". but how exactly? no clue
- ogogmad 2y ago
  Making a computer game is very expensive and time-consuming. This technology might allow a 12 year old to produce a fully working AAA-quality game on their own for almost nothing. But sigh it's an early demo that needs some improving. [edited out some barbs I wrote because I find some comments on this website REALLY annoying]
  - ThouYS 2y ago
    lol
awfulneutral 2y ago
Well, in the future you could imagine that instead of programming a game, you can just generate each individual frame on the fly at 60fps. You could be playing 2D Mario and then the game could have him morph into 3D and take off into space or something. You could also generate any software or OS frontend on the fly really, if you can make it so the AI can keep track of your data and make it consistent enough to be usable. Does this have positive or negative value? I don't know.
golol 2y ago
Do you want household androids? Because this kind of stuff is on the level of research a bery large step towards that. Think as it as ab example where we can make a model understand a lot of physical common sense stuff, which is the goal for robotics right now.
- suddenlybananas 2y ago
  This is really not the avenue for house-hold robots. Interacting with the actual physical world is very different from creating a video game.
  - deleted 2y ago
    [deleted]
  - sangnoir 2y ago
    > Interacting with the actual physical world is very different from creating a video game The major difference being the former scales very poorly for generating training data compared to the latter. Genie 2 is not even a video game and has worse fidelity that video games, the upside is it probably scales even better than video games for generating training scenarios. If you want androids in teal life, Genie 2 (or similar systems) is how you bootstrap the agent AI. The training pipeline will be: raw video -> Genie 2 -> game engine with rules -> physical robot
    - youoy 2y ago
      > The training pipeline will be: raw video -> Genie 2 -> game engine with rules -> physical robot One of those arrows is not like the others
      - sangnoir 2y ago
        The final step is an oversimplification: purpose-built simulator -> deconstructed robot on a lab workbench -> controlled space -> "real world" with constraints -> real world Any model would have to succeed in one stage before it can proceed to the next one.
mitthrowaway2 2y ago
I'm not an expert in this space but I can see the value. It allows an endless loop of generating novel scenarios and evaluating an AI agent's performance within that scenario (for example, "go up the stairs"). A world with one minute of coherence is about enough to evaluate whether the AI's actions were in the right direction or not. When you then want to run an agent on a real task in the real world, with video-input data, you can run the same policy that it learned in dream-world simulation. The real world has coherence, so the AI agent's actions just need to string together well enough minute-by-minute to work toward achieving a goal. You could use real video games to do this but I guess there'd be a risk of over-fitting; maybe it would learn too precisely what a staircase looks like in Minecraft, but fail to generalize that to the staircase in your home. If they can simulate dream worlds (as well as, presumably, worlds from real photos), then they can train their agents this way. This would only be training high-level decision policies (ie, WASD inputs). For something like a robot, lower level motor control loops would still be needed to execute those commands. Of course you could just do your training in the real world directly, because it already has coherence and plenty of environmental variety. But the learning process involves lots of learning from failure, and that would probably be even more expensive than this expensive simulator. Despite the claims I don't think it does much to help with AI safety. It can help avoid hilarious disasters of an AI-in-training crashing a speedboat onto the riverbank, but I don't think there's much here that helps with the deeper problems of value-alignment. This also seems like an effective way to train robo-killbots who perceive the world as a dreamlike first-person shooter.
modeless 2y ago
> It's cool because it's new - but that will quickly wear off, and once that's gone, what's left? To have this perspective you must believe that this will never get better than it currently is, its limitations will never be fixed, and it will never lead to any other applications. I don't know how people can continue to look at these things with such a lack of imagination given the pace of progress in the field.
- zamadatix 2y ago
  I think the problem is less to do with imagination and more to do with being willing to fail a metric shit ton to find out how, every once in a while, you didn't fail due to some really important and surprising reason you wouldn't have found nearly as quickly only ever going after what you were already certain of.
xandrius 2y ago
Nothing is of value until it is.
3abiton 2y ago
This is an incredible start. The potential is immense, yes there arekinks, but in 10 years?
rlupi 2y ago
We, humans, use dreams for consolidating memory and information recall, process emotions and rehearse feelings in different imagined contexts, mental housekeeping / pruning away partial or unnecessary information, replay recent events to review and analyze, etc. Dreaming and sleeping is incredibly expensive, we spend 33% of our "availability" on average asleep. This kind of work is a step toward building similar tools for general AI agents (IMHO).
araes 2y ago
Ruining the video game industry. The implication the technology is plausible takes away all further interest in having anything except "end game content." All motivation to make further games is removed because now "somebody" can spool out a 3D adventure game instantly with a line of text. It implies you'll waste a year of your time, and just before release, out pops the dramatically better AI product to steal away all further business and reduce all time you've spent to meaningless. Everybody then waits indefinitely for the "endgame gear." https://xkcd.com/989/ https://xkcd.com/989/ Exactly like LLMs and image generators almost completely took away all business for normal writers and normal painters, because now all managers want is the AI. Now there's endless articles about how "somebody" prefers "AI" for every task. Now the market won't invest in anything unless it has "AI" in the name. Now people idiotically add "AI" to everything just to have the investment.

by brink 2y ago

deleted 2y ago

[deleted]

adverbly 2y ago

> What is actually of value here? Noone knows yet. AI technology like this is closer to scientific research than it is to product development. AI is basically new magic, and people are in a "discovery" phase where we are still trying to figure out what is possible. Nothing of value was immediately created when they discovered DNA. Productization came much later when it was combined with other technologies to fit a particular use case.

Menu_Overview 2y ago

Well, what's next? Beyond prototyping, I imagine this is an early step towards more practical agents building their own world model. Better problem solving. Prompt: Here's a blueprint of my new house and a photo of my existing furniture. Show me some interior design options.

ilaksh 2y ago

It's an obviously amazing research development. You just don't like AI. It can be used for training agents, prototyping, video generation, and is quite possibly a glimpse of a whole new type of entertainment or a new way to create video games. What's the point of the massive amount of money spent on video games in general? Or all of the energy spent moving people back and forth to an office? Or expensive meals at restaurants? Or trillions in weaponry? Or television shows or movies?

nightski 2y ago
Video games bring billions of real people joy. This is sitting in some lab at Google inaccessible to anyone.
- lassenordahl 2y ago
  Is your argument that them sharing research progress and demos doesn't benefit anybody purely because we can't immediately play around with them? I feel like sharing early closed-source blog-posts is part of the research process. I'm sure someone in this thread has thought of a use case that the Google team missed. Open/closed source arguments here feel premature IMO.
  - nightski 2y ago
    It's not part of the research process. Being part of the search process would involve a publication and sharing code/data/results/methods. It's not research unless it can be verified by peers. This is just a marketing fluff piece that does not benefit anyone and is ego stroking at best.
    - lassenordahl 2y ago
      Hm yeah - I think you and I just have differing opinions on the research process. I'd be a bit more vague, and define the publication process as something similar to you. I still think things like this are important, and at least give folks a bit of time to ideate on what will be possible in a few years. Of course having the model or architecture on hand would be nice, but I'm not holding that against Google here.

ThouYS 2y ago

same q here. what can I do with this "world model" that I can't do with a game like minecraft or counter strike? asked the same thing a while back, and the answers boiled down to "somehow helps RL agents train". but how exactly? no clue

ogogmad 2y ago
Making a computer game is very expensive and time-consuming. This technology might allow a 12 year old to produce a fully working AAA-quality game on their own for almost nothing. But sigh it's an early demo that needs some improving. [edited out some barbs I wrote because I find some comments on this website REALLY annoying]
- ThouYS 2y ago
  lol

awfulneutral 2y ago

Well, in the future you could imagine that instead of programming a game, you can just generate each individual frame on the fly at 60fps. You could be playing 2D Mario and then the game could have him morph into 3D and take off into space or something. You could also generate any software or OS frontend on the fly really, if you can make it so the AI can keep track of your data and make it consistent enough to be usable. Does this have positive or negative value? I don't know.

golol 2y ago

Do you want household androids? Because this kind of stuff is on the level of research a bery large step towards that. Think as it as ab example where we can make a model understand a lot of physical common sense stuff, which is the goal for robotics right now.

suddenlybananas 2y ago
This is really not the avenue for house-hold robots. Interacting with the actual physical world is very different from creating a video game.
- deleted 2y ago
  [deleted]
- sangnoir 2y ago
  > Interacting with the actual physical world is very different from creating a video game The major difference being the former scales very poorly for generating training data compared to the latter. Genie 2 is not even a video game and has worse fidelity that video games, the upside is it probably scales even better than video games for generating training scenarios. If you want androids in teal life, Genie 2 (or similar systems) is how you bootstrap the agent AI. The training pipeline will be: raw video -> Genie 2 -> game engine with rules -> physical robot
  - youoy 2y ago
    > The training pipeline will be: raw video -> Genie 2 -> game engine with rules -> physical robot One of those arrows is not like the others
    - sangnoir 2y ago
      The final step is an oversimplification: purpose-built simulator -> deconstructed robot on a lab workbench -> controlled space -> "real world" with constraints -> real world Any model would have to succeed in one stage before it can proceed to the next one.

mitthrowaway2 2y ago

I'm not an expert in this space but I can see the value. It allows an endless loop of generating novel scenarios and evaluating an AI agent's performance within that scenario (for example, "go up the stairs"). A world with one minute of coherence is about enough to evaluate whether the AI's actions were in the right direction or not. When you then want to run an agent on a real task in the real world, with video-input data, you can run the same policy that it learned in dream-world simulation. The real world has coherence, so the AI agent's actions just need to string together well enough minute-by-minute to work toward achieving a goal. You could use real video games to do this but I guess there'd be a risk of over-fitting; maybe it would learn too precisely what a staircase looks like in Minecraft, but fail to generalize that to the staircase in your home. If they can simulate dream worlds (as well as, presumably, worlds from real photos), then they can train their agents this way. This would only be training high-level decision policies (ie, WASD inputs). For something like a robot, lower level motor control loops would still be needed to execute those commands. Of course you could just do your training in the real world directly, because it already has coherence and plenty of environmental variety. But the learning process involves lots of learning from failure, and that would probably be even more expensive than this expensive simulator. Despite the claims I don't think it does much to help with AI safety. It can help avoid hilarious disasters of an AI-in-training crashing a speedboat onto the riverbank, but I don't think there's much here that helps with the deeper problems of value-alignment. This also seems like an effective way to train robo-killbots who perceive the world as a dreamlike first-person shooter.

modeless 2y ago

> It's cool because it's new - but that will quickly wear off, and once that's gone, what's left? To have this perspective you must believe that this will never get better than it currently is, its limitations will never be fixed, and it will never lead to any other applications. I don't know how people can continue to look at these things with such a lack of imagination given the pace of progress in the field.

zamadatix 2y ago
I think the problem is less to do with imagination and more to do with being willing to fail a metric shit ton to find out how, every once in a while, you didn't fail due to some really important and surprising reason you wouldn't have found nearly as quickly only ever going after what you were already certain of.

xandrius 2y ago

Nothing is of value until it is.

3abiton 2y ago

This is an incredible start. The potential is immense, yes there arekinks, but in 10 years?

rlupi 2y ago

We, humans, use dreams for consolidating memory and information recall, process emotions and rehearse feelings in different imagined contexts, mental housekeeping / pruning away partial or unnecessary information, replay recent events to review and analyze, etc. Dreaming and sleeping is incredibly expensive, we spend 33% of our "availability" on average asleep. This kind of work is a step toward building similar tools for general AI agents (IMHO).

araes 2y ago

Ruining the video game industry. The implication the technology is plausible takes away all further interest in having anything except "end game content." All motivation to make further games is removed because now "somebody" can spool out a 3D adventure game instantly with a line of text. It implies you'll waste a year of your time, and just before release, out pops the dramatically better AI product to steal away all further business and reduce all time you've spent to meaningless. Everybody then waits indefinitely for the "endgame gear." https://xkcd.com/989/ https://xkcd.com/989/ Exactly like LLMs and image generators almost completely took away all business for normal writers and normal painters, because now all managers want is the AI. Now there's endless articles about how "somebody" prefers "AI" for every task. Now the market won't invest in anything unless it has "AI" in the name. Now people idiotically add "AI" to everything just to have the investment.