16 ms·
Genie 3: A new frontier for world models
- mason_mpls 1y agoThe demo looks like they’re being very gentle with the AI, this doesn’t look like much of an advancement.
- 93po 1y agoI wouldn't want to be a hollywood production studio or game developer right now.
- tkgally 1y agoSame here. Though if I were a 17-year-old film fan or gamer with an imaginative drive, I would be really excited about the powerful creative tools that might become available to me soon.
- Mouvelie 1y agoI don't know if you had the teenage years I had, but there would be A LOT of NSFW content made on that thing.
- mclau157 1y agoHollywood maybe for small scenes, but gamers would quickly realize and destroy this level of quality and continuity vs. a 3D game engine with defined meshes
- 93po 1y agoi meant more in 5 years when its significantly better
- ducktective 1y agoWell, what would you want to be? Frontend dev? Mobile dev? Script writer? Logo designer? Junior lawyer?
- myaccountonhn 1y agoI actually think indie game dev is quite safe from AI (well its already insanely competitive). It might change the field, or shrink the market but I think AI has a chance at replacing workers where the only metric that matters is $$$ and productivity. I just don't see myself consuming, for example, an AI generated autobiography or any AI generated book. As long as enough people feel that way the market will continue to be there.
- rane 1y agoIs it though? Won't AI make the barrier of entry to indie game dev even lower, as assets and code will be able to be created effortlessly.
- hooverd 1y agoThere's already a glut of open world slop!
- vaenaes 1y agoA common refrain among the least creative people in the world.
- yanis_t 1y ago> Text rendering. Clear and legible text is often only generated when provided in the input world description. Reminds me of when image AIs weren't able to generate text. It wasn't too long until they fixed it.
- reactordev 1y agoAnd made hands 10x worse. Now hands are good, text is good, image is good, so we’ll have to play where’s Waldo all over again trying to find the flaw. It’s going to eventually get to a point where it’s one of those infinite zoom videos where the AI watermark is the size of 1/3rd of a pixel. What I’d really love to see more of is augmented video. Like, the stormtrooper vlogs. Runway has some good stuff but man is it all expensive.
- maerF0x0 1y agosomeone mentioned physics. Which might be an interesting conundrum because an important characteristic of games is that some part of them is both novel and unrealistic. (They're less fun if they're too real)
- reactordev 1y agoIt depends on the genre. Simulation “games” tend to love realism of simulation while providing an accelerated time. Others like you said, are more fun with plausible physics or physics that bend the rules a little bit. Sometimes, a game just about funky physics becomes a hit - Goat Simulator. Walking/Running/Steps have already been solved pretty well with NN’s, but simulation of vehicle engines and vehicle physics have not. Not to my knowledge. I suspect iRacing would be extremely interested in such a model. edit I take it back, PINN’s are a thing and now I have a new rabbit hole…
- TheAceOfHearts 1y agoI wouldn't say that the text problem has been fully fixed. It has certainly gotten a lot better, but even gpt-image-1 still fails occasionally when generating text.
- yanis_t 1y agoAnd unfortunately not possible to play around for the general public.
- cj 1y ago> World models are also a key stepping stone on the path to AGI, since they make it possible to train AI agents in an unlimited curriculum of rich simulation environments. I don't think Humans are the target market for this model, at least right now. Sounds like the use case is creating worlds for AI agents to play in.
- hodgehog11 1y agoThis kind of announcement without an appropriate demo to verify their claims is pretty common with DeepMind at this point. They barely even discuss their limitations, so as always, this should be taken with a grain of salt.
- Miraste 1y agoMost of the big labs never go into their models' limitations. OpenAI does it best, despite their inveterate hype-building. Their releases always have a reasonable limitations section, usually with text/image/video examples of failures.
- horhay 1y agoGoogle does a good job with that too usually. Which makes their last two announcements (IMO success and Genie 3) being a bit light on details is somewhat surprising.
- qingcharles 1y agoHere's a (weaker) competitor that's live: https://odyssey.world/introducing-interactive-video https://odyssey.world/introducing-interactive-video
- romanovcode 1y agoYeah, honestly - what's the point of announcing it then? I DECLARE BANKRUPTCY vibes here
- zb3 1y agoYet another unavailable model from Google.. if I can't use it, I don't care. Tell me about it when it's ready to use.
- Tadpole9181 1y agoWhat a strange take. Do you not care about news coming from the James Webb Telescope either, just because you can't play with the telescope personally? It's a whitepaper release to share the STOTA research. This doesn't seem like an economically viable model, nor does it look polished enough to be practically usable.
- growthwtf 1y agoI think it's a perfectly valid take coming from some intersection of an engineering mindset and FOSS culture. And, the comparison you bring up is a bit of a category error. We know how James Webb works and it's developed by an international consortium of researchers. One of our most trusted international institutions, and very verifiable. We do not know how Genie works, it is unverifiable to non-Google researchers, and there are not enough technical details to move much external teams forward. Worst case, this page could be a total fabrication intended to derail competition by lying about what Google is _actually_ spending their time on. We really don't know. I don't say this to defend the other comment and say you're wrong, because I empathize with both points. But I do think that treating Google with total credulity would be a mistake, and the James Webb comparison is a disservice to the JW team.
- zb3 1y agoJames Webb Telescope is not something that can be - and is released. AI models are, and others are announcing them when they're available, but DeepMind introduces noise here with their "trust us, that works, now go away" approach.
- delusional 1y ago> James Webb Telescope is not something that can be - and is released I would actually turn that around. The Telescope is released. It's flying around up there taking photos. If they kept it in some garage while releasing flashy PR pages about how groundbreaking it is, then I'd be pretty skeptical.
- NoScopeNinja 1y agoIt sounds cool that Genie 3 can make whole worlds you can explore, but I wonder how soon regular people will actually get to try it out?
- qingcharles 1y agoThese guys are working on the same thing and have a real demo you can play: https://odyssey.world/introducing-interactive-video https://odyssey.world/introducing-interactive-video
- yoavm 1y agoWow, a few years ago, if you've shown me this and Genie 3, I'd assume there were at least 10 years of development between them. This looks worse than Doom.
- qingcharles 1y agoThe rate of change is insane these days. I remember Sora launching and thinking "wow" and within weeks it looked like hot garbage.
- scoot 1y agoI'm pretty sure that got debunked as a handful of overfitted examples that fall apart as soon as you leave the area that was 3D recorded. Google's new tech is on another level entirely.
- brotchie 1y agoFirst AI thing that’s made me feel a bit of derealization… …and this is the worst the capabilities will ever be. Watching the video created a glimmer of doubt that perhaps my current reality is a future version of myself, or some other consciousness, that’s living its life in an AI hallucinated environment.
- deleted 1y ago[deleted]
- curwin 1y agoSame, even worse that the world was generated on the fly—nobody even hand-crafted it. That makes it even more depressing.
- drstewart 1y ago... so like the real world?
- thatfrenchguy 1y agoI mean, Minecraft worlds are generated on the fly and they've unleashed quite a it of creativity in children
- simianparrot 1y agoThe same argument can be used for anything. Personal jetpacks are the worst they’ll ever be. Doesn’t mean they’re any close to being useful.
- delusional 1y agoIt's also just wrong. Plenty of things get worse.
- crossbody 1y agoNot really. E.g. clay tablets, physical books will not be meaningfully better.
- modeless 1y agoConsistency over multiple minutes and it runs in real time at 720p? I did not expect world models to be this good yet. > Genie 3’s consistency is an emergent capability So this just happened from scaling the model, rather than being a consequence of deliberate architecture changes? Edit: here is some commentary on limitations from someone who tried it: https://x.com/tejasdkulkarni/status/1952737669894574264 https://x.com/tejasdkulkarni/status/1952737669894574264 > - Physics is still hard and there are obvious failure cases when I tried the classical intuitive physics experiments from psychology (tower of blocks). > - Social and multi-agent interactions are tricky to handle. 1vs1 combat games do not work > - Long instruction following and simple combinatorial game logic fails (e.g. collect some points / keys etc, go to the door, unlock and so on) > - Action space is limited > - It is far from being a real game engines and has a long way to go but this is a clear glimpse into the future. Even with these limitations, this is still bonkers. It suggests to me that world models may have a bigger part to play in robotics and real world AI than I realized. Future robots may learn in their dreams...
- kfarr 1y agoBitter lesson strikes again!
- nxobject 1y ago_Especially_ given the goal of a world model using a rasters-only frame-by-frame approach. Holy shit.
- kranke155 1y agoIt is truly remarkable. Even if you expected this to happen eventually, like I did, it seems like your assumptions are 2-10x of the time it’s needed to progress. It makes me think that Stargate might actually lead to AGI/ASI
- ivape 1y agoSo this just happened from scaling the model Unbelievable. How is this not a miracle? So we're just stumbling onto breakthroughs?
- deleted 1y ago[deleted]
- Oarch 1y agoI don't think I've ever seen a presentation that's had me question reality multiple times before. My mind is suitably blown.
- andhuman 1y agoNow this could be the killer app VR's been looking for.
- gundmc 1y agoInteresting! This feels like they're trying to position it as a competitor to Nvidia's Omniverse, which is based on the Universal Scene Descriptor format as the backbone. I wonder what format world objects can be ingested into Genie in - e.g. for the manufacturing use cases mentioned.
- dzonga 1y agomovies are about to become cheap to produce. good writers will remain scarce though. maybe we will have personalized movies written entirely through A.I
- mclau157 1y agoI can see this being incredible for history lessons and history school lectures
- MarkusQ 1y agoSome physicist once said "I endeavor to never write more clearly than I think"; in the same way, history probably shouldn't be presented more vividly than it's understood. (We already have this problem with people remembering incidental details and emotional vibes from historical fiction as if they were established historical fact; VR diffusion delusions would make this much worse.)
- scotty79 1y agoHistory is mostly made up. You can be sure mostly about general facts. The other 80% are just narratives.
- ecshafer 1y agoIf you read actual history the historians typically go into quite a lot of depth on why they think X happened as opposed to Y, and what the limitations are on the theories and the reasoning. The amount of archaeological and written records we have is very important to those facts.
- quesera 1y agoAlso true of the present. :)
- ivape 1y agoIt's going to replace video games.
- mclau157 1y agoDo people play video games to look at pretty scenery? No most people are testing skills in video games and this will not test skill for a while
- Workaccount2 1y agoI wonder how hard it would be to get VR output? That's an insane product right there just waiting to happen. Too bad Google sleeps so hard on the tech they create.
- SeanaldMcDnld 1y agoConsistent output and spatial coherence across each eye, maybe a couple years? But meeting head tracking accuracy and latency requirements, I’d bet decades. There’s no way any of this tech reduces end to end latency to acceptable levels, without a massive change in hardware. We’ll probably see someone use reprojection techniques in a year or so and claim they’ve done it. But true generated pixels straight to the headset based on head tracking, is so so far away.
- kridsdale3 1y agoAgree. So I'll make a wild bet of "20 years". And hope for the best.
- nosignono 1y agoYou don't have to do it in real time, per se. I imagine a world in which the renderer and the world generation are decoupled. For example, you could descriptively articulate what you wanted to achieve and have it generate a world, quietly do some structure from motion (or just generate the models and textures), and those those as assets in a game engine for the actual moment to moment rendering. You'd have some "please wait in this lobby space while we generate the universe" moments, but those are easy to hide with clever design.
- bigmadshoe 1y agoI think your timeline is off, at least for a tech demo. This model already runs at 24fps, and I bet could be made to run at >75fps by scaling hardware and distilling/quantizing the model to only work on certain environments. The two eye problem seems pretty trivial to me: add another image decoding head with the sole task of decoding the other eye. Training data for this can be plentifully gathered through simulated 3D data, or running existing 2D data (e.g. youtube videos) through slow mono to stereo models. This should add minimal latency as it's another head vs. subsequent layers. If you can train the model to allow WASD movement + mouse, head tracking is not very different. I think with enough effort we could probably build a VR experience using this today. Getting it onto affordable hardware could be a totally different story, but certainly not decades. Maybe I'm missing something though!
- fnands 1y agoDamn, I'm getting Black Mirror vibes from this. Maybe because I watched the Eulogy episode last night. Really great work though, impressive to see.
- hnthrow90348765 1y agoThink of the pornographic possibilities /s
- Mouvelie 1y agoWhy even be sarcastic about it ? There is no human invention that has not exploded thanks (or because of) pornographic possibilities. HD-DVD vs Blueray, Internet...I'd even argue that XR is not as big as it could be because it is really clamped down to deviant usage !
- ollin 1y agoThis is very encouraging progress, and probably what Demis was teasing [1] last month. A few speculations on technical details based on staring at the released clips: 1. You can see fine textures "jump" every 4 frames - which means they're most likely using a 4x-temporal-downscaling VAE with at least 4-frame interaction latency (unless the VAE is also control-conditional). Unfortunately I didn't see any real-time footage to confirm the latency (at one point they intercut screen recordings with "fingers on keyboard" b-roll? hmm). 2. There's some 16x16 spatial blocking during fast motion which could mean 16x16 spatial downscaling in the VAE. Combined with 1, this would mean 24x1280x720/(4x16x16) = 21,600 tokens per second, or around 1.3 million tokens per minute. 3. The first frame of each clip looks a bit sharper and less videogamey than later stationary frames, which suggests this is could be a combination of text-to-image + image-to-world system (where the t2i system is trained on general data but the i2w system is finetuned on game data with labeled controls). Noticeable in e.g. the dirt/textures in [2]. I still noticed some trend towards more contrast/saturation over time, but it's not as bad as in other autoregressive video models I've seen. [1] https://x.com/demishassabis/status/1940248521111961988 https://x.com/demishassabis/status/1940248521111961988 [2] https://deepmind.google/api/blob/website/media/genie_environmental_consistency_6_96KPmd3.mp4 https://deepmind.google/api/blob/website/media/genie_environ...
- ollin 1y agoRegarding latency, I found a live video of gameplay here [1] and it looks like closer to 1.1s keypress-to-photon latency (33 frames @ 30fps) based on when the onscreen keys start lighting up vs when the camera starts moving. This writeup [2] from someone who tried the Genie 3 research preview mentions that "while there is some control lag, I was told that this is due to the infrastructure used to serve the model rather than the model itself" so a lot of this latency may be added by their client/server streaming setup. [1] https://x.com/holynski_/status/1952756737800651144 https://x.com/holynski_/status/1952756737800651144 [2] https://togelius.blogspot.com/2025/08/genie-3-and-future-of-neural-game.html https://togelius.blogspot.com/2025/08/genie-3-and-future-of-...
- rotexo 1y agoYou know that thing in anxiety dreams where you feel very uncoordinated and your attempts to manipulate your surroundings result in unpredictable consequences? Like you try to slam on the brake pedal but your car doesn’t slow down, or you’re trying to get a leash on your dog to lead it out of a dangerous situation and you keep failing to hook it on the collar? Maybe that’s extra latency because your brain is trying to render the environment at the same time as it is acting.
- crossbody 1y agoI am much more convinced now that the Simulation Argument is correct
- sercanov 1y agoyeah the whole explosion around AI made me lean more to simulation theory. it's literally happening in front of our eyes and we're a baby civilization
- Fraterkes 1y agoI’m seeing a lot of variations on this in this thread, but we have been able to render photoreal things, and do intricate physical simulations, for a long time. This is mostly impressive because it is a real-time way to generate and render big, intricate worlds. But if you believe reality is a simulation, why would these “efficient” world-generation methods convince you of anything? The tech our reality would have to be running on is still inconceivable science fiction.
- ivape 1y agobut we have been able to render photoreal things, and do intricate physical simulations, for a long time. Not like this we haven't. This is convincing because I can have any of you close your eyes and imagine a world where pink rabbits hand out parking tickets. We're a neurolink away from going from thought > to prompt > to fantasy.
- crossbody 1y agoAgree with ivape. To add: our reality does not have to be rendered in it's entirety, we'll just have very convincing and unscripted first-person view simulations. Only what you look at is getting rendered (e.g. tiny structures only get rendered when you use microscope).
- Fraterkes 1y agoI guess I should have clarified: when you talk about reality being a simulation, do you mean that we collectively live in a simulated universe, or that you personally are playing a very realistic vr game?
- alec_irl 1y agoWhat is the purpose of this? It seems designed to muddy the waters of reality vs. falsehood and put creatives in film/tv out of jobs. Real Jurassic Park moment here
- Centigonal 1y agoThey mention some possible applications in the video. Training environments for robotics (use sample data to simulate the surface of mars or the inside of a nuclear reactor), educational worlds for students (like the old Encarta virtual tours), and disaster preparedness simulations (e.g. training firefighters on an endless variety of burning homes). Obviously, none of these are super viable given the low accuracy and steerability of world models out today, but positive applications for this kind of tech do exist. Also (I'm speculating now instead of restating the video), I think pretty soon someone will hook up a real time version of this to a voice model, and we will get some kind of interactive voice + keyboard (or VR) lucid dream experience.
- OhioMan2943 1y agoArrogant profit seeking capitalist dick measuring that will break our societies and ours and our childrens worldviews under some pathetic label of "exploring a new scientific frontier"
- sercanov 1y agolike how? is this mainly realtime inference?
- idencap 1y agowhat a time to be alive
- draculero 1y agowhat a time to be simulated! :D
- lackoftactics 1y agoI thought I was not going to see too many negative comments here, yet I was mistaken. I thought if it's not LLM, people would have a more nuanced take and could look at the research with an open mind. The examples on the website are probably cherry-picked, but progress is really nice compared to Genie 2. It's a nice step towards gains in embodied AI. Good work, DeepMind.
- Uehreka 1y agoA lot of the negativity around this post is about the fact that there’s no demo and no open weights, which is Correct Negativity. Like don’t get me wrong, it would be cool for something like this to exist, but I’ve generally learned not to trust AI companies’ descriptions of their models until someone (or I) can actually get their hands on it and see if it’s usable at all. A description of a model that isn’t going to be released to the public isn’t very interesting to me.
- whywhywhywhy 1y ago> but I’ve generally learned not to trust AI companies’ descriptions of their models Sora was described very similar to this as a "world simulator" but ultimately it never materialized. This one is a bit more hopeful from the videos though.
- rowanG077 1y agoI'm not sure this is interesting beyond the wow effect. Unless we can actually get the world out of the AI. The real reason chatgpt and friends actually have customers is that the text interface is actually durable and easily to build upon after generation. It's also super ez to feed text into a fresh cycle. But this, while looking fancy, doesn't seem to be on the path to actually working out. Unless there is a sane export to unreal or something.
- xlbuttplug2 1y agoWhat would scare me is if this becomes economically viable enough to release to the public, rather than staying an unlimited budget type of demo.
- nkotov 1y agoI wonder how far are we from being able to use this at home as a form of entertainment.
- timeattack 1y agoAdvances in generative AI are making me progressively more and more depressive. Creativity is taken from us at exponential rate. And I don't buy argument from people who are saying they are excited to live in this age. I can get that if that technology stopped at current state and remained to be just tools for our creative endeavours, but it doesn't seem to be an endgame here. Instead it aims to be a complete replacement. Granted, you can say "you still can play musical instruments/paint pictures/etc for yourself", but I don't think there was ever a period of time where creative works were just created for sake of itself rather for sharing it with others at masse. So what is final state here for us? Return to menial not-yet-automated work? And when this would be eventually automated, what's left? Plug our brains to personalized autogenerated worlds that are tailored to trigger related neuronal circuitry for producing ever increasing dopamine levels and finally burn our brains out (which is arguably already happening with tiktok-style leasure)? And how you are supposed to pay for that, if all work is automated? How economics of that is supposed to work? Looks like a pretty decent explanation of Fermi paradox. No-one would know how technology works, there are no easily available resources left to make use of simpler tech and planet is littered to the point of no return. How to even find the value in living given all of that?
- delfinom 1y agoAll I know is I am investing into suicide booth startups
- taberiand 1y agoSo that the robots have a leisure activity, or so that humans get a quick escape in the face of runaway climate change?
- wahnfrieden 1y agoAutomation only leads to more labor if we allow that employer relation to dictate so. Automation affords leisure time (for everything besides labor that life has to offer, including optional labor-like pursuits) but it’s currently unevenly distributed who gets to benefit from that
- seydor 1y agoWould progress in these be faster if they created 3d meshes and animations instead of full frame videos?
- zbrw 1y agoI believe that the corpus of video data to train on with video far exceeds that of 3D data. It's also much cheaper to produce video data. So I'd expect that this is probably the quickest way forward from a current world state perspective. Additionally, video seems like a pretty forward output shape to me - 2D image with a time component. If we were talking 3D assets and animations I wouldn't even know where to start with modeling that as input data for training. That seems really hard to model as a fixed input size problem to me. If there was comparable 3D data available for training, I'd guess that we'd see different issues with different approaches. A couple of examples that I could think of quickly: Using these to build games, might be easier if we could interact with the underlying "assets". Getting photorealistic results with intricate detail (e.g. hair, vegetation) might be easier with video based solutions.
- nosignono 1y agoIf the fidelity of the video is high enough, you could use SFM to build point clouds from the generated video frames and essentially do photogrammatry on the assets from a genie video.
- teamonkey 1y agoI always wonder why they’re not chasing more pragmatic and lower-hanging fruit first. There’s absolutely no reason that a game needs to be generated frame-by-frame like this. It seems like a deeply unserious approach to making games. (My feeling is that it must be easier to train this way.)
- seydor 1y agowell actually image output is fixed and there s lots of training data. Neural networks can learn anything in their latent space so there is no need to impose 3D rendering constraints, and it s not evident that it's less efficient (for the model). 3D model rendering would be useful however for interfacing with robots.
- kouteiheika 1y ago> To that end, we're exploring how we can make Genie 3 available to additional testers in the future. No need to explore; I can tell you how. Release the weights to the general public so that everyone can play with it and non-Google researchers can build their work upon it. Of course this isn't going to happen because "safety". Even telling us how many parameters this model has is "unsafe".
- Dlanv 1y agoModern AI wouldn't exist without Google's contributions. Yet they're a for-profit company. I'm ok with them keeping some things closed source every now and then.
- Davidzheng 1y agoThis is one of the most insane feats of AI I have ever seen to be honest.
- obayesshelton 1y agoStrap on a headset and we are one step closer to being in a simulation.
- artificialprint 1y agoJokes on u, I'm already in a simulation
- sirolimus 1y agoNot open source, not worth it. Next.
- jl6 1y agoHave they explained anywhere what hardware resources it takes to run this in 720p at 24fps with minutes-long context?
- deleted 1y ago[deleted]
- addisonj 1y agoReally impressive... but wow this is light on details. While I don't fully align with the sentiment of other commenters that this is meaningless unless you can go hands on... it is crazy to think of how different this announcement is than a few years ago when this would be accompanied by an actual paper that shared the research. Instead... we get this thing that has a few aspects of a paper - authors, demos, a bibtex citation(!) - but none of the actual research shared. I was discussing with a friend that my biggest concern with AI right now is not that it isn't capable of doing things... but that we switched from research/academic mode to full value extraction so fast that we are way out over our skis in terms of what is being promised, which, in the realm of exciting new field of academic research is pretty low-stakes all things considered... to being terrifying when we bet policy and economics on it. To be clear, I am not against commercialization, but the dissonance of this product announcement made to look like research written in this way at the same time that one of the preeminent mathematicians writing about how our shift in funding of real academic research is having real, serious impact is... uh... not confidence inspiring for the long term.
- dev0p 1y agoThat's completely bonkers. We are making machines dream of explorable, editable, interactable worlds. I wonder how much it costs to run something like this.
- demirbey05 1y agoThis is bad use of AI, we spend our compute to make science faster. I am pretty confident computational cost of this will be maybe 100x of chatgpt query. I don't want to think even environmental effects.
- deleted 1y ago[deleted]
- netdur 1y agoMark Zuckberd must very very upset looking at this, I expect him to throw another billion dollars at google engineers
- idiotsecant 1y agoA Mind needs a few things: The ability to synthesize sensor data about the outside world into a form that can be compressed into important features, the ability to choose which of those features to pay attention to, the ability to model the physical world around it, find reasonable solutions to problems, and simulate its actions before taking them, The ability to understand and simulate the actions of other Minds, the ability to compress events into important features and store them in memory, the ability to retrieve those memories and appropriate times and in appropriate clarity, etc. I feel like as time goes on more and more of these important features are showing up as disconnected proofs of concept. I think eventually we'll have all the pieces and someone will just need to hook them together. I am more and more convinced that AGI is just going to eventually happen and we'll barely notice because we'll get there inch by inch, with more and more amazing things every day.
- superjan 1y agoThere are very few people visible in the demo’s. I suppose that is harder?
- badmonster 1y agoa massive leap forward for real-time world modeling
- Bjorkbat 1y agoGenuinely technically impressive, but I have a weird issue with calling these world simulator models. To me, they're video game simulator models. I've only ever seen demos of these models where things happen from a first-person or 3rd-person perspective, often in the sort of context where you are controlling some sort of playable avatar. I've never seen a demo where they prompted a model to simulate a forest ecology and it simulated the complex interplay of life. Hence, it feels like a video game simulator, or put another way, a simulator of a simulator of a world model.
- Bjorkbat 1y agoAlso, to drive my point further home, in one of the demos they were operating a jetski during a festival. If the jetski bumps into a small Chinese lantern, it will move the lantern. Impressive. However, when the jetski bumped into some sort of floating structure the structure itself was completely unaffected while the jetski simply stopped moving. This is a pretty clear example of video game physics at work. In the real world, both the jetski and floating structure would be much more affected by a collision, but in the context of video game physics such an interaction makes sense. So yeah, it's a video game simulator, not a world simulator.
- rohit89 1y agoThe goal is to eventually be able to model physics and all the various interactions accurately.
- Bjorkbat 1y agoSure, but if you're trying to get there by training a model on video games then you're likely going to wind up inadvertently creating a video game simulator rather than a physics simulator. I don't doubt they're trying to create a world simulator model, I just think they're inadvertently creating a video game simulator model.
- rohit89 1y ago
- phgn 1y agoSo we cannot use this yet? While watching the video I was just imagining the $ increasing by the second. But then it's not available at all yet :(
- ACAVJW4H 1y agoWondering What happens when we peer through a microscope or telescope?
- koakuma-chan 1y agoDamn, this reminds me of those Chinese FMV games on Steam.
- deleted 1y ago[deleted]
- _hark 1y agoVery cool! I've done research on reinforcement/imitation learning in world models. A great intro to these ideas is here: https://worldmodels.github.io/ https://worldmodels.github.io/ I'm most excited for when these methods will make a meaningful difference in robotics. RL is still not quite there for long-horizon, sparse reward tasks in non-zero-sum environments, even with a perfect simulator; e.g. an assistant which books travel for you. Pay attention to when virtual agents start to really work well as a leading signal for this. Virtual agents are strictly easier than physical ones. Compounding on that, mismatches between the simulated dynamics and real dynamics make the problem harder (sim2real problem). Although with domain randomization and online corrections (control loop, search) this is less of an issue these days. Multi-scale effects are also tricky: the characteristic temporal length scale for many actions in robotics can be quite different from the temporal scale of the task (e.g. manipulating ingredients to cook a meal). Locomotion was solved first because it's periodic imo. Check out PufferAI if you're scale-pilled for RL: just do RL bigger, better, get the basics right. Check out Physical Intelligence for the same in robotics, with a more imitation/offline RL feel.
- jimmySixDOF 1y agoWhat gets me is the egocentric perspective it has naturally produced from its training data, where you have the perception of a 3D 6 degrees of freedom world space around you. Once it's running at 90 frames per second and working in a meshed geometry space, this will intersect with augmented virtual XR headsets, and the metaverse will become an interaction arena for working with artificial intelligence using our physical action, our gaze, our location, and a million other points of background noise telemetry, all of which will be integrated into what we now today call context and the response will be adjusting in a useful, meaningful way what we see painted into our environment. Imagine the world as a tangible user interface.
- jimmySixDOF 1y agoaaaannnnddddd ... it's happening: https://x.com/bilawalsidhu/status/1954229425199034753 https://x.com/bilawalsidhu/status/1954229425199034753
- mhitza 1y agoJust imagine if the developers of Star Citizen had access to this technology, how much more they could have squeezed from unsuspecting backers.
- red_hare 1y agoCan you imagine explaining to someone from the 1800s that we've created a fully generative virtual world experience and the demo was "painting a wall blue"
- mattjreid 1y agoThey would be impressed by the paint roller - it wasn't invented until the 1940s.
- sandspar 1y agoThere's an old documentary where a film crew transported some of those "uncontacted tribe" guys to central London. Rather than being in awe of jet engines or networked internet, the tribesmen guys spent most of their time admiring house construction. "You're telling me there's METAL inside these walls?" I guess we tend to appreciate what we can understand.
- scoot 1y agoI'm pretty sure this has been debunked as an online myth, and no such documentary exists. If you can show otherwise, I'd love to watch it
- zarzavat 1y agoThe documentary is real. I don't remember a house construction scene but it's been over 10 years since I watched it. It's not an uncontacted tribe, but tribes from Vanuatu and the Solomon Islands. Meet the Natives: https://www.youtube.com/watch?v=S1DqarK4NlA https://www.youtube.com/watch?v=S1DqarK4NlA There is a second season where a different tribe go to USA.
- SirHumphrey 1y agoReading works of early computer scientists (mathematicians?) like Ada Lovelace or Alan Turing it seems to me that they would be a lot less surprised than some current observers. The idea of artificial mind comes up a lot and they weren't witness to 30 years of slow and uninspiring NLP developments.
- deleted 1y ago[deleted]
- sys32768 1y agoI'm imagining how these worlds combined with AI NPCs could help people learn real-world skills, or overcome serious anxiety disorders, etc.
- forrestthewoods 1y agoThey’re very clever to only turn 90 degrees. I’d like to see a couple of 1080s with a little bit of 120 degree zig zagging along the way please.
- kennyadam 1y agoThe festival of light jetski and the helicopter over cliffs both turn 180+, I think there are one or two more too.
- deleted 1y ago[deleted]
- ralusek 1y agoIt's interesting, because I was always a bit confused and annoyed by the Giant's Drink/Mind Game that Ender plays in Ender's Game. It just always felt so different to how games I knew played, it felt odd that he would "discover" things that the developers hadn't intended, because I always just thought "wait, someone had to build that into the game just in case he happened to do that one specific thing?" Or if it was implied that they didn't do that, then my thought was "that's not how this works, how is it coming up with new/emergent stories?" This feels almost exactly like that, especially the weird/dreamlike quality to it.
- internetter 1y agoI feel like this tech is a dead end. If it could instead generate 3d models which are then rendered, that would be immensely useful. Eliminates memory and playtime constraints, allows it to be embedded in applications like games. But this? Where do we go from here? Even if we eliminate all graphical issues and get latency from 1s to 0, what purpose does it serve?
- calebh 1y agoI think the most likely path forward for commercialization/widespread use is to use AI as a post-processing filter for low poly games. Imagine if you could take low quality/low poly assets, run it through a game engine to add some basic lighting, then pass this through AI to get a photo-realistic image. This solves the most egregious cases of world inconsistency and still allows for creative human fine-tuning. The trick will be getting the post-processor to run at a reasonable frame rate.
- internetter 1y agoDon’t we already have upscalers which are frequently used in games for this purpose? Maybe they could go further and get better but I’d expect a model specifically designed to improve the quality of an existing image to be better/more efficient at doing so than an image generation model retrofitted to this purpose.
- arjie 1y agoThis is beautiful. An incredible device that could expand people's view of history and science. We could create such immersive experiences with this. I know that everyone always worries about trapping people in a simulation of reality etc. etc. but this would have blown my mind as a child. Even Riven was unbelievable to me. I spent hours in Terragen.
- guybedo 1y agoa lot to unpack here, i've added a detailed summary here: https://extraakt.com/extraakts/google-s-genie-3-capabilities-implications-and-future-of-ai https://extraakt.com/extraakts/google-s-genie-3-capabilities...
- jp1016 1y agoThis looks incredibly promising not just for AI research but for practical use cases in game development. Being able to generate dynamic, navigable 3D environments from text prompts could save studios hundreds of hours of manual asset design and prototyping. It could also be a game-changer for indie devs who don’t have big teams. Another interesting angle is retrofitting existing 2D content (like videos, images, or even map data) into interactive 3D experiences. Imagine integrating something like this into Google Maps suddenly street view becomes a fully explorable 3D simulation generated from just text or limited visual data.
- creata 1y agoIt just generates video, though, doesn't it? How are you going to get usable assets out of that?
- CharlieDigital 1y agoWhy wouldn't one be able to train an AI model to extract 3D models/assets out of an image/still from video?
- teamonkey 1y agoThat would be more useful and there are some services that attempt to do that, though I don’t know of any that do it well enough that a human isn’t needed to clean up the mess. Genie 3 isn’t that though. I don’t think it’s actually intended to be used for games at all.
- cranium 1y agoI find the model very impressive, but how could it be used in the wild? They mention robots (maybe to test them cheaply in completely different environments?), but I don't see the use in games except during development to generate ideas/assets.
- muskmusk 1y agoJesus. This is starting to feel pretty **ing exponential.
- SpaceManNabs 1y agoSo are foundational models real finally now? Are they just multimodal for everything? Are foundational time series models included in this category?
- nektro 1y agogoogle pushing new levels of evil with this one
- bluehat974 1y agoIt's feel like Ready Player One on Vision Pro will arrive soon
- kayge 1y agoHopefully they'll skip this[0] part: "...once we can rollback some of Halliday's ad restrictions, we estimate we can sell up to 80% of an individual's visual field before inducing seizures!" [0] https://www.youtube.com/watch?v=KpPE85Jogjw https://www.youtube.com/watch?v=KpPE85Jogjw
- yahoozoo 1y agoWhat format do these world models output? Since it's interactive, it's not just a video...does DeepMind have some kind of proprietary runtime or what?
- guybedo 1y agoit's simulations all the way down
- whatever1 1y agoThis is scary. I don’t have a benchmark to propose but in don’t think my brain can imagine things with greater fidelity than this. I can probably write down the physics better but I think these systems have reached parity with at least my imagination model
- unboxingelf 1y agoThe Simulation Theory presents the following trilemma, one of which must be true: 1. Almost all human-level civilizations go extinct before reaching a technologically mature “posthuman” stage capable of running high-fidelity ancestor simulations. 2. Almost no posthuman civilizations are interested in running simulations of their evolutionary history or beings like their ancestors. 3. We are almost certainly living in a computer simulation.
- lotyrin 1y agoIf you take the idea of it needing to be a constructed simulation you get the dream argument. If you add that one can't verify anyone else having subjective experience you get Boltzmann brain. If you add the idea that maybe the ancestor simulations are designed to teach us virtuous behavior through repeated visits to simulation worlds you get the karmic cycle, and Boltzmann brain + karmic cycle is roughly the egg theory. I think some/all of these things can roughly true at the same time. Imagine an infinite space full of chaotic noise that arises a solitary Boltzmann brain, top level universe and top level intelligence. This brain, seeking purpose and company in the void, dreams of itself in various situations (lower level universes) and some of those universes' societies seek to improve themselves through deliberate construction of karmic cycle ancestor simulation. A hierarchy of self-similar universes. It was incredibly comforting to me to think that perhaps the reason my fellow human beings are so poor at empathy, inclusion, justice, is that this is a karmic kindergarten where we're intended to be learning these skills (and the consequences for failing to perform them) and so of course we're bad at it, it's why we're here.
- crazygringo 1y agoBut there are lots of critiques of that supposed trilemma. Why would beings in simulations be conscious? Or maybe running simulations is really expensive and so it's done sometimes (more than "almost none") but only sometimes (nowhere near "we are almost certainly"). Or simulations are common but limited? You don't need to simulate a universe if all you want to do is simulate a city. The "trilemma" is an extreme example of black-and-white thinking. In the real world, things cost resources and so there are tradeoffs -- so middle grounds are the rule, not extremes.
- lotyrin 1y agoKinda wish the ski scenario had "yeti" as an event you could trigger.
- mason_mpls 1y agoThe claims being made in this announcement are not demonstrated in the video. A very careful first person walk in an AI video isn’t very impressive these days…
- qwertox 1y agoThis is revolutionary. I mean, we already could see this coming, but now it's here. With limitations, but this is the beginning. In game engines it's the engineers, the software developers who make sure triangles are at the perfect location, mapping to the correct pixels, but this here, this is now like a drawing made by a computer, frame by frame, with no triangles computed.
- j_timberlake 1y agoPeople are thinking "how are video games going to use this?" That's not the point, video games are worth chump-change compared to robotics. Training AIs on real-world robotic arms scaled poorly, so they're looking for paths that leverage what AI scales well at.
- maerF0x0 1y agoI'm still struggling to imagine a world where predicting the next pixel wins over over building a deterministic thing that is then ran. Eg: Using AI to generate textures, wire models, motion sequences which themselves sum up to something that local graphics card can then render into a scene. I'm very much not an expert in this space, but to me it seems if you do that, then you can tweak the wire model, the texture, move the camera to wherever you want in the scene etc.
- wolttam 1y agoAt some point it will be computationally cheaper to predict the next pixel than to classically render the scene, when talking about scenes beyond a certain graphical fidelity. The model can infinitely zoom in to some surface and depict(/predict) what would really be there. Trying to do so via classical rendering introduces many technical challenges
- brap 1y agoI imagine a future where the “high level” stuff in the environment is pre defined by a human (with or without assistance from AI), and then AI sort of fills in the blanks on the fly. So for example, a game designer might tell the AI the floor is made of mud, but won’t tell the AI what it looks like if the player decides to dig a 10 ft hole in the mud, or how difficult it is to dig, or what the mud sounds like when thrown out of the hole, or what a certain NPC might say when thrown down the hole, etc.
- deleted 1y ago[deleted]
- lossolo 1y ago> At some point it will be computationally cheaper to predict the next pixel than to classically render the scene, This is already happening to some extent, some games struggle to reach 60 FPS at 4K resolution with maximum graphics settings using traditional rasterization alone, so technologies like DLSS 3 frame generation are used to improve performance.
- Vipitis 1y agoI wish they would share more about how it works. Maybe a reseach paper for once? we didn't even get a technical report. From my best guess: it's a video generation model like the ones we already head. But they condition inputs (movement direction, viewangle). Perhaps they aren't relative inputs but absolute and there is a bit of state simulation going on? [although some demo videos show physics interactions like bumping against objects - so that might be unlikely, or maybe it's 2D and the up axis is generated??]. It's clearly trained on a game engine as I can see screenspace reflection artefacts being learned. They also train on photoscans/splats... some non realistic elements look significantly lower fidelity too.. some inconsistencies I have noticed in the demo videos: - wingsuit discollcusions are lower fidelity (maybe initialized by high resolution image?) - garden demo has different "geometry" for each variation, look at the 2nd hose only existing in one version (new "geometry" is made up when first looked at, not beforehand). - school demo has half a caroutside the window? and a suspiciously repeating pattern (infinite loop patterns are common in transformer models that lack parameters, so they can scale this even more! also might be greedy sampling for stability) - museum scene has odd reflection in the amethyst box, like the rear mammoth doesn't have reflections on the right most side of the box before it's shown through the box. The tusk reflection just pops in. This isn't fresnel effect.
- 3abiton 1y agoI feel after the 2017 transformer paper, and its impact on current state of AI, and google stocks, it seems Google is much more hesistant to keep things under their wings for now. Sadly, so.
- slj 1y agoEveryone is in agreement, this is impressive stuff. Mind blowing, even. But have the good people at Google decided why exactly we need to build the torment nexus?
- OrangeMusic 1y agohuh?
- poszlem 1y agoI suspect he is referencing https://knowyourmeme.com/memes/torment-nexus https://knowyourmeme.com/memes/torment-nexus
- nh23423fefe 1y agohumans make oblique meme references for status. and some imagine that's better content than LLM output
- pedalpete 1y agoWe were working towards this years ago with Doarama/Ayvri, and I remember fondly in 2018 an investor literally yelling at me that I didn't know what I was talking about and AI would never be able to do this. Less than a decade later, here we are. Our product was a virtual 3d world made up of satellite data. Think of a very quick, higher-res version of google earth, but the most important bit was that you uploaded a GPS track and it re-created the world around that space. The camera was always focused on the target, so it wasn't a first person point of view, which, for the most part, our brains aren't very good at understanding over an extended period of time. For those curious about the use case, our product was used by every paraglider in the world, commercial drone operations, transportation infrastructure sales/planning, out-door events promotions (specifically bike and ultramarathon races). Though I suspect we will see a new form of media come from this. I don't pretend to suggest exactly what this media will be, but mixing this with your photos we can see the potential for an infinitely re-framable and zoomable type of photo media. Creating any "watchable" content will be challenging if the camera is not target focused, and it makes it difficult to create a storyline if you can't dictate where the viewer is pointed.
- swalsh 1y agoTo be fair, I'm seeing the demo video, and I still don't believe it's possible. This is sci-fi tech.
- nextfx 1y agoAnother case of moving the goalposts until you score a goal.
- brikym 1y agoThe elephant in the room is the porn capabilities. Onlyfans will be dead in 10 years.
- ProofHouse 1y agoCan anyone specifically working or with expertise in this field, give even a best guest breakdown (or better) of the technology and architecture, system design or possibly even the compute requirement's of how they think this was implemented? Very curious as to how thing works and methods employed, as they are atm tight lipped generally. So kind of curious for those who are specialists in this space what they could surmise or speculate on the implementation of Genie 3
- aussieguy1234 1y ago"developing simulated environments for open-ended learning and robotics" What this means is that a robot model could be trained 1000x faster on GPUs compared to training a robot in the physical world where normal spacetime constraints apply.
- deleted 1y ago[deleted]
- Terretta 1y agoDeeply disillusioned — the cyclist was not a pelican. (See "Exploring locations and historical settings" scene 5.)
- terrytys 1y agoMake this programmable to the details would be a game-changer
- jrmg 1y agoI lost a lot of trust in Google’s AI announcements after being initially amazed an impressed by this debacle: https://arstechnica.com/information-technology/2023/12/google-admits-it-fudged-a-gemini-ai-demo-video-which-critics-say-misled-viewers/ https://arstechnica.com/information-technology/2023/12/googl...
- _giorgio_ 1y agoIt works only with a text prompt? No way to give it an image as a starting point?
- makaking 1y agoThe idea of a possible new AI winter to come seems less likely with each new announcement. Robust world models can be used as an effectively infinite source of training data.
- fedorlong 1y agoI feel like it’s gonna shake up the gaming world big time down the road. And I just made a site to record the impact of it: https://www.genie3ai.app/ https://www.genie3ai.app/
- seydor 1y agoWe need more of these models that can truly understand world building and how the world works. Then we can talk about real AGI
- mdp2021 1y ago> world model Another linguistic devastation. A "world model" is in epistemology the content of a representation of states of thing - all states of things, facts and logic. This use of the expression "world model" seems to be a reduction. An that's too bad, because we needed the idea in its good form to speak about what neural networks contain, in this LLM sub-era. Like the new widespread sloppy use of the expression "AI", this does not contribute to collective mental clarity.
- VeejayRampay 1y agothis is crazy, really cool
- ultimate4448 1y ago[dead]
- rcarmo 1y agoI can't believe it's been 24 hours and nobody mentioned the word "holodeck" :)
- budoso 1y agoThis better be enough of a technological leap for Half Life 3 to come out
- w_for_wumbo 1y agoThe next version of this, paired with a generative sound model + VR. Matched with the tech that already exists that is able to convert thoughts to words, you could pre-empt what someone wants and display it to them in real-time. Allowing them to explore their conscious and unconscious, meeting and interacting and forming deeper relationships with the aspects of themselves that they had forgotten about.
- federico_puddu 1y agohave they released a research paper / do we have details on the architecture and training?
- deleted 1y ago[deleted]
- mgh2 1y agoThis will be great for VR, especially for Vison Pro
- cy2025421 1y ago[dead]
- cy2025421 1y ago[dead]
- moises527 1y ago[dead]
- superfa 1y agoI'm really curious about what scenarios Genie 3 can be applied to. What I can think of are: flow state, meditation, gaming, and creative work like novel or video creation. I've also seen some websites built around this term, like https://www.genie3.fun/ https://www.genie3.fun/