3 ms·
Related recent project you can try out yourself (Chrome only) which hallucinates new frames of a Minecraft style game: https://oasis.decart.ai/ https://oasis.de
by simonw 2y ago
Related recent project you can try out yourself (Chrome only) which hallucinates new frames of a Minecraft style game: https://oasis.decart.ai/ https://oasis.decart.ai/
That one would reimagine the world any time you look at the sky or ground. Sounds like Genie2 solves that: "Genie 2 is capable of remembering parts of the world that are no longer in view and then rendering them accurately when they become observable again."
- echelon 2y agoThis blows Decart's Oasis (which raised $25 million at $500 million valuation) and World Labs (which raised $230 million in complete stealth) out of the water. Google is firing warning shots to kill off interest in funding competing startups in this space. I suspect that in 6 months it won't matter as we'll have completely open source Chinese world models. They're already starting to kill video foundation model companies' entire value prop by releasing open models and weights. Hunyuan blows Runway and OpenAI's Sora completely out of the water, and it's 100% open source. How do companies like Pika compete with free? Meta and Chinese companies are not the leaders in the space, so they're salting the earth with insanely powerful SOTA open models to prevent anyone from becoming a runaway success. Meta is still playing its cards close to its chest so they can keep the best pieces private, but these Chinese companies are dropping innovation left and right like there's no tomorrow. The game theory here is that if you're a foundation model "company", you're dead - big tech will kill you. You don't have a product, and you're paying a lot to do research that isn't necessarily tied to customer demand. If you're a leading AI research+product company, everyone else will release their code/research to create a thousand competitors to you.
- Workaccount2 2y agoI strongly suspect that like open ai and O1, for profit companies are going to start locking down whatever advances they find. There is still an enormous amount of long hanging fruit that anyone can harvest right now, but eventually big advances are going to require big budgets and I can only imagine how technically tight lipped they will be with those.
- senko 2y ago> The game theory here is that if you're a foundation model "company", you're dead - big tech will kill you. You don't have a product, and you're paying a lot to do research that isn't necessarily tied to customer demand. Basically, the foundation model companies are outsourced R&D labs for big tech. They can be kept at arms length (like OpenAI with Microsoft and Anthropic with Amazon) or be bought outright (like Inflection, although that was a weird one). Both OpenAI and Anthropic are trying to move away from being pure model companies. > If you're a leading AI research+product company, everyone else will release their code/research to create a thousand competitors to you. Trillion dollar question - is there a competitive edge / moat in vertical integration in AI? Apple proved there was in hardware + os (which were unbundled in wintel times). For AI, right now, I can't see one, but I'm just a random internet comentator, who knows.
- refulgentis 2y agoI think not, it feels more like a utility to me until someone pulls their API.
- mrandish 2y ago> Chinese companies are not the leaders in the space, so they're salting the earth with insanely powerful SOTA open models to prevent anyone from becoming a runaway success. While it would be interesting if Chinese companies were releasing their best full models as an intentional strategy to reduce VC funding availability for western AI startups, it would be downright fascinating if the Chinese government was supporting this as a broader geopolitical strategy to slow down the West. It does make sense but would require a remarkable level of insight, coordination and commitment to a costly yet uncertain strategy.
- whiplash451 2y agoI don't think it requires a remarkable level of insight. The overall cost for the Chinese government is probably very small in the grand scheme of things. And it makes a lot of sense from a geopolitical strategy.
- whiplash451 2y agoThe game has indeed become brutal for foundational model companies. I am less worried for AI research+product companies: they have likely secured revenue streams with real customers and built domain knowledge in the meantime.
- ilaksh 2y agoThere is another recent project that is more general game generation very similar to Genie 2. I can't remember the name. GameGen-X came out last month. https://arxiv.org/html/2411.00769v1 https://arxiv.org/html/2411.00769v1
- psb217 2y agoRE: "Genie 2 is capable of remembering parts of the world that are no longer in view and then rendering them accurately when they become observable again." -- This claim is almost certainly wildly misleading. This claim is technically true if there's any scenario where their agent, eg, briefly looked down at the ground and then back up at the sky and at least one of the clouds in the sky was the same as before looking down. However, I expect most people will interpret the claim far more broadly than the model can support. It's classic weasel wording.
- pfortuny 2y ago"remember parts of the world..." not even "some"... That is a tell-tale.
- isotypic 2y agoLooking at how no samples other than the 3 samples in the "Long horizon memory" section have any camera movement which puts something offscreen and then back onscreen, it certainly seems that they are stretching the capabilities as far as they can in writing.
- drusepth 2y agoYeah, my best guess is they're probably including the previous N frames as context into generating the next model. This works to preserve continuity over a short amount of time (as you say, briefly looking at the ground and then back up), but only a short period of time. For these kinds of models to be "playable" by humans (and, I'd argue, most fledgling AI agents), the world state needs to be encoded in the context, not just a visual representation of what the player most recently saw.
- wongarsu 2y agoHowever the architecture they describe really sounds like it should still have that issue. I doubt they really solved it. Which is a big problem for the agent-training use case they keep reiterating on the website. Agents are like speedrunners: if there is a stupid exploit, the agent will probably find and use it. And for Oasis the speedrunning meta for getting to the nether is to find anything red, make it fill the screen, and repeat until the world-generating AI thinks you look at lava and must be in the nether