4 ms·

Any estimates of how much one of these cost to generate and keep a minute of context? Secondly, any estimate of how much the price could fall in 5-10 years?

wongarsu 2y ago
Oasis (the Minecraft world model) can serve about 5 players on 8 H100 in real-time at 20fps in 360p. This is a much more capable model with two orders of magnitude more context. They pretty much say it can't be played real-time, which I read as they generate less than 15fps@240p on 8 GPUs. Probably why they talk so much about using it for AI training and evaluation rather than human use. There is a distilled version that works in real-time, but they don't show anything from that version (which is a statement in itself). For reducing the price, ASICs like etched may be the way forwards [1]. The models will get bigger for a time, but there may be a lot of room for models that can exploit purpose-built hardware. 1: https://www.etched.com https://www.etched.com
- onlyrealcuzzo 2y ago
  > Probably why they talk so much about using it for AI training and evaluation rather than human use. What would they do / how would they use this output to make a better AI?
  - bionhoward 2y ago
    Embodied cognition is a core theory for AGI; this would enable a vast array of bodies, environments, and situations, that high level of diversity can empower AI adaptability. For a straightforward example, this could help Waymo rehearse driving in various cities and weather / traffic settings
    - Rastonbury 2y ago
      Not meaning to pick at that example but a broader question the value of these, what use cases outside of games are they willing to let AI that is meant to interact with the real world be trained on AI synthetic data, that is like black box on black box, double the training and inference cost Even in games I expect a game playing model to exploit glitches present in world building one I think it's great that Google is researching these, but I can't see the return and if there is it is many steps away
      - nine_k 2y ago
        I bet the military is keenly interested.
- latchkey 2y ago
  Hey! I'd love to know how this performs on 8xMI300x in comparison. Reach out to me?
llm_trw 2y ago
The price of LLMs has fallen 1,000 times in the last year for the same quality tokens. It's not clear if video models will follow the same trajectory.
- Rastonbury 2y ago
  I saw a demo of stable diffusion work so fast the images change as you type
reissbaker 2y ago
They don't give much info on parameter count, etc so it's hard to say concretely: Oasis (AI Minecraft) apparently runs on a single H100 [1], but this is presumably much larger — both due to higher fidelity, and due to the 60s context window instead of 1s context window for Oasis. But in 5-10 years regardless of what it takes to run now, the price will drop massively, and my bet is this would be playable in real-time. Context length will be solvable simply by increased VRAM (i.e. an H200 has 141GB per GPU, vs 80GB for an H100). Although Google is probably running these on TPUs, TPUs should follow a similar trajectory. In the intermediate term my guess is that this kind of world model will be useful for training 3D model generators, so that you can go from sketch -> running in-engine extremely quickly. 1: https://www.tweaktown.com/news/101466/oasis-ai-and-single-nvidia-h100-gpu-has-created-playable-minecraft-at-720p-20-fps/index.html https://www.tweaktown.com/news/101466/oasis-ai-and-single-nv...

wongarsu 2y ago

Oasis (the Minecraft world model) can serve about 5 players on 8 H100 in real-time at 20fps in 360p. This is a much more capable model with two orders of magnitude more context. They pretty much say it can't be played real-time, which I read as they generate less than 15fps@240p on 8 GPUs. Probably why they talk so much about using it for AI training and evaluation rather than human use. There is a distilled version that works in real-time, but they don't show anything from that version (which is a statement in itself). For reducing the price, ASICs like etched may be the way forwards [1]. The models will get bigger for a time, but there may be a lot of room for models that can exploit purpose-built hardware. 1: https://www.etched.com https://www.etched.com

onlyrealcuzzo 2y ago
> Probably why they talk so much about using it for AI training and evaluation rather than human use. What would they do / how would they use this output to make a better AI?
- bionhoward 2y ago
  Embodied cognition is a core theory for AGI; this would enable a vast array of bodies, environments, and situations, that high level of diversity can empower AI adaptability. For a straightforward example, this could help Waymo rehearse driving in various cities and weather / traffic settings
  - Rastonbury 2y ago
    Not meaning to pick at that example but a broader question the value of these, what use cases outside of games are they willing to let AI that is meant to interact with the real world be trained on AI synthetic data, that is like black box on black box, double the training and inference cost Even in games I expect a game playing model to exploit glitches present in world building one I think it's great that Google is researching these, but I can't see the return and if there is it is many steps away
    - nine_k 2y ago
      I bet the military is keenly interested.
latchkey 2y ago
Hey! I'd love to know how this performs on 8xMI300x in comparison. Reach out to me?

llm_trw 2y ago

The price of LLMs has fallen 1,000 times in the last year for the same quality tokens. It's not clear if video models will follow the same trajectory.

Rastonbury 2y ago
I saw a demo of stable diffusion work so fast the images change as you type

reissbaker 2y ago

They don't give much info on parameter count, etc so it's hard to say concretely: Oasis (AI Minecraft) apparently runs on a single H100 [1], but this is presumably much larger — both due to higher fidelity, and due to the 60s context window instead of 1s context window for Oasis. But in 5-10 years regardless of what it takes to run now, the price will drop massively, and my bet is this would be playable in real-time. Context length will be solvable simply by increased VRAM (i.e. an H200 has 141GB per GPU, vs 80GB for an H100). Although Google is probably running these on TPUs, TPUs should follow a similar trajectory. In the intermediate term my guess is that this kind of world model will be useful for training 3D model generators, so that you can go from sketch -> running in-engine extremely quickly. 1: https://www.tweaktown.com/news/101466/oasis-ai-and-single-nvidia-h100-gpu-has-created-playable-minecraft-at-720p-20-fps/index.html https://www.tweaktown.com/news/101466/oasis-ai-and-single-nv...