4 ms·

This Sora 2 generation of Cyberpunk 2077 gameplay managed to reproduce it extremely closely, which is baffling: https://x.com/elder_plinius/status/1973124528680

This Sora 2 generation of Cyberpunk 2077 gameplay managed to reproduce it extremely closely, which is baffling: https://x.com/elder_plinius/status/1973124528680345871 https://x.com/elder_plinius/status/1973124528680345871 > How the FUCK does Sora 2 have such a perfect memory of this Cyberpunk side mission that it knows the map location, biome/terrain, vehicle design, voices, and even the name of the gang you're fighting for, all without being prompted for any of those specifics?? > Sora basically got two details wrong, which is that the Basilisk tank doesn't have wheels (it hovers) and Panam is inside the tank rather than on the turret. I suppose there's a fair amount of video tutorials for this mission scattered around the internet, but still––it's a SIDE mission! Everyone already assumed that Sora was trained on YouTube, but "generate gameplay of Cyberpunk 2077 with the Basilisk Tank and Panam" would have generated incoherent slop in most other image/video models, not verbatim gameplay footage that is consistent. For reference, this is what you get when you give the same prompt to Veo 3 Fast (trained by the company that owns YouTube): https://x.com/minimaxir/status/1973192357559542169 https://x.com/minimaxir/status/1973192357559542169

Klonoar 1y ago
> Everyone already assumed that Sora was trained on YouTube Doesn't this already answer your question...? "Let's Play" type videos and streams have been a thing for years now, even for more obscure games. It very well could've been trained on Cyberpunk videos of that mission.
- minimaxir 1y ago
  It's hard for me to believe that the model coherently memorized both the video and audio of a relatively obscure Let's Play, and that a simple prompt was enough to surface it (the use of the term "Basilisk tank" would also likely not be in video metadata either). That is the reason the person who made that tweet, who has far more prompting experience than myself, was shocked.
  - Klonoar 1y ago
    It’s hard for you to believe, sure, and I recognize the context of who tweeted it. I still maintain that’s the kernel it’s getting it from. It’s impressive, I’m just not really shocked by it as a concept.
mxwsn 1y ago
That's really interesting. What if they RAG search related videos from the prompt, and condition on that to generate? That might explain fidelity like this
- minimaxir 1y ago
  An interesting counterexample is "a screen recording of the boot screen and menus for a user playing Mario Kart 64 on the N64, they play a grand prix and start to race" where the UI flow matches the real Mario Kart 64, but the UI itself is wrong: https://x.com/fofrAI/status/1973151142097154426 https://x.com/fofrAI/status/1973151142097154426
  - suddenlybananas 1y ago
    I like the player being in "1th" while being behind everyone else. Still crazy though.

Klonoar 1y ago

> Everyone already assumed that Sora was trained on YouTube Doesn't this already answer your question...? "Let's Play" type videos and streams have been a thing for years now, even for more obscure games. It very well could've been trained on Cyberpunk videos of that mission.

minimaxir 1y ago
It's hard for me to believe that the model coherently memorized both the video and audio of a relatively obscure Let's Play, and that a simple prompt was enough to surface it (the use of the term "Basilisk tank" would also likely not be in video metadata either). That is the reason the person who made that tweet, who has far more prompting experience than myself, was shocked.
- Klonoar 1y ago
  It’s hard for you to believe, sure, and I recognize the context of who tweeted it. I still maintain that’s the kernel it’s getting it from. It’s impressive, I’m just not really shocked by it as a concept.

mxwsn 1y ago

That's really interesting. What if they RAG search related videos from the prompt, and condition on that to generate? That might explain fidelity like this

minimaxir 1y ago
An interesting counterexample is "a screen recording of the boot screen and menus for a user playing Mario Kart 64 on the N64, they play a grand prix and start to race" where the UI flow matches the real Mario Kart 64, but the UI itself is wrong: https://x.com/fofrAI/status/1973151142097154426 https://x.com/fofrAI/status/1973151142097154426
- suddenlybananas 1y ago
  I like the player being in "1th" while being behind everyone else. Still crazy though.