4 ms·

> improving on real world physics and understanding Can't help but flashback to when Sora was first previewed and the hype crew were claiming it was a world ph

> improving on real world physics and understanding Can't help but flashback to when Sora was first previewed and the hype crew were claiming it was a world physics engine and now when it finally shipped months and months later no one uses it and it didn't deliver on any of the hype at all.

deleted 2y ago
[deleted]
torginus 2y ago
I wonder if it would be possible to create an AI image/video editor very much like blender or photoshop - where you get to drag objects around, perhaps based on a transformer(?) model, where each token(s) encode an object similarly how a 3D game engine encodes a game object - like latent vectors matrix rows that correspond to position in world space, size, texture, etc. The 'renderer' would be a neural net that would take this soup of tokens, and resolve it to a 2d frame. The underlying logic engine would be a human one - or perhaps a traditional video game engine, that emits the tokens from which the context can be built up and decoded into an image.
- deleted 2y ago
  [deleted]
danielbln 2y ago
No-one uses it because there are better and cheaper and faster and more open models out there. Had they released it right after the announcement they could have surfed the hype wave. And of course do these models simulate the world to some degree, they have to. It's not a universe simulator to be sure, but stuff like gravity, momentum, etc. is kind of inferred,
- echelon 2y ago
  Kling, Hailuo Minimax, and Google Veo are the best tools. Kling is the leader by an extremely wide margin. Wan video is a brand new open weights model that is developing a nice research and tooling ecosystem around it. Netflix has been playing with it. Further research and tooling will be built on top of it for sure. Sora is garbage (currently [1]) and Runway, Luma, and Pika are mid. Pika kind of left the race to focus on social media content and memes. There are a bunch of other stealth / research foundation models that will be rolling out soon. It's a hyper competitive market and consumers are going to win. [1] Given the insane capabilities of the just released OpenAI 4o image gen, I expect the next Sora to be impressive.

deleted 2y ago

[deleted]

torginus 2y ago

I wonder if it would be possible to create an AI image/video editor very much like blender or photoshop - where you get to drag objects around, perhaps based on a transformer(?) model, where each token(s) encode an object similarly how a 3D game engine encodes a game object - like latent vectors matrix rows that correspond to position in world space, size, texture, etc. The 'renderer' would be a neural net that would take this soup of tokens, and resolve it to a 2d frame. The underlying logic engine would be a human one - or perhaps a traditional video game engine, that emits the tokens from which the context can be built up and decoded into an image.

deleted 2y ago
[deleted]

danielbln 2y ago

No-one uses it because there are better and cheaper and faster and more open models out there. Had they released it right after the announcement they could have surfed the hype wave. And of course do these models simulate the world to some degree, they have to. It's not a universe simulator to be sure, but stuff like gravity, momentum, etc. is kind of inferred,

echelon 2y ago
Kling, Hailuo Minimax, and Google Veo are the best tools. Kling is the leader by an extremely wide margin. Wan video is a brand new open weights model that is developing a nice research and tooling ecosystem around it. Netflix has been playing with it. Further research and tooling will be built on top of it for sure. Sora is garbage (currently [1]) and Runway, Luma, and Pika are mid. Pika kind of left the race to focus on social media content and memes. There are a bunch of other stealth / research foundation models that will be rolling out soon. It's a hyper competitive market and consumers are going to win. [1] Given the insane capabilities of the just released OpenAI 4o image gen, I expect the next Sora to be impressive.