3 ms·
I find this extremely disappointing. A diffusion transformer trained on Minecraft frames and accelerated on an ASIC... Okay? From the demo(that doesn't work on
by gessha 2y ago
I find this extremely disappointing. A diffusion transformer trained on Minecraft frames and accelerated on an ASIC... Okay?
From the demo(that doesn't work on Firefox) you can see that it's overfit to the training set and it doesn't have a consistent state transition.
If you define it as a Markov decision process with states being images, actions being keyboard/mouse inputs, the probability transition being the transformer model, the model is a very poor one. Turning the mouse around shouldn't result in a completely different world, it should result in the exact same point of space from different camera orientation. You can fake it by fudging with the training data and augmenting with walking a bit, doing a 360 camera rotation and continuing the exploration but that will just overfit to that specific seed.
The page says their ASICs model inference supports 60+ players. Where are they shown playing together? What's the point of touting multiplayer performance when realistically, the poor state transition will mean those 60+ players are playing single player DeepDream Minecraft?