4 ms·
Minecraft would be a pretty good medium for finding out the answer to this question actually. Stick a multimodal LLM thats already got language into Minecraft,
by codeulike 4y ago
Minecraft would be a pretty good medium for finding out the answer to this question actually.
Stick a multimodal LLM thats already got language into Minecraft, train it up and leave it to fend for itself (it will need to make shelter, find food, not fall off high things etc).
Then you could use the chat to ask it about its world.
- detrites 4y agoAwesome idea in its own right, I hope someone does it. Reminds me of the GAN Mario AI systems of a few years back: https://youtube.com/watch?v=CI3FRsSAa_U https://youtube.com/watch?v=CI3FRsSAa_U
- luxcem 4y agoMinecraft or any sufficiently complex simulated environment. But I think people will put AI models in physical robot really soon too if it's not already happening.
- codeulike 4y agoBut why not just train using a simulation of a robot in a VR-style environment and run it at 100x the speed?
- whamlastxmas 4y agoThis is what they do already for self driving learning. Also helps train on extreme outlier cases