4 ms·
I was thinking about this before and I thought of having a 3d physics engine that the AI could create objects in and simulate things to see their physical viabi
by darepublic 4y ago
I was thinking about this before and I thought of having a 3d physics engine that the AI could create objects in and simulate things to see their physical viability. Could also help with question answering that requires that spatial knowledge / real world simulation.
- zone411 4y agoThis has been a thing for a while. For example, here are a couple random papers from 2017: https://openaccess.thecvf.com/content_cvpr_2017/html/Varol_Learning_From_Synthetic_CVPR_2017_paper.html https://openaccess.thecvf.com/content_cvpr_2017/html/Varol_L..., https://openaccess.thecvf.com/content_ICCV_2017_workshops/w23/html/Abbasnejad_Using_Synthetic_Data_ICCV_2017_paper.html https://openaccess.thecvf.com/content_ICCV_2017_workshops/w2... or a newer one about deformable objects: https://arxiv.org/abs/2107.08898 https://arxiv.org/abs/2107.08898. You can even use a robot to manipulate things in real life to create synthetic data for a neural net.
- ilaksh 4y agoYes I think that the latest in ML everything else will help to create those traditional simulations. But also I think that what an AI that can for example really answer questions about a video would need to do to be really effective would be to basically do compressed versions of those simulations using the spatial-temporal-abstract latent space. Which should be a better model than just the textual space.