4 ms·
This is a great example of how we (collectively) are using virtual worlds as "dreams" for Deep learning. In the Deep Vision world we as a group are trying to s
by AndrewKemendo 10y ago
This is a great example of how we (collectively) are using virtual worlds as "dreams" for Deep learning.
In the Deep Vision world we as a group are trying to segment, classify and reinforce our NN training on labeled real world data. The challenge is, it's a very manual process to label data - specifically images. The more that we can do inside the computer, for example automatically labeling pixels inside an image, without having to acquire and label real world data (or making it easier to do with real world data) the easier and faster training becomes that can be applied to real world use cases.
The trick is making the virtual world match the real world as closely as possible so that the nets we make are accurate representations of real world scenarios.
- bonoboTP 10y agoIt's not just for deep learning, it's for any computer vision research. Synthetic datasets have been used in the past too, but this makes it much more accessible.
- AndrewKemendo 10y agoIndeed... However CV is becoming more and more Deep Net based.
- deleted 10y ago[deleted]
- alex_hirner 10y agoIn order to transfer learnings from virtual worlds to the real world more efficiently, I imagine a module that translates real world footed to an abstract representation that is the same as for rendered footage. In other words, you would train a NN on input from a footage of a walkthrough of a real building to output rendered footage from the same path (3D data from architects is already there). The middle layer of this quasi autoencoder then is the basis to train fully simulated tasks, e.g. autonomous vehicles. In a way, it would be similar to colorization of b/w footage. Would that scale training data?
- AndrewKemendo 10y agoYea, I mean that is basically what we do with our home furnishings app - except we have to SFM to build the models. The challenge is labeling - or autolabeling pixels. One thing we are trying to work out is how do you label and build nets on volumes - rather than just pixels? I'm thinking it's going to be a magnitude harder.
- justinlardinois 10y agoI was surprised, since I didn't think it would be realistic enough for this purpose, but Grand Theft Auto V is being used in this manner to train self-driving cars. https://www.technologyreview.com/s/602317/self-driving-cars-can-learn-a-lot-by-playing-grand-theft-auto/ https://www.technologyreview.com/s/602317/self-driving-cars-...
- monatron 10y agoI find this incredibly interesting. Do you have any more information on the subject you can point me to?
- AndrewKemendo 10y agoBest thing is to probably search around for "Synthetic Training" Here are two examples of some of the research done in this area: http://www.cv-foundation.org/openaccess/content_iccv_2015/papers/Peng_Learning_Deep_Object_ICCV_2015_paper.pdf http://www.cv-foundation.org/openaccess/content_iccv_2015/pa... https://arxiv.org/pdf/1503.03163.pdf https://arxiv.org/pdf/1503.03163.pdf
- qiuwch 10y agoI created a project https://github.com/qiuwch/synthetic-computer-vision https://github.com/qiuwch/synthetic-computer-vision trying to summarize works that used synthetic data for computer vision. Might be worth to have a look.
- emanuelev 10y agoYou might want to add the ICL-NUIM dataset for SLAM/Visual Odometers: https://www.doc.ic.ac.uk/~ahanda/VaFRIC/iclnuim.html https://www.doc.ic.ac.uk/~ahanda/VaFRIC/iclnuim.html