It seems like this stuff would work better if they combined things like 3d modeling. (maybe built on Nurbs or CSG or something) with the deep learning and also

It seems like this stuff would work better if they combined things like 3d modeling. (maybe built on Nurbs or CSG or something) with the deep learning and also just tried to do less in one step or even with one model. Or maybe it's a more flexible neural model at the core but it's still compositional and more precisely trained on constituent parts. Train the system to nail down core concepts first and then build off of that. Not saying it's easy to do that. But take as an example a robot operating indoors. You may want it to do many domestic tasks and so have a degree of generalization, but it doesn't need to be able to model landscape scenery or birds. And the environment has quite a lot of regular shapes in it since it's man made. With so many regular shapes, flat walls, rectangles and straight edges, the vision system should first be able to nail the representation of that. Once you have a solid understanding of indoor 3d reconstruction, the structures used there can be leveraged for things like imagining described scenarios.