5 ms·

That makes sense. I think there's opportunity for generative ML to eventually help here. An open dataset of (images, description) -> 3d model would go a long

That makes sense. I think there's opportunity for generative ML to eventually help here. An open dataset of (images, description) -> 3d model would go a long way. Check out this paper on using GANs to generate voxel-based models: http://3dgan.csail.mit.edu/ http://3dgan.csail.mit.edu/ I've been studying and working with GANs for about a year now. They are still very exciting, and I'd love to try to expand my codebase to new types of data. Additionally, there are some recent techniques that haven't been tried with voxel-based renderings. Perhaps there is another algorithm that can help go from voxel -> polygons as well. I think with the right tech, time, and execution this could be a matter of: 1. Take a picture 2. Generate until you get the 3d model you want

AndrewKemendo 10y ago
That's the exact approach that I think is going to work. Well, not exact cause I don't like their voxel building generation method. I think a GAN + Procedural Generator is the winner. edit: Let me know if you want to work on this cause it's an active area of research for us. See my HN profile for contact.
- ahrasheed 10y ago
  Can you clarify what you mean by Procedural Generator? Isn't a generative model already a procedural generator? Its just that a model generated in case of the referred paper is voxel based. Did you mean, generate parameters of a pre-specified model e.g https://graphics.ethz.ch/~edibra/Publications/HS-Nets%20-%20Estimating%20Human%20Body%20Shape%20from%20Silhouettes%20with%20Convolutional%20Neural%20Networks.pdf https://graphics.ethz.ch/~edibra/Publications/HS-Nets%20-%20... , although this paper is just learning a regression to the human body model (not using GANs). Curious to know more about your train of thought. I am working as a researcher in the domain and thinking of experimenting with GANs for 3D model estimation using similar inputs as the one in the paper I referred to.
dharma1 10y ago
I think generative models could work but both 2d GAN and 3D voxel GAN are very low res

AndrewKemendo 10y ago

That's the exact approach that I think is going to work. Well, not exact cause I don't like their voxel building generation method. I think a GAN + Procedural Generator is the winner. edit: Let me know if you want to work on this cause it's an active area of research for us. See my HN profile for contact.

ahrasheed 10y ago
Can you clarify what you mean by Procedural Generator? Isn't a generative model already a procedural generator? Its just that a model generated in case of the referred paper is voxel based. Did you mean, generate parameters of a pre-specified model e.g https://graphics.ethz.ch/~edibra/Publications/HS-Nets%20-%20Estimating%20Human%20Body%20Shape%20from%20Silhouettes%20with%20Convolutional%20Neural%20Networks.pdf https://graphics.ethz.ch/~edibra/Publications/HS-Nets%20-%20... , although this paper is just learning a regression to the human body model (not using GANs). Curious to know more about your train of thought. I am working as a researcher in the domain and thinking of experimenting with GANs for 3D model estimation using similar inputs as the one in the paper I referred to.

dharma1 10y ago

I think generative models could work but both 2d GAN and 3D voxel GAN are very low res