3 ms·
It'd be great if you could use something like this for quick-and-dirty 3D asset generation, but is there any way to convert neural radiance fields to meshes? It
by heyitsguay 6y ago
It'd be great if you could use something like this for quick-and-dirty 3D asset generation, but is there any way to convert neural radiance fields to meshes? It'd be cool if you could take a photo or two of an object and get a 3d mesh out quickly.
- faeyanpiraat 6y agoSoftware already exists for this purpose. You have to take photos of an object from all angles (more the better), and a mesh is generated. I haven’t had much success with it though, that technology certainly has room for improvement.
- heyitsguay 6y agoRight, I'm aware of the structure from motion stuff that's already out there and its large view requirement. I've had a similar experience. Hence my question about using something like PixelNeRF instead.
- Fronzie 6y agoApparantly the good old marching cubes can be used for that: https://www.matthewtancik.com/nerf https://www.matthewtancik.com/nerf shows a video about halfway on the page.
- heyitsguay 6y agoNice find! Time to put together a pipeline... (1) take a few photos of a thing you want to model, (2) use a segmentation algorithm to mask out the background, (3) run a NeRF algorithm, (4) use marching cubes to generate a mesh.
- harperlee 6y ago(5) use the volume as a prior for a probabilistically programmed model of known objects (a la picture [1]); (6) render a model scene in blender; (7) use the difference between the modeled scene and reality to guide attention to the robot moving the camera; (8) make the robot arm poke the thing, move it, rotate it, crash it; (...); (?) Terminator. [1] https://github.com/mrkulk/MIT-Picture https://github.com/mrkulk/MIT-Picture