3 ms·

What I want to see is a model that can generate 3D models for use in applications such as Blender. It would provide a good starting point for someone with talen

by j_m_b 4y ago

What I want to see is a model that can generate 3D models for use in applications such as Blender. It would provide a good starting point for someone with talent to make beautiful. Or just save people like me time for making games.

deleted 4y ago
[deleted]
sphars 4y ago
I've been looking into some of the 3D model generators this past week, and there is some work happening in that field. See the following non-exhaustive list: https://github.com/snap-research/NeROIC https://github.com/snap-research/NeROIC https://github.com/threedle/text2mesh https://github.com/threedle/text2mesh https://github.com/AutodeskAILab/Clip-Forge https://github.com/AutodeskAILab/Clip-Forge https://nv-tlabs.github.io/GET3D/ https://nv-tlabs.github.io/GET3D/
- castlecrasher2 4y ago
  Have you by chance tried out NeROIC? I'm a 3D printing enthusiast, mostly video game stuff, and it seems like like it would be excellent for that purpose.
  - sphars 4y ago
    I actually have been trying it out this week, and in fact it's currently trying to process the video generation, like their example shows. While I was able to follow their steps for training using their dataset, and generate the lighting/depth maps for the milkcarton example, the video generation is taking a long time (over 24 hours so far, using a 3070Ti with 8GB VRAM). From what I understand with NeROIC, it's not particularly meant to be able to generate an 3D model that can be imported into Blender (or other software). It requires more work to take the meshes it generates to do something with it. See https://github.com/snap-research/NeROIC/issues/10 https://github.com/snap-research/NeROIC/issues/10 I too was looking into it to generate 3D models for some software I've been working on.
    - castlecrasher2 4y ago
      Thanks! I imagine one could use something like ZBrush's dynamesh to create a usable mesh from the output. Shame the library doesn't provide it by default, though.
videlov 4y ago
I recently saw a company that seems to be doing just that[0], but I’m not sure if their model is open source [0] https://www.kaedim3d.com/ https://www.kaedim3d.com/
appleflaxen 4y ago
Agree! And it seems like this should be the easiest thing to train! Let the AI generate random blender files. Render them. Train on source => render mapping (which is 1:many) Repeat. You should be able to get an incredibly high-fidelity decoder to go from image to blender source.
mbel 4y ago
Here's an attempt at doing that: https://dreamfusionpaper.github.io/ https://dreamfusionpaper.github.io/
masterspy7 4y ago
Funnily enough, you might not even need a 3D model if the txt2video is good enough. Whatever you wanted to render in Blender could just be rendered via text prompt (when this becomes 100x better).
cercatrova 4y ago
> 3D Then you're in luck today as this was just submitted: DreamFusion: Text-to-3D using 2D Diffusion https://news.ycombinator.com/item?id=33025446 https://news.ycombinator.com/item?id=33025446

by j_m_b 4y ago

deleted 4y ago

[deleted]

sphars 4y ago

I've been looking into some of the 3D model generators this past week, and there is some work happening in that field. See the following non-exhaustive list: https://github.com/snap-research/NeROIC https://github.com/snap-research/NeROIC https://github.com/threedle/text2mesh https://github.com/threedle/text2mesh https://github.com/AutodeskAILab/Clip-Forge https://github.com/AutodeskAILab/Clip-Forge https://nv-tlabs.github.io/GET3D/ https://nv-tlabs.github.io/GET3D/

castlecrasher2 4y ago
Have you by chance tried out NeROIC? I'm a 3D printing enthusiast, mostly video game stuff, and it seems like like it would be excellent for that purpose.
- sphars 4y ago
  I actually have been trying it out this week, and in fact it's currently trying to process the video generation, like their example shows. While I was able to follow their steps for training using their dataset, and generate the lighting/depth maps for the milkcarton example, the video generation is taking a long time (over 24 hours so far, using a 3070Ti with 8GB VRAM). From what I understand with NeROIC, it's not particularly meant to be able to generate an 3D model that can be imported into Blender (or other software). It requires more work to take the meshes it generates to do something with it. See https://github.com/snap-research/NeROIC/issues/10 https://github.com/snap-research/NeROIC/issues/10 I too was looking into it to generate 3D models for some software I've been working on.
  - castlecrasher2 4y ago
    Thanks! I imagine one could use something like ZBrush's dynamesh to create a usable mesh from the output. Shame the library doesn't provide it by default, though.

videlov 4y ago

I recently saw a company that seems to be doing just that[0], but I’m not sure if their model is open source [0] https://www.kaedim3d.com/ https://www.kaedim3d.com/

appleflaxen 4y ago

Agree! And it seems like this should be the easiest thing to train! Let the AI generate random blender files. Render them. Train on source => render mapping (which is 1:many) Repeat. You should be able to get an incredibly high-fidelity decoder to go from image to blender source.

mbel 4y ago

Here's an attempt at doing that: https://dreamfusionpaper.github.io/ https://dreamfusionpaper.github.io/

masterspy7 4y ago

Funnily enough, you might not even need a 3D model if the txt2video is good enough. Whatever you wanted to render in Blender could just be rendered via text prompt (when this becomes 100x better).

cercatrova 4y ago

> 3D Then you're in luck today as this was just submitted: DreamFusion: Text-to-3D using 2D Diffusion https://news.ycombinator.com/item?id=33025446 https://news.ycombinator.com/item?id=33025446