4 ms·

Amazing! I do work in 3D computer vision and I've really wanted to get into 3D generative asset design as a side project at least. If the author or anyone else

Amazing! I do work in 3D computer vision and I've really wanted to get into 3D generative asset design as a side project at least. If the author or anyone else is interested in collaborating, I'd be down to talk and shop something.

gaogao 7y ago
Sure! I've been wanting to incorporate this exact thing for a "And let there be light simulator!" Essentially, people in VR would be able to point and speak to create objects in a shared space.
- heyitsguay 7y ago
  Cool! I've got some Unity VR experience, though i don't know much about incorporating ML pipelines into it. I'm guessing there's resources out there? So basically you'd want speech -> text -> object?
  - gaogao 7y ago
    Exactly!
  - JabavuAdams 7y ago
    I've got some of this working in Python, outside of Unity. Last time I looked at integrating with Unity there seemed to be some Google API requirements that were hard/annoying to meet in my few hours of digging. Should take another look. All the VR HMDs have mics.
- starstorms 7y ago
  This would be really incredible! Could maybe be a way to quickly make video game assets for testing and such. Although to make the meshes performant in a game engine you'd need to decimate them quite a bit as the marching cubes I used here produces really dense meshes.
  - heyitsguay 7y ago
    Do you think the resulting meshes are amenable to auto-decimation tools? Also (directed to the previous poster), while VR applications sound cool, i must admit my interest really lies in improving the asset generation tools themselves. The work done in the original post is super inspiring, I'd love to see a more robust asset generation pipeline built up from tools like that.
JabavuAdams 7y ago
Oh wow, count me in. You could use Google Cloud Speech to go from voice -> text, and then Google Cloud Natural Language to go from text -> simple command interface. I've got both working in Python, but don't do anything with the parsed text. I want to do things like "create sphere" "move that up" "no, bigger" etc.

gaogao 7y ago

Sure! I've been wanting to incorporate this exact thing for a "And let there be light simulator!" Essentially, people in VR would be able to point and speak to create objects in a shared space.

heyitsguay 7y ago
Cool! I've got some Unity VR experience, though i don't know much about incorporating ML pipelines into it. I'm guessing there's resources out there? So basically you'd want speech -> text -> object?
- gaogao 7y ago
  Exactly!
- JabavuAdams 7y ago
  I've got some of this working in Python, outside of Unity. Last time I looked at integrating with Unity there seemed to be some Google API requirements that were hard/annoying to meet in my few hours of digging. Should take another look. All the VR HMDs have mics.
starstorms 7y ago
This would be really incredible! Could maybe be a way to quickly make video game assets for testing and such. Although to make the meshes performant in a game engine you'd need to decimate them quite a bit as the marching cubes I used here produces really dense meshes.
- heyitsguay 7y ago
  Do you think the resulting meshes are amenable to auto-decimation tools? Also (directed to the previous poster), while VR applications sound cool, i must admit my interest really lies in improving the asset generation tools themselves. The work done in the original post is super inspiring, I'd love to see a more robust asset generation pipeline built up from tools like that.

JabavuAdams 7y ago

Oh wow, count me in. You could use Google Cloud Speech to go from voice -> text, and then Google Cloud Natural Language to go from text -> simple command interface. I've got both working in Python, but don't do anything with the parsed text. I want to do things like "create sphere" "move that up" "no, bigger" etc.