4 ms·

Neuralangelo: High-Fidelity Neural Surface Reconstruction

84 points by bx376 3y ago

rallyforthesun 3y ago
i wonder if this produces better results than Reality Capture?
lucubratory 3y ago
So it's like photogrammetry, but you can do it with a phone video and it's significantly higher quality than previous implementations before cleanup. I wasn't able to find the compute requirements on the page, hopefully not too onerous. Something like this could be significantly beneficial for 3D asset production for games if there's less cleaning up of the photogrammetry models by hand, and for films it might actually be even better if they just remove the cleanup process altogether for small CGI detail in backgrounds where this sort of thing wouldn't be distinguishable from a photoreal model.
- pdc56 3y ago
  I found compute requirements in the github issue tracker: https://github.com/NVlabs/neuralangelo/issues/4 https://github.com/NVlabs/neuralangelo/issues/4 Not good, kinda scary actually
  - danielEM 3y ago
    But these are requirements to train ANN and I guess the question was about requirements to process video stream.
    - lucubratory 3y ago
      Yes, only interested in the inference cost.
  - junon 3y ago
    Training and usage are two different beasts. Training is always going to be expensive, but most users aren't going to do training themselves.
q_andrew 3y ago
Pretty amazing results. It would be fun to work as a traveling 3D capture artist, going to novel locations and scanning assets to use in fantasy worlds.
- sheepscreek 3y ago
  > It would be fun to work as a traveling 3D capture artist, going to novel locations and scanning assets to use in fantasy worlds. When I read this, I immediately thought of Nikon’s Natural Intelligence campaign[1]. This comment was on point with their thinking. Akin to how being an “influencer” is now a legitimate profession. I can’t wait to see how people get incredibly creative to make a living using new AI tech. 1. Nikon Fights Back Against AI - https://petapixel.com/2023/06/15/nikon-fights-back-against-ai-images-touts-natural-intelligence/ https://petapixel.com/2023/06/15/nikon-fights-back-against-a...
- MikePlacid 3y ago
  Or rather - scanning assets in phantasy worlds and using the results for bot development. A bot, looking at the screen and using a Bluetooth keyboard for commands would be much less detectable.
  - serf 3y ago
    > A bot, looking at the screen and using a Bluetooth keyboard for commands would be much less detectable. this has been one of the preferred ways to go about cheating for some time; sometimes even with things like video-splitters way-back-when. >Or rather - scanning assets in phantasy worlds and using the results for bot development yeah but the thing with games is that the data is there, and often times trivial to deobfuscate -- and when you DO get to it, it's perfect 1:1 rather than trained AI best guesses; it's not some natural reality where a best-guest approximation is the best you can hope for. The bar for bot development will surely drop further and further, but until efforts towards keeping The Data away from Mean Person are strengthened then the easiest way forward is through data mining.
- i8comments 3y ago
  Run this through Google Maps Street View.
ur-whale 3y ago
Looks like ML is getting better every day at solving the generalized inverse problem.
cubefox 3y ago
I assume it could be modified to retain the surface colors.

84 points by bx376 3y ago

i wonder if this produces better results than Reality Capture?

So it's like photogrammetry, but you can do it with a phone video and it's significantly higher quality than previous implementations before cleanup. I wasn't able to find the compute requirements on the page, hopefully not too onerous. Something like this could be significantly beneficial for 3D asset production for games if there's less cleaning up of the photogrammetry models by hand, and for films it might actually be even better if they just remove the cleanup process altogether for small CGI detail in backgrounds where this sort of thing wouldn't be distinguishable from a photoreal model.

pdc56 3y ago
I found compute requirements in the github issue tracker: https://github.com/NVlabs/neuralangelo/issues/4 https://github.com/NVlabs/neuralangelo/issues/4 Not good, kinda scary actually
- danielEM 3y ago
  But these are requirements to train ANN and I guess the question was about requirements to process video stream.
  - lucubratory 3y ago
    Yes, only interested in the inference cost.
- junon 3y ago
  Training and usage are two different beasts. Training is always going to be expensive, but most users aren't going to do training themselves.

q_andrew 3y ago

Pretty amazing results. It would be fun to work as a traveling 3D capture artist, going to novel locations and scanning assets to use in fantasy worlds.

sheepscreek 3y ago
> It would be fun to work as a traveling 3D capture artist, going to novel locations and scanning assets to use in fantasy worlds. When I read this, I immediately thought of Nikon’s Natural Intelligence campaign[1]. This comment was on point with their thinking. Akin to how being an “influencer” is now a legitimate profession. I can’t wait to see how people get incredibly creative to make a living using new AI tech. 1. Nikon Fights Back Against AI - https://petapixel.com/2023/06/15/nikon-fights-back-against-ai-images-touts-natural-intelligence/ https://petapixel.com/2023/06/15/nikon-fights-back-against-a...
MikePlacid 3y ago
Or rather - scanning assets in phantasy worlds and using the results for bot development. A bot, looking at the screen and using a Bluetooth keyboard for commands would be much less detectable.
- serf 3y ago
  > A bot, looking at the screen and using a Bluetooth keyboard for commands would be much less detectable. this has been one of the preferred ways to go about cheating for some time; sometimes even with things like video-splitters way-back-when. >Or rather - scanning assets in phantasy worlds and using the results for bot development yeah but the thing with games is that the data is there, and often times trivial to deobfuscate -- and when you DO get to it, it's perfect 1:1 rather than trained AI best guesses; it's not some natural reality where a best-guest approximation is the best you can hope for. The bar for bot development will surely drop further and further, but until efforts towards keeping The Data away from Mean Person are strengthened then the easiest way forward is through data mining.
i8comments 3y ago
Run this through Google Maps Street View.

ur-whale 3y ago

Looks like ML is getting better every day at solving the generalized inverse problem.

cubefox 3y ago

I assume it could be modified to retain the surface colors.