3 ms·

Nvidia’s New AI: Instant Neural Graphics

54 points by ColinWright 5y ago

zamadatix 5y ago
If the video's presentation style isn't for you the original site it's talking about is probably more your cup of tea: https://nvlabs.github.io/instant-ngp/ https://nvlabs.github.io/instant-ngp/
pedalpete 5y ago
I was talking about generating 3D worlds from photographs just two years ago when working on my metaverse for outdoor sports app Ayvri.com - everyone said I was crazy. Amazing to see how far the state of the art has come.
jiggawatts 5y ago
This is a much bigger thing than people might realise. This is like a vertical cliff in in the graph of progress, a true phase-change marking a "before" and "after" in the timeline of the history of computer graphics. In 2012 I saw that scene in Prometheus where the scientists throw some drones up in the air. They zip away down the corridors of the alien temple, mapping it in 3D in real time. That scene resonated with me, because I realised that this wasn't some "distant future tech", but probably already doable. The hardware of the day was nearly capable of it, what we were missing was the algorithm. For the first time I feel in computer history, we've reached the point where the hardware has massively outstripped our mathematics. Our algorithms used to be decades or centuries ahead! George Boole died 100 years before widespread use of Boolean logic. Gauss died 200 years before computers started using his maths heavily. Etc.. Today we have teraflops available to us on our desktops, but the mathematics is yet to catch up. This paper is a huge "catching up". It's the first high-performance, general-purpose, robust, and incremental reverse renderer. That's a very generic thing with many applications, not just in computer graphics. Imagine what this technology will be able to do with just a few years of refinement. On a future RTX 4080, it may be possible to build 3D models in real time from a video feed! Super resolution extracted from multiple frames! Drones mapping 3D spaces as they fly through them! The "enhance!" capability from CSI might not be fantasy any more! We live in exciting times...
mchusma 5y ago
Very cool. I'm slightly confused by the term training though, isn't it more Inference than training? A model that in near realtime upscales images or creates a 3d reconstruction?

54 points by ColinWright 5y ago

If the video's presentation style isn't for you the original site it's talking about is probably more your cup of tea: https://nvlabs.github.io/instant-ngp/ https://nvlabs.github.io/instant-ngp/

pedalpete 5y ago

I was talking about generating 3D worlds from photographs just two years ago when working on my metaverse for outdoor sports app Ayvri.com - everyone said I was crazy. Amazing to see how far the state of the art has come.

jiggawatts 5y ago

This is a much bigger thing than people might realise. This is like a vertical cliff in in the graph of progress, a true phase-change marking a "before" and "after" in the timeline of the history of computer graphics. In 2012 I saw that scene in Prometheus where the scientists throw some drones up in the air. They zip away down the corridors of the alien temple, mapping it in 3D in real time. That scene resonated with me, because I realised that this wasn't some "distant future tech", but probably already doable. The hardware of the day was nearly capable of it, what we were missing was the algorithm. For the first time I feel in computer history, we've reached the point where the hardware has massively outstripped our mathematics. Our algorithms used to be decades or centuries ahead! George Boole died 100 years before widespread use of Boolean logic. Gauss died 200 years before computers started using his maths heavily. Etc.. Today we have teraflops available to us on our desktops, but the mathematics is yet to catch up. This paper is a huge "catching up". It's the first high-performance, general-purpose, robust, and incremental reverse renderer. That's a very generic thing with many applications, not just in computer graphics. Imagine what this technology will be able to do with just a few years of refinement. On a future RTX 4080, it may be possible to build 3D models in real time from a video feed! Super resolution extracted from multiple frames! Drones mapping 3D spaces as they fly through them! The "enhance!" capability from CSI might not be fantasy any more! We live in exciting times...

mchusma 5y ago

Very cool. I'm slightly confused by the term training though, isn't it more Inference than training? A model that in near realtime upscales images or creates a 3d reconstruction?