5 ms·
Nvidia's Crazy New Neural Engine Is Redefining Realism in Graphics
- reaperman 4y agoThis is a weird third party AI generated speech video. Does anyone have a link to the official NVidia video where they are getting all these clips from?
- christoph 4y agoI believe the original video is this one: https://research.nvidia.com/labs/rtr/neural_appearance_models/assets/nvidia_neural_materials_video-2023-05.mp4 https://research.nvidia.com/labs/rtr/neural_appearance_model...
- cs702 4y agoHoly mackarel! Those demos are impressive. Sooner than you expect, games and virtual reality are going to look as good as movies -- scratch that, they are going to look better than movies. We live in interesting times.
- gumballindie 4y agoMobile VR won't work as nicely tho, it will be tether VR that stands a chance.
- smoldesu 4y agoI don't see why they both can't exist. Mobile VR already works great, and the power of mobile chipsets is only increasing. Tethered VR will only dominate for as long as mobile GPUs struggle to output stereo 2048x2048~ish video, which... isn't going to be for long. Combined with SOTA upscaling, we may already be there by some standards. I say all this as someone with a Quest who tethers to a PC for Beat Saber and Half Life Alyx. Tethered experiences rule - but untethered ones are really not that far off.
- gumballindie 4y ago> but untethered ones are really not that far off. I am not sure how you measure this but you can't run proper games on mobile vr. Mobile is limited, you can't fit in a GPU the same size of a desktop GPU. John Carmack has an interesting talk about what happens when mobile chips get too small and crowded. We simply can't battle physics. It would be awesome if we had the same experience tho.
- smoldesu 4y agoObviously the two will never coincide. Mobile GPUs will eventually reach a "good enough" stage though, and arguably we're already there. The quality of Quest-native games like Beat Saber is almost identical to the version on PC. Older games like Resident Evil 4 play just like normal. Visually it can be 'meh', but the tech is there and the option to stream from a more powerful desktop still exists. It uses comparatively weak chipsets to deliver pretty-damn-good visuals at a price point lower than most consoles. I would argue that your thesis of "you can't run proper games on mobile vr" is wrong. Today, you can go play DOOM 3 or Half Life in VR, untethered, on a sub-$500 headset. That should startle everyone working on tethered systems.
- gumballindie 4y agoInteresting, I need to give it another go.
- tomxor 4y ago1 sample: I only play quest2 untethered, don't have a gaming pc and don't want to spend time on that right now. Games are more than just graphics, and a lot of fun can be had without huge GPUs. The quest works well as a portable VR console if you focus on fun games rather than pretty games.
- ekianjo 4y agoProblem is most people dont care about VR. The market is far from exploding and its not because of a lack of trying.
- 4y ago
- hanniabu 4y ago> they are going to look better than movies. Movies will be made with this stuff. No expensive actors, sets, or long editing times that come with traditional animations.
- kevingadd 4y agoYeah, Unreal Engine is already being used during shoots in stuff like the Volume to be able to modify the virtual "set" they're shooting on
- WithinReason 4y agoVery impressive results I would say. Link to paper: https://research.nvidia.com/labs/rtr/neural_appearance_models/ https://research.nvidia.com/labs/rtr/neural_appearance_model... Similar one, about texture compression: https://research.nvidia.com/publication/2023-08_random-access-neural-compression-material-textures https://research.nvidia.com/publication/2023-08_random-acces...
- ulnarkressty 4y ago> equal contribution, order determined by a rock-paper-scissors tournament. I hope to see more and more bizarre ways of picking the author order on these kind of papers.
- codetrotter 4y agoI want to see one where it says > equal contribution, three of the authors are the real authors, the other seven names were generated by an AI
- TOMDM 4y agoWhy?
- deleted 4y ago[deleted]
- codetrotter 4y agoJust for fun
- lucubratory 4y agoAlready been done and already been banned. Journals don't like it any more than any other publication.
- qrian 4y agoI've seen one determined by a harry potter quiz. > Order of authorship was determined by the top score each person received across all three levels of the official Pottermore “Back to Hogwarts Quiz.” For information about the exact scores (and houses), please contact the authors. https://compass.onlinelibrary.wiley.com/doi/abs/10.1111/spc3.12425 https://compass.onlinelibrary.wiley.com/doi/abs/10.1111/spc3...
- deleted 4y ago[deleted]
- HellDunkel 4y agoAs far as i understand the neural engine is trained on texture BRDFs which are too memory intensive for most rendering usecases (offline & realtime). The hierarchical model delivers sub-pixel accuracy outperforming mipmaps. So far so good but this is no replacement to tracing a ton of rays. Temporal stability and versatility is questionable. Will be interesting to see how this compares to Unreals Substrate.
- low_tech_love 4y agoCool, but I'm sure there must be some better link for this material. Quote from a YouTube comment: Now if we could only get text to speech from "OK" to "blows you away", instead of "your drunk uncle talking from down an empty well"
- boulos 4y agoSome of the results on https://rhasspy.github.io/piper-samples/ https://rhasspy.github.io/piper-samples/ are super impressive (via https://www.home-assistant.io/blog/2023/04/27/year-of-the-voice-chapter-2/ https://www.home-assistant.io/blog/2023/04/27/year-of-the-vo...).
- etiam 4y agoI'm going to have to defer to Louis CK on that one https://www.youtube.com/watch?v=PdFB7q89_3U https://www.youtube.com/watch?v=PdFB7q89_3U
- yCombLinks 4y agoIt's a funny bit, but this particular voice over is awful. Simply terrible, F- quality. There are 10 other links intthis thread that would have produced a better voice, or the publisher could have just read the material themselves.
- low_tech_love 4y agoIt’s hilarious, but in this case I think it doesn’t apply. There is next to zero content here that is actually contributed by the person who made the video, and the voice over is pretty horrible. But it did get to front page so maybe I’m just wrong!
- etiam 4y agoOkay. Then I think I see where you're coming from. From my perspective this was the mainly the first contact with the research presentation and the renderings, and I think those are undeniably very impressive to most people at this point in time. But viewing it as a "Youtube value-add", or whatever the jargon for that phenomenon might be, I guess it's close at hand to judge it as poor, lazy work that doesn't do the original material justice. I will also admit it is something of a funny juxtaposition to have the photorealistic, high-definition graphics along with voiceover that is very obviously artificial.
- kasperni 4y agoFrom the paper -------------- Our system is running on Direct3D 12 using hardware-accelerated ray tracing through DirectX Raytracing (DXR). All results are generated on an NVIDIA GeForce RTX 4090 GPU at resolution 1920 × 1080
- kman82 4y agoThe day vr looks like avatar is the day I buy a vr headset
- xwdv 4y agoThis is the climax of 3D graphics rendering!!
- joering2 4y agoIf you see progress of graphics design in the last 40 years, do you have any idea what would be considered a "progress of graphics design" in the year 2100 ? I mean seriously, it looks like in the next maybe 5-15 years, GPU will be able to render graphics that even another GPU wouldn't be able to distinguish whether its real or fake.
- imiric 4y agoGraphics aren't really the problem anymore. We've already reached photorealism, and iterative improvements to render it in real-time are developing quickly, while the hardware gets more powerful, efficient and cheaper. The bulk of the progress will be in human-computer interfaces. We're still mostly interacting with keyboards, mice and 2D touchscreens; voice recognition is still primitive; XR is still experimental and nobody wants to strap a heavy headset to their face, etc.
- clnq 4y agoFinally — a use for the Rockwell Retro Encabulator.
- eljost 4y agoThe audio also reminded me of the Fallout games.
- whyenot 4y agoI still don’t understand what the function is of the sinusoidal dingle arm.
- Jupe 4y agoI believe its purpose is to reduce sinusoidal repleneration, but usually limited to situations which need forescent skor motion. About one minute in I had a "Turbo Encabulator" feeling. So much so I'm not sure the original article is even real... I'm in no position to validate it without spending many hours scouring the content.
- andrewmcwatters 4y agoI think graphics technology has already "arrived" for the most part. There's tons of stuff that surpasses what Crytek did that blew away the industry 16 years ago, but a lot of that wasn't just technology, it's that they had world-class artists. If you go back to that game today, there's a lot you can nitpick, but I think the biggest problem that remains today is that it's still extremely labor intensive to make assets. At Planimeter, I worry more about asset creation turnaround time than anything else. I worry about it more than the tech we write, I worry about it more than fiction writing, or audio engineering. Just nothing else compares. It's the most expensive part of any game development pipeline, and yet industry-wide accessible photogrammetry is half-backed, and it also doesn't help hobbyists who do 2.5D or 2D work. So yeah, this neural engine stuff is superphenominal, for people who care about PBR workflows and photorealistic pipelines. This is obviously the most computationally complex work that can be addressed today, anyway, which I appreciate. But the people who can actually access and harness that tech is just so absolutely tiny. I guess I just care a lot about this one particular sector of the industry where you have these world-class bedroom professionals who become studio professionals and Unreal and Nvidia are probably the only orgs in the world who cater to them, but for some reason I have to put in a lot of effort to articulate why what we have today, despite being so much more powerful than what we've had 20 years ago is less accessible and less functional and less empowering than what we had then. I think it's primarily the labor factor in artwork, but also accessible engine tech today is actually worse than what we were using then, simply because no one really uses the id Tech family of engines anymore besides the most modern incarnations of them, and even to this day no class of engine compares to old versions of id Tech. Not even Unreal, by a long shot, despite having industry leading rendering capabilities. Everything else about that engine is half-baked or unusable.
- mmcconnell1618 4y agoI wonder how much AI models can help. If they are trained on enough real world content, they should be able to produce rough models based on descriptions: "Generate a tropical island about 5km in width by 10km length with an active volcano in the center" and then then the artists adjust/massage as needed.
- 4y ago
- est 4y agohttps://research.nvidia.com/labs/par/Perfusion/ https://research.nvidia.com/labs/par/Perfusion/ this gets less attraction, but I think it could be the next hit after LoRA.
- rasz 4y agoTLDR: "Neural material are compiled to real-time shader code." aka procedurally generated textures. Clever for all the mundane repeating pattern/uniform surfaces.
- Curel 4y agoHack By Rizky
- Laaas 4y agoLink to paper: https://research.nvidia.com/labs/rtr/neural_appearance_models/ https://research.nvidia.com/labs/rtr/neural_appearance_model...
- LarsDu88 4y agoThis tech can be used to rapidly generate the material needed for a very realistic paint job. Realistically, only the biggest triple A studio will be using this correctly and in the nice looking games that would benefit from this
- HellDunkel 4y agoI don‘t see that coming. Texture BRDF scanning is extremly difficult even more so if to be used as input to train the model.