9 ms·
Real-Time Global Illumination by Precomputed Local Reconstruction
- londons_explore 9y agoThe approximate dynamic occluders is the most impressive part of the demo. Games etc. really need that stuff in real time.
- mikkom 9y agoit's important to note that this is for "mostly static scenes" so I'm not sure how suitable this is for games.
- exDM69 9y agoMost games are "mostly static scenes", where the level and lighting stays almost the same. Of course there is a varying amount of dynamic, moving objects but they don't necessarily need to have perfect indirect illumination (that would be hard to spot anyway, since the objects are on the move). The primary contributor is from the games industry and the results they show were generated in <4ms, so this is probably specifically targeted at games. There were even some "approximate dynamic occluders", which seem to work alright as long as the occluder isn't too near the camera.
- Chattered 9y agoYes, but are most games mostly static scenes because the tech is limited? Getting a decent looking scene which can be rendered quickly has historically relied on precomputed light maps, light volumes and space partitioning, which massively restrict the sort of game you can make if you want scenes to look nice. If more could be done in realtime, we would see games which were not mostly static scenes. I compare the situation to the 90s games. Machines did hardware scrolling and were optimised for blitting sprite sets. And so, lo and behold, most games are scrolling shooters and beat em ups. I always felt gpus have put similar constraints on our imaginations, hence most games are mostly static scenes.
- ndh2 9y agoGames need more. The occluders from the paper completely absorb light. A more realistic occluder would also reflect light.
- dualogy 9y ago> The occluders from the paper completely absorb light. A more realistic occluder would also reflect light. My understanding, "approximate occluders" aren't visible-geometry but stand-in replacements (very coarse to represent much finer-grained visible-geometry) during the GI pass(es). Probably even geometric primitives (sphere, box, torus etc pp) to, indeed, simply approximate occlusion. Physically-based reflections across all geometry already works brilliantly well for the last few years in real-time.
- sillysaurus3 9y agoIsn't it frustrating? All of this effort, and it looks nothing like real camcorder footage. That's not dismissive -- no one has ever made any program that outputs a string of images indistinguishable from a real camcorder. It's just that hard. I think whatever the next leap forward looks like, it will come from a nontraditional approach. Something strange, like powering your real-time lighting model by an actual camcorder -- set it up, point it at a real-world scene, then write a program that analyzes the way the light and color behaves in the camcorder's ground truth input. Then you'd somehow extrapolate that behavior across the rest of your scene. That last step sounds a lot like "Just add magic," but we have deep learning pipelines now. You could train it against your camcorder's input feed. Neural nets tend to work well when you have a reliable model, and we have the perfect one. So more precisely, you'd train your neural net against the camera's input video stream: at each generation, the program would try to paint your scene using whatever it thinks is the best guess for how the colors should look. Then you move your camcorder around, capturing how the colors actually looked, giving the pipeline enough data to correct itself. Rinse and repeat a few thousand times. The key to realism, and the central problem, is that colors affect colors around them. The way colors blend and fade across a wall has to be exactly right. There's no room for deviation from real life. Our visual systems have been tuned for a billion years to notice that. There are all kinds of issues with this idea: the real-world scene would need to be identical to the virtual scene, at least to start. The program would need to know the camera's orientation in order to figure out how to backproject the real-life illumination data onto the virtual scene. But at the end of it, you should wind up with a scene whose colors behave identically to real life. It seems like a promising approach because it gets rid of the whole idea of diffuse/ambient/specular maps, which don't correspond to reality anyway My favorite example: What does it mean to multiply a light's RGB color by a diffuse texture's RGB value? Nothing! It's a completely meaningless operation which happens to approximate reality quite well. There are huge advantages with that approach, like the flexibility of letting an artist create textures. But if the goal is precise, exact realism as defined by your camcorder, then we might be able to mimic nature directly. (Those dynamic occluders looked incredibly cool, by the way!)
- mixedbit 9y agoOne of the problems with your idea is that the camera position is one of the parameters of the rendering equation, so the same surface looks different from different points in 3D space. To fully capture how light interacts with the space, you can't make one photo, you would need to place the camera in infinite number of positions and store the results in infinite amount of memory.
- olegkikin 9y agoThat is incredible. Finally, realistic-looking GI in real time.
- sjtrny 9y agoYou know it's a good paper when you reach the end and ask "is that it?" and yet no one else has thought of it before. Side note: local-global approximation seems to do well in graphics and visual tasks. For example in the field of alpha matting the state of the art for a while was KNN Matting which sampled locally and globally. Most methods since then have taken a similar approach.
- bicubic 9y agoDoes this support dynamic moving lights? That's the 'wow' moment that's going to feel like a move to next generation graphics - when all/most lights are fully dynamic and contribute to GI.
- wlesieutre 9y agoHaven't read the paper yet, but it looks like yes. The caption at the top says "Real-time global illumination rendered using our method in 3.9ms with fully dynamic lights, cameras and diffuse surface materials." It's a couple of years old now, but Crassin's voxel cone tracing paper is a neat one: http://research.nvidia.com/publication/interactive-indirect-illumination-using-voxel-cone-tracing http://research.nvidia.com/publication/interactive-indirect-...
- m12k 9y agoAs far as I can tell this is mostly the same capability as the Enlighten middleware used by e.g. Unity and Frostbite (i.e. recent Battlefield titles) with some small improvements on top (e.g. the opaque occluder approximation at the end). A demo for reference: https://www.youtube.com/watch?v=ARorKHRTI80 https://www.youtube.com/watch?v=ARorKHRTI80 Is there some improvement in the technique beyond that - e.g. some clever speedup across the board?
- Agentlien 9y agoI feel I should point out that Frostbite is used by a lot more than battlefield. Including Anthem, Dragon Age Inquisition, the latest Fifa, Mirror's Edge Catalyst and the Need for Speed games by Ghost Games (which is where I work).
- mstade 9y agoAnd don't forget about Star Wars: Battlefront and the upcoming Battlefront 2. Ridiculously good looking games.
- qznc 9y agoUnity seems to have "shines through wall" errors? https://imgur.com/a/0dSYA https://imgur.com/a/0dSYA
- sorenjan 9y agoThe article mentions that they used a Nvidia Titan X Pascal, and that it took < 5 ms to compute. 5 ms is still a large part of the time budget for each frame, and most users have a slower GPU than that.
- sclangdon 9y agoI remember buying a new GPU everytime a new edition of Quake came out. Those were the days...
- virgil_disgr4ce 9y agoHas anything changed, other than the particular game?
- klodolph 9y agoYes. Quake 2 was December 1997, Quake 3 was December 1999. In that span, the game went from "use a GPU and you'll get colored lights and a higher resolution" to "use a GPU or else you don't get to play the game at all". GPU power seems to be increasing as fast as it always was, but the things we can do with that extra power are less impressive. So if you're playing games, it's not so big a deal any more if your graphics card is five years old.
- seabee 9y agoUp to a point; obsolescence happens by games requiring graphics APIs that didn't exist 5 years ago.
- klodolph 9y agoThere aren't many games that fall into that category. For example, look at Direct3D 12. You can count on your fingers the number of games that require D3D 12, and it's supported by graphics cards made in 2011 (although not all of them). Just look at the new API features we've been getting, they aren't really the kind of things you'd rewrite your engine for, with the exception of Vulkan / D3D 12 / Metal / etc. But Vulkan / D3D 12 are really just API changes that don't reflect underlying hardware changes. On top of that, a large percentage of games are made on top of engine middleware with good support for mediocre hardware (like Unreal and Unity, both of which run on phones if you want).
- kevindqc 9y agoAnyone have links to papers describing how GI works in Unreal and Unity?
- djmips 9y agoEnlighten is used in UE4 and Unity as I understand it. You can read about enlighten in Frostbite circa 2010 here: http://advances.realtimerendering.com/s2010/Martin-Einarsson-RadiosityArchitecture(SIGGRAPH%202010%20Advanced%20RealTime%20Rendering%20Course).pdf http://advances.realtimerendering.com/s2010/Martin-Einarsson...
- yuhe00 9y agoUE4 doesn't use Enlighten out of the box. It uses its own system called Lightmass, which generates very nice static lightmaps, but is not that great for dynamic GI (it has some basic support through what's called "indirect lighting caches" https://docs.unrealengine.com/latest/INT/Engine/Rendering/LightingAndShadows/IndirectLightingCache/ https://docs.unrealengine.com/latest/INT/Engine/Rendering/Li...). However, you can get Enlighten as a licensed plugin.
- rayuela 9y agoLink to the paper (pdf): http://research.nvidia.com/sites/default/files/pubs/2017-02_Real-Time-Global-Illumination/light-field-probes-final.pdf http://research.nvidia.com/sites/default/files/pubs/2017-02_...
- tobyhinloopen 9y agoIt took me a minute to appreciate it, didn't know what to look for. But wow!
- virgil_disgr4ce 9y agoWhat should I look for?
- theandrewbailey 9y agoLight bouncing around, lighting up things that aren't in direct light.
- Geee 9y agoThis kind of lighting (global illumination) is typically "baked in", i.e. lights don't move around.
- agorg_louk 9y agoDoes anybody know where the Gallery scene is from?