3 ms·
From what I can tell, the whole reason the road "looks like a video game" is that it's using a source texture and then stretching it along the road's geometry,
by derefr 6y ago
From what I can tell, the whole reason the road "looks like a video game" is that it's using a source texture and then stretching it along the road's geometry, "diluting" the level of detail as seen from the camera viewpoint.
The image enhancement here is happening "in camera", like a fragment shader; but it's using a lot more context than the sort of local/cellular kernels that fragment shaders usually are.
Look at e.g. the gutters on the roads, where there are leaves/dirt — they get filled in with tons of extra perceptual texture (texture that exists in the projection plane, rather than texture that exists "on" the road), bringing them up to the same LoD as the rest of the road.
If that texture was "pushed back" to the origin texture, said texture would have to exist at some sort of ridiculous resolution that modern graphics cards wouldn't have a hope of rendering.
- gmueckl 6y agoThe part about ridiculous resolutions for texture detail like leaves isn't necessarily true. There are very old techniques for scattering small decals densely across surfaces. The basic trick is to use coarse textures that encode the positioning of decals within each texel footprint.
- derefr 6y agoSure, but no matter what the geometry/bump maps/etc. are on the road, they're not going to render out as anything but an aliased, badly-lit mess unless you're heavily supersampling them (and then doing ray-tracing to the supersampled image.) We're talking about "objects" where the crucial detail-features of their meshes for getting correct color math out (e.g. the normal+reflectance maps) would be, on average, scaled to be smaller than a pixel from the viewing distance+angle, with potentially many objects contributing to the final color of each pixel. So, rather than rendering the road at 8x-16x and then downsampling it, in order to get a pile of leaves to appear to have ~1.5x LoD at the projection plane, you can just achieve that 1.5x LoD pixel by pixel at the projection plane. Or, to put that another way: when a digital artist is painting a landscape, they don't have to zoom in by 8x, paint every individual blade of grass, and then zoom out. They can just use an artistic technique at 1x zoom that approximates the texture that would have been created by downsampling "photographic" detail painted in at 8x to your screen resolution. Here, the ML model is playing the role of the digital artist, "painting" on the projection plane.
- enriquto 6y agoThe problem is not texture antialiasing (which is essentially solved). The problem is shape antialiasing, whether many distant small shapes are so close together that become a sort of translucent buzz, which is very computationally heavy to compute.
- gmueckl 6y agoREYES renderers had a pretty nifty solution for shape antialiasing with their slicing and dicing of geometry into sub-pixel sized primitives. That was in the 80s, just for reference. Unfortunately, this method has a pretty complex implementation and just doesn't play all that well with hardware rasterization or path tracing and has since fallen out of favor.
- Gauge_Irrahphe 6y agoI don't think so. I think it's because the texture is already photo-like, and then there is a normal map applied on top of that. It basically creates two layers of shading that don't fit together. Few games manage to avoid that problem.
- derefr 6y agoInteresting hypothesis. I think this would predict that you’d get more realistic results if you were using “bare, unlit” textures (e.g. texture images + normal maps created by using 3D scanning or even computational tomography, as you would if you wanted to 3D-print a copy of the object) and then applying regular normal maps to them. Anyone have screenshots of 3D game objects that were explicitly mentioned as being authored this way?