3 ms·

For someone who has done zero 3D graphics in a couple of decades, why do displacement maps exist? At first blush they don't appear to be more computationally e

For someone who has done zero 3D graphics in a couple of decades, why do displacement maps exist? At first blush they don't appear to be more computationally efficient than more complex geometry.

gcr 2y ago
The reason that comes to my mind is to save memory to transfer from the CPU to the GPU. If you do your tesselation inside the vertex shader, then you can send a low-poly mesh to the graphics card, which saves a lot on vertex buffers (e.g. uv coordinates and other per-vertex attributes). The vertex shader still emits the same number of vertices to the rest of the pipeline, but inter-stage bandwidth is more plentiful than CPU-GPU bandwidth so I can see that coming out ahead. I’m not an expert though. Perhaps someone with a better understanding can clear this up, I’m curious too…
MindSpunk 2y ago
It's less about being "less work" and more about what GPUs are and are not good at. Displacement maps can closely approximate the geometric detail of having 1 triangle per pixel. All the work for displacement maps happens on a per-pixel basis inside a fragment shader (in simple terms, a little program that runs for each pixel of a triangle). You can wrap this displacement map over a single, large triangle and get the visual appearance of a much denser mesh. The alternative approach of subdividing the mesh is orders of magnitude less efficient because GPUs are _very_ bad at drawing tiny polygons. It's just how GPUs and the graphics pipelines are implemented. A 'tiny polygon' is determined from how many pixels it covers, as you start dropping below a couple dozen pixels per triangle you start hitting nasty performance cliffs inside the GPU because of the specific ways triangles are drawn. Displacement maps works around this problem because you're logically only drawing single big polygons but doing the work in a shader where the GPU is much more efficient.
- Etherlord87 2y ago
  > Displacement maps can closely approximate the geometric detail of having 1 triangle per pixel. All the work for displacement maps happens on a per-pixel basis inside a fragment shader (in simple terms, a little program that runs for each pixel of a triangle). You can wrap this displacement map over a single, large triangle and get the visual appearance of a much denser mesh. I think you're wrong here... Can you provide an example of an engine where this is actually the case? To my knowledge (and I checked on Wikipedia [1] to make sure), a displacement map simply displaces VERTICES along their normals (so white color moves a maximum distance along normal, and black either doesn't move at all or moves maximum distance along the inverted normal - inwards). This means you need to heavily subdivide your geometry. Or you can remesh your geometry, with the simplest remeshing algorithms being just voxelizers - and that's what we see the OP doing - except in the place where all the 3D graphics complain on the limitation of voxelization, he hypes it as this new retro look he invented. What seems to confirm my interpretation of all of this is how the OP describes this being done on the CPU, and - while he tries to downplay it - using rather decent hardware - I mean, Steam Deck is not exactly ancient hardware, and for an extremely simple scene with nothing else going on, he's happy being above 60 FPS on 800p resolution! [1] https://en.wikipedia.org/wiki/Displacement_mapping https://en.wikipedia.org/wiki/Displacement_mapping
  - avallach 2y ago
    The magic lies in tessellation. Tessellation is an efficient GPU process of heavily subdividing your mesh, so that displacement maps can add visible geometric details afterwards. And because it's dynamic you can selectively apply it only to the meshes that are close to the camera. These are reasons why it's better than subdividing the mesh at preprocessing stage and "baking in " the displacement into vertex positions.
    - Etherlord87 2y ago
      Good point! It's not just the LOD, it may be also the fact that the parallelization makes GPUs a better fit for subdivision than CPUs, and surely there's a matter of connection bandwith to the GPUs: the vertex coordinates, as well as lots of other vertex-specific data, needs only to be sent for the base vertices, the new vertices get their new values from interpolating old values, and the textures like displacement map control the difference from interpolated value to desired value. Of course a texture would have been just some weird compromise of resolution of the sent data, except you don't have to provide a texture for every attribute, and more importantly, such a texture might be static (e.g. if encoded in normal space it may work throughout an animation of an object).
pixelesque 2y ago
It allows you to defer subdivision of the mesh (maybe down to micropoly level for displacement) until the final render stage, rather than baking in the subdivision earlier and having to deal with dense geometry for things like animation where it's better to have lighter-weight meshes.
modeless 2y ago
I think displacement maps are not that frequently used in game graphics these days. The typical thing is regular geometry with normal maps. However, triangle soup is very difficult to modify in any way. The cool thing about displacement maps is that they can downsampled trivially, so if you're doing runtime tesselation you can get smooth LOD scaling (at least down to your base mesh) which is nice. Less usefully, they can also be tiled, blended, interpolated, scrolled, animated, etc. On the other hand, UE5's Nanite achieves the same LOD scaling but far better, without the limitation of not being able to scale down past the base mesh and with a built-in workaround for the issues GPUs have with small triangles. It's possible that people might use displacement maps in the authoring process, which can then be baked into static meshes for Nanite. Then it would just be a convenient-ish way for artists to author and distribute complex materials.

For someone who has done zero 3D graphics in a couple of decades, why do displacement maps exist? At first blush they don't appear to be more computationally efficient than more complex geometry.

gcr 2y ago

The reason that comes to my mind is to save memory to transfer from the CPU to the GPU. If you do your tesselation inside the vertex shader, then you can send a low-poly mesh to the graphics card, which saves a lot on vertex buffers (e.g. uv coordinates and other per-vertex attributes). The vertex shader still emits the same number of vertices to the rest of the pipeline, but inter-stage bandwidth is more plentiful than CPU-GPU bandwidth so I can see that coming out ahead. I’m not an expert though. Perhaps someone with a better understanding can clear this up, I’m curious too…

MindSpunk 2y ago

It's less about being "less work" and more about what GPUs are and are not good at. Displacement maps can closely approximate the geometric detail of having 1 triangle per pixel. All the work for displacement maps happens on a per-pixel basis inside a fragment shader (in simple terms, a little program that runs for each pixel of a triangle). You can wrap this displacement map over a single, large triangle and get the visual appearance of a much denser mesh. The alternative approach of subdividing the mesh is orders of magnitude less efficient because GPUs are _very_ bad at drawing tiny polygons. It's just how GPUs and the graphics pipelines are implemented. A 'tiny polygon' is determined from how many pixels it covers, as you start dropping below a couple dozen pixels per triangle you start hitting nasty performance cliffs inside the GPU because of the specific ways triangles are drawn. Displacement maps works around this problem because you're logically only drawing single big polygons but doing the work in a shader where the GPU is much more efficient.

Etherlord87 2y ago
> Displacement maps can closely approximate the geometric detail of having 1 triangle per pixel. All the work for displacement maps happens on a per-pixel basis inside a fragment shader (in simple terms, a little program that runs for each pixel of a triangle). You can wrap this displacement map over a single, large triangle and get the visual appearance of a much denser mesh. I think you're wrong here... Can you provide an example of an engine where this is actually the case? To my knowledge (and I checked on Wikipedia [1] to make sure), a displacement map simply displaces VERTICES along their normals (so white color moves a maximum distance along normal, and black either doesn't move at all or moves maximum distance along the inverted normal - inwards). This means you need to heavily subdivide your geometry. Or you can remesh your geometry, with the simplest remeshing algorithms being just voxelizers - and that's what we see the OP doing - except in the place where all the 3D graphics complain on the limitation of voxelization, he hypes it as this new retro look he invented. What seems to confirm my interpretation of all of this is how the OP describes this being done on the CPU, and - while he tries to downplay it - using rather decent hardware - I mean, Steam Deck is not exactly ancient hardware, and for an extremely simple scene with nothing else going on, he's happy being above 60 FPS on 800p resolution! [1] https://en.wikipedia.org/wiki/Displacement_mapping https://en.wikipedia.org/wiki/Displacement_mapping
- avallach 2y ago
  The magic lies in tessellation. Tessellation is an efficient GPU process of heavily subdividing your mesh, so that displacement maps can add visible geometric details afterwards. And because it's dynamic you can selectively apply it only to the meshes that are close to the camera. These are reasons why it's better than subdividing the mesh at preprocessing stage and "baking in " the displacement into vertex positions.
  - Etherlord87 2y ago
    Good point! It's not just the LOD, it may be also the fact that the parallelization makes GPUs a better fit for subdivision than CPUs, and surely there's a matter of connection bandwith to the GPUs: the vertex coordinates, as well as lots of other vertex-specific data, needs only to be sent for the base vertices, the new vertices get their new values from interpolating old values, and the textures like displacement map control the difference from interpolated value to desired value. Of course a texture would have been just some weird compromise of resolution of the sent data, except you don't have to provide a texture for every attribute, and more importantly, such a texture might be static (e.g. if encoded in normal space it may work throughout an animation of an object).

pixelesque 2y ago

It allows you to defer subdivision of the mesh (maybe down to micropoly level for displacement) until the final render stage, rather than baking in the subdivision earlier and having to deal with dense geometry for things like animation where it's better to have lighter-weight meshes.

modeless 2y ago

I think displacement maps are not that frequently used in game graphics these days. The typical thing is regular geometry with normal maps. However, triangle soup is very difficult to modify in any way. The cool thing about displacement maps is that they can downsampled trivially, so if you're doing runtime tesselation you can get smooth LOD scaling (at least down to your base mesh) which is nice. Less usefully, they can also be tiled, blended, interpolated, scrolled, animated, etc. On the other hand, UE5's Nanite achieves the same LOD scaling but far better, without the limitation of not being able to scale down past the base mesh and with a built-in workaround for the issues GPUs have with small triangles. It's possible that people might use displacement maps in the authoring process, which can then be baked into static meshes for Nanite. Then it would just be a convenient-ish way for artists to author and distribute complex materials.