6 ms·

Normal 1: what normal maps are and how they work

134 points by atomlib 7y ago

klodolph 7y ago
It would be nice to see object space / tangent space normal maps make an appearance. The article only talks about tangent space normal maps, which are actually more complicated than object space normal maps.
- bluescrn 7y ago
  It’s extremely rare to see anything other than tangent-space normal maps used these days, at least in games - as they allow the same map to be used on surfaces of any orientation, or on animated meshes
  - klodolph 7y ago
    It’s not that rare. The surfaces can have any orientation, they just can’t be deformed easily. But the idea here is that it’s easier to understand. If you show an object space normal map, and then show the rendered model, you can say, “aha, it’s really obvious how that works”. And then you can introduce the tangent space normal map, and describe how you use transformation matrices to convert tangent space to object space. Showing the object-space normal map gives you insight into how tangent space normal maps work.
robbrown451 7y ago
This is interesting, but one thing throughout is that he keeps talking about how normals control how the light "bounces" off the surface, for instance: "Normals are vectors that we use to define how light bounces from a surface." This makes sense for such things as specular highlights, but most of what we are seeing is simply shading. That is, the brightness of the surface is simply a matter of how much light is hitting the surface, due to the angle of the surface to rays of light from the light source. I would word it "how a surface is illuminated" rather than "how light bounces from a surface."
- magicalhippo 7y ago
  Most of what we're seeing is a non-trivial function of the light entering a local interaction point, and the light leaving it and hitting the camera or our eyes[1]. Since the entry and/or exit angle matters, perturbing the geometrically flat normals (from triangles) makes the surface appear to have curvature. This is important even for purely diffuse reflections. So I disagree with your wording. [1]: In some cases, such as skin, it's also important that the entry point and the exit point typically does not coincide. However for most materials one can usually get away with ignoring this aspect.
- fenwick67 7y ago
  Normal maps influence diffuse lighting as well. All lighting is just light bouncing off of things, I'm not sure why you're quibbling over this.
  - Jasper_ 7y ago
    They're talking about the incoming irradiance vs. the surface reflectance. Both are related to the surface normal, but the former has to do with surface visibility, rather than reflectance. You'll notice that BDRFs don't include the Lambert cos term, since they only describe how they deal with incoming irradiance. Put simply, a surface can't bounce any light if it doesn't receive any light to begin with. It's a weird, nitpicky statement, but hey, that's what this site is about, right? > All lighting is just light bouncing off of things In real-time graphics, bounce lighting is pretty rare. Typically you cache incoming irradiance into some form offline to emulate bounce lighting. Most of our scenes are lit with analytic lighting. A key directional e.g. sunlight for most light, and then punctual sources like point and spotlights for additional lighting tweaks.
  - robbrown451 7y ago
    I agree that normal maps influence diffuse lighting. Not sure I agree that lighting is always "light bouncing off things", since with a diffuse surface, it scatters it in all directions approximately equally. If you consider that a bounce, ok, but it isn't the same as "mirror type" reflection where the bounce angle works similarly to a ball bouncing off a surface (angle of incidence equals angle of reflection). So, I don't agree that the normal's effect on diffuse lighting is on the way light bounces off things, instead it is the quantity of light that hits the surface. I just think he could change the wording (as I suggested) to make it more accurate, so it betters people's understanding of the more general subject. It would be a simple change that would be easy to make, so it's intended as constructive criticism rather than just quibbling.
airstrike 7y ago
artstation.com is always so good. its community is as valuable as HN's except it's got digital artists rather than coders
bane 7y ago
This is a pretty old technique in the demoscene ('called bump mapping there'). There's probably a few hundred demos from the late 90s that use this technique on everything from Amigas to DOS systems of the time.
- jedimastert 7y ago
  Bump maps are subtly different from normal maps. Bump maps are a grayscale map that holds height info (tangential to the face normal). Slightly smaller file, but holds slightly less information.
- Jasper_ 7y ago
  Normal mapping has a long history. Jim Blinn did the original work in 1978, with "Simulation of Wrinkled Surfaces" [0] which precomputes the normal pertubation into a height texture ("bumpmap") and does some sleight of hand arithmetic to integrate it back, and there was a ton of noise about putting it in hardware around 1998 or so. The modern normal map was introduced by Kilgard as a rotation on top of the unperturbed surface normal [1] in surface tangent space, allowing it to be encoded with three channels, and compressed to two. [0] https://www.microsoft.com/en-us/research/wp-content/uploads/1978/01/p286-blinn.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/... [1] http://developer.download.nvidia.com/assets/gamedev/docs/GDC2K_gpu_bump.pdf http://developer.download.nvidia.com/assets/gamedev/docs/GDC...
  - airstrike 7y ago
    How the hell people like you have links to things like [0] will never cease to amaze me. This is why HN is irreplaceable imho. This place is too damn interesting Thank you for sharing!
    - chadcmulligan 7y ago
      If you want a nice trip through the history have a look at graphics gems - all online now http://www.realtimerendering.com/resources/GraphicsGems/ http://www.realtimerendering.com/resources/GraphicsGems/
jedimastert 7y ago
Subtle (or I guess just more technical) distinction to make: the red, green, and blue components of a particular pixel in a normal maps are the "x", "y", and "z" components of the normal vector, where "z" is defined as the tangent to the plane the image is taken from, and x and y are relative to the image. It's basically the same thing, but might help if you're more familiar with vector/matrix math
ehnto 7y ago
I'm an "enterprisey" software developer and have tried a few times to make a game by myself. When it comes to putting software and math to work, I've never been more impressed than with graphics rendering pipelines. It's a realm of software I'm not sure I will ever dive into, just marvel from afar. I "get" a lot of the techniques on a conceptual level, but I'm not sure I'll ever grok the code that pulls it off without spending much more time than I'm willing to commit to it.
- dahart 7y ago
  FWIW, it doesn’t take that much time if you start with a good setup. Writing your own game engine in order to write normal maps will take a loooong time, but you could whip up some normal maps on (for example) ShaderToy in an afternoon, easy, and there are tons of great examples there to reference and learn from.
- jayd16 7y ago
  It's actually not so hard to groc once you emerce yourself in it. The hard part is you don't really experience these same patterns in enterprise work. It'll click eventually. The main thing for me was realizing the goals of the pipeline. You need to process each pixel on its own in parallel to get the speed you need. All these texture techniques are in service of prebaking data down such that it can be effeciently prefetched before the shader needs it. At least for me, understanding the why helped me understand the what.
deleted 7y ago
[deleted]
crazygringo 7y ago
Very interesting and now I understand the concept of how bump mapping works better. But there seems to be an error: the first example that shows it applying to a sphere: https://cdnb.artstation.com/p/media_assets/images/images/000/444/501/medium/Comparacion_esferas.jpg?1560566676 https://cdnb.artstation.com/p/media_assets/images/images/000... The hard-transition version on the left shows the outline of the sphere with clear line segments, but the smooth-transition one on the right shows a perfectly circular outline. (The clearest comparison can be seen in the darkest section, the bottom-left edge.) So smoothing groups are clearly doing something in addition to changing how light reflects and this is not the full story. Does anyone know how the perfectly smooth, non-polygonal outline (silhouette) is achieved? This post is clearly not providing the full story.
- magicalhippo 7y ago
  These are not the same objects, or he turned on some tessellation as well. Normal mapping by itself does not help with the silhouette.

134 points by atomlib 7y ago

It would be nice to see object space / tangent space normal maps make an appearance. The article only talks about tangent space normal maps, which are actually more complicated than object space normal maps.

bluescrn 7y ago
It’s extremely rare to see anything other than tangent-space normal maps used these days, at least in games - as they allow the same map to be used on surfaces of any orientation, or on animated meshes
- klodolph 7y ago
  It’s not that rare. The surfaces can have any orientation, they just can’t be deformed easily. But the idea here is that it’s easier to understand. If you show an object space normal map, and then show the rendered model, you can say, “aha, it’s really obvious how that works”. And then you can introduce the tangent space normal map, and describe how you use transformation matrices to convert tangent space to object space. Showing the object-space normal map gives you insight into how tangent space normal maps work.

robbrown451 7y ago

This is interesting, but one thing throughout is that he keeps talking about how normals control how the light "bounces" off the surface, for instance: "Normals are vectors that we use to define how light bounces from a surface." This makes sense for such things as specular highlights, but most of what we are seeing is simply shading. That is, the brightness of the surface is simply a matter of how much light is hitting the surface, due to the angle of the surface to rays of light from the light source. I would word it "how a surface is illuminated" rather than "how light bounces from a surface."

magicalhippo 7y ago
Most of what we're seeing is a non-trivial function of the light entering a local interaction point, and the light leaving it and hitting the camera or our eyes[1]. Since the entry and/or exit angle matters, perturbing the geometrically flat normals (from triangles) makes the surface appear to have curvature. This is important even for purely diffuse reflections. So I disagree with your wording. [1]: In some cases, such as skin, it's also important that the entry point and the exit point typically does not coincide. However for most materials one can usually get away with ignoring this aspect.
fenwick67 7y ago
Normal maps influence diffuse lighting as well. All lighting is just light bouncing off of things, I'm not sure why you're quibbling over this.
- Jasper_ 7y ago
  They're talking about the incoming irradiance vs. the surface reflectance. Both are related to the surface normal, but the former has to do with surface visibility, rather than reflectance. You'll notice that BDRFs don't include the Lambert cos term, since they only describe how they deal with incoming irradiance. Put simply, a surface can't bounce any light if it doesn't receive any light to begin with. It's a weird, nitpicky statement, but hey, that's what this site is about, right? > All lighting is just light bouncing off of things In real-time graphics, bounce lighting is pretty rare. Typically you cache incoming irradiance into some form offline to emulate bounce lighting. Most of our scenes are lit with analytic lighting. A key directional e.g. sunlight for most light, and then punctual sources like point and spotlights for additional lighting tweaks.
- robbrown451 7y ago
  I agree that normal maps influence diffuse lighting. Not sure I agree that lighting is always "light bouncing off things", since with a diffuse surface, it scatters it in all directions approximately equally. If you consider that a bounce, ok, but it isn't the same as "mirror type" reflection where the bounce angle works similarly to a ball bouncing off a surface (angle of incidence equals angle of reflection). So, I don't agree that the normal's effect on diffuse lighting is on the way light bounces off things, instead it is the quantity of light that hits the surface. I just think he could change the wording (as I suggested) to make it more accurate, so it betters people's understanding of the more general subject. It would be a simple change that would be easy to make, so it's intended as constructive criticism rather than just quibbling.

airstrike 7y ago

artstation.com is always so good. its community is as valuable as HN's except it's got digital artists rather than coders

bane 7y ago

This is a pretty old technique in the demoscene ('called bump mapping there'). There's probably a few hundred demos from the late 90s that use this technique on everything from Amigas to DOS systems of the time.

jedimastert 7y ago
Bump maps are subtly different from normal maps. Bump maps are a grayscale map that holds height info (tangential to the face normal). Slightly smaller file, but holds slightly less information.
Jasper_ 7y ago
Normal mapping has a long history. Jim Blinn did the original work in 1978, with "Simulation of Wrinkled Surfaces" [0] which precomputes the normal pertubation into a height texture ("bumpmap") and does some sleight of hand arithmetic to integrate it back, and there was a ton of noise about putting it in hardware around 1998 or so. The modern normal map was introduced by Kilgard as a rotation on top of the unperturbed surface normal [1] in surface tangent space, allowing it to be encoded with three channels, and compressed to two. [0] https://www.microsoft.com/en-us/research/wp-content/uploads/1978/01/p286-blinn.pdf https://www.microsoft.com/en-us/research/wp-content/uploads/... [1] http://developer.download.nvidia.com/assets/gamedev/docs/GDC2K_gpu_bump.pdf http://developer.download.nvidia.com/assets/gamedev/docs/GDC...
- airstrike 7y ago
  How the hell people like you have links to things like [0] will never cease to amaze me. This is why HN is irreplaceable imho. This place is too damn interesting Thank you for sharing!
  - chadcmulligan 7y ago
    If you want a nice trip through the history have a look at graphics gems - all online now http://www.realtimerendering.com/resources/GraphicsGems/ http://www.realtimerendering.com/resources/GraphicsGems/

jedimastert 7y ago

Subtle (or I guess just more technical) distinction to make: the red, green, and blue components of a particular pixel in a normal maps are the "x", "y", and "z" components of the normal vector, where "z" is defined as the tangent to the plane the image is taken from, and x and y are relative to the image. It's basically the same thing, but might help if you're more familiar with vector/matrix math

ehnto 7y ago

I'm an "enterprisey" software developer and have tried a few times to make a game by myself. When it comes to putting software and math to work, I've never been more impressed than with graphics rendering pipelines. It's a realm of software I'm not sure I will ever dive into, just marvel from afar. I "get" a lot of the techniques on a conceptual level, but I'm not sure I'll ever grok the code that pulls it off without spending much more time than I'm willing to commit to it.

dahart 7y ago
FWIW, it doesn’t take that much time if you start with a good setup. Writing your own game engine in order to write normal maps will take a loooong time, but you could whip up some normal maps on (for example) ShaderToy in an afternoon, easy, and there are tons of great examples there to reference and learn from.
jayd16 7y ago
It's actually not so hard to groc once you emerce yourself in it. The hard part is you don't really experience these same patterns in enterprise work. It'll click eventually. The main thing for me was realizing the goals of the pipeline. You need to process each pixel on its own in parallel to get the speed you need. All these texture techniques are in service of prebaking data down such that it can be effeciently prefetched before the shader needs it. At least for me, understanding the why helped me understand the what.

deleted 7y ago

[deleted]

crazygringo 7y ago

Very interesting and now I understand the concept of how bump mapping works better. But there seems to be an error: the first example that shows it applying to a sphere: https://cdnb.artstation.com/p/media_assets/images/images/000/444/501/medium/Comparacion_esferas.jpg?1560566676 https://cdnb.artstation.com/p/media_assets/images/images/000... The hard-transition version on the left shows the outline of the sphere with clear line segments, but the smooth-transition one on the right shows a perfectly circular outline. (The clearest comparison can be seen in the darkest section, the bottom-left edge.) So smoothing groups are clearly doing something in addition to changing how light reflects and this is not the full story. Does anyone know how the perfectly smooth, non-polygonal outline (silhouette) is achieved? This post is clearly not providing the full story.

magicalhippo 7y ago
These are not the same objects, or he turned on some tessellation as well. Normal mapping by itself does not help with the silhouette.