Normal map

From polycount
Revision as of 11:02, 14 September 2010 by EricChadwick (Talk | contribs)

Jump to: navigation, search

Normal Map

What is a Normal Map?

A Normal Map is usually used to fake high-res geometry detail when it's mapped onto a low-res mesh. The pixels of the normal map each store a normal, a vector that describes the surface slope of the original high-res mesh at that point. The red, green, and blue channels of the normal map are used to control the direction of each pixel's normal.

When a normal map is applied to a low-poly mesh, the texture pixels control the direction each of the pixels on the low-poly mesh will be facing in 3D space, creating the illusion of more surface detail or better curvature. However, the silhouette of the model doesn't change.

File:NormalMap$Whatif normalmap mapped2.jpg File:NormalMap$Whatif normalmap low.jpg
A model with a normal map.<
>image by James Ku 		The low-resolution wireframe.<
>image by James Ku


<<Anchor(TangentSpaceVsObjectSpace)>>

Tangent-Space vs. Object-Space

Normal maps can be made in either of two basic flavors: tangent-space or object-space. Object-space is also called local-space or model-space, same thing. World-space is basically the same as object-space, except it requires the model to remain in its original orientation, neither rotating nor deforming, so it's almost never used.

Tangent-space normal map

Predominantly-blue colors. Object can rotate and deform. Good for deforming meshes, like characters, animals, flags, etc. <
> <
> File:NormalMap$green plus.png Maps can be reused easily, like on differently-shaped meshes. <
> File:NormalMap$green plus.png Maps can be tiled and mirrored easily, though some games might not support mirroring very well. <
> File:NormalMap$green plus.png Easier to overlay painted details. <
> File:NormalMap$green plus.png Easier to use image compression. <
> File:NormalMap$red x.png More difficult to avoid smoothing problems from the low-poly vertex normals (see Smoothing Groups and Hard Edges). <
> File:NormalMap$red x.png Slightly slower performance than an object-space map (but not by much).

Object-space normal map

Rainbow colors. Objects can rotate, but usually shouldn't be deformed, unless the shader has been modified to support deformation. <
> <
> File:NormalMap$green plus.png Easier to generate high-quality curvature because it completely ignores the crude smoothing of the low-poly vertex normals. <
> File:NormalMap$green plus.png Slightly better performance than a tangent-space map (but not by much). <
> File:NormalMap$red x.png Can't easily reuse maps, different mesh shapes require unique maps. <
> File:NormalMap$red x.png Difficult to tile properly, and mirroring requires specific shader support. <
> File:NormalMap$red x.png Harder to overlay painted details because the base colors vary across the surface of the mesh. Painted details must be converted into Object Space to be combined properly with the OS map. <
> File:NormalMap$red x.png They don't compress very well, since the blue channel can't be recreated in the shader like with tangent-space maps. Also the three color channels contain very different data which doesn't compress well, creating many artifacts. Using a half-resolution object-space map is one option.

<<Anchor(ConvertingBetweenSpaces)>>

Converting Between Spaces

Normal maps can be converted between tangent space and object space, in order to use them with different blending tools and shaders, which require one type or the other.

Diogo "fozi" Teixeira created a tool called NSpace that converts an object-space normal map into a tangent-space map, which then works seamlessly in the Max viewport. He converts the map by using the same tangent basis that 3ds Max uses for its hardware shader. To see the results, load the converted map via the Normal Bump map and enable "Show Hardware Map in Viewport". Osman "osman" Tsjardiwal created a GUI for NSpace, you can download it here, just put it in the same folder as the NSpace exe and run it. Diogo has further plans for the tool as well.

Joe "EarthQuake" Wilson said: "[8Monkey Labs has] a tool that lets you load up your reference mesh and object space map. Then load up your tangent normals, and adjust some sliders for things like tile and amount. We need to load up a mesh to know how to correctly orient the tangent normals or else things will come out upside down or reverse etc. It mostly works, but it tends to "bend" the resulting normals, so you gotta split the mesh up into some smoothing groups before you run it, and then I usually will just composite this "combo" texture over my orig map in Photoshop."

RGB Channels

Shaders can use different techniques to render tangent-space normal maps, but the normal map directions are usually consistent within a game. Usually the red channel of a tangent-space normal map stores the X axis (pointing the normals predominantly leftwards or rightwards), the green channel stores the Y axis (pointing the normals predominantly upwards or downwards), and the blue channel stores the Z axis (pointing the normals outwards away from the surface).

If you see lighting coming from the wrong angle when you're looking at your normal-mapped model, and the model is using a tangent-space normal map, the normal map shader might be expecting the red or green channel (or both) to point in the opposite direction. To fix this either change the shader, or simply invert the appropriate color channels in an image editor, so that the black pixels become white and the white pixels become black.

Some shaders expect the color channels to be swapped or re-arranged to work with a particular compression format. For example the DXT5_nm format usually expects the X axis to be in the alpha channel, the Y axis to be in the green channel, and the red and blue channels to be empty.

<<Anchor(TangentBasis)>>

Tangent Basis

Tangent-space normal maps use a special kind of vertex data called the tangent basis. This is similar to UV coordinates except it provides directionality across the surface, it forms a surface-relative coordinate system for the per-pixel normals stored in the normal map.

Light rays are in world space, but the normals stored in the normal map are in tangent space. When a normal-mapped model is being rendered, the light rays must be converted from world space into tangent space, using the tangent basis to get there. At that point the incoming light rays are compared against the directions of the normals in the normal map, and this determines how much each pixel of the mesh is going to be lit. Alternatively, instead of converting the light rays some shaders will convert the normals in the normal map from tangent space into world space. Then those world-space normals are compared against the light rays, and the model is lit appropriately. The method depends on who wrote the shader, but the end result is the same.

Unfortunately for artists, there are many different ways to calculate the tangent basis: 3ds Max, Maya, DirectX 9, NVMeshMender, Eric Lengyel, a custom solution, etc. This means a normal map baked in one application probably won't shade correctly in another. Artists must do some testing with different baking tools to find which works best with their output. When the renderer (or game engine) renders your game model, the shader must use the same tangent basis as the normal map baker, otherwise you'll get incorrect lighting, especially across the seams between UV shells.

The xNormal SDK supports custom tangent basis methods. When a programmer uses it to implement their renderer's own tangent basis, artists can then use Xnormal to bake normal maps that will match their renderer perfectly.

The UVs and the vertex normals on the low-res mesh directly influence the coloring of a tangent-space normal map when it is baked. Each tangent basis vertex is a combination of three things: the mesh vertex's normal (influenced by smoothing), the vertex's tangent (usually derived from the V texture coordinate), and the vertex's bitangent (derived in code, also called the binormal). These three vectors create an axis for each vertex, giving it a specific orientation in the tangent space. These axes are used to properly transform the incoming lighting from world space into tangent space, so your normal-mapped model will be lit correctly.

When a triangle's vertex normals are pointing straight out, and a pixel in the normal map is neutral blue (128,128,255) this means that pixel's normal will be pointing straight out from the surface of the low-poly mesh. When that pixel normal is tilted towards the left or the right in the tangent coordinate space, it will get either more or less red color, depending on whether the normal map is set to store the X axis as either a positive or a negative value. Same goes for when the normal is tilted up or down in tangent space, it will either get more or less green color. If the vertex normals aren't exactly perpendicular to the triangle, the normal map pixels will be tinted away from neutral blue as well. The vertex normals and the pixel normals in the normal map are combined together to create the final per-pixel surface normals.

Shaders are written to use a particular direction or "handedness" for the X and Y axes in a normal map. Most apps tend to prefer +X (red facing right) and +Y (green facing up), while others like 3ds Max prefer +X and -Y. This is why you often need to invert the green channel of a normal map to get it to render correctly in this or that app... the shader is expecting a particular handedness.

When you look at a tangent-space normal map for a character, you typically see different colors along the UV seams. This is because the UV shells are often oriented at different angles on the mesh, a necessary evil when translating the 3D mesh into 2D textures. The body might be mapped with a vertical shell, and the arm mapped with a horizontal one. This requires the normals in the normal map to be twisted for the different orientations of those UV shells. The UVs are twisted, so the normals must be twisted in order to compensate. The tangent basis helps reorient (twist) the lighting as it comes into the surface's local space, so the lighting will then look uniform across the normal mapped mesh.

When an artist tiles a tangent-space normal map across an arbitrary mesh, like a landscape, this tends to shade correctly because the mesh has a uniform direction in tangent space. If the mesh has discontinuous UV coordinates (UV seams), or the normal map has large directional gradients across it, the tangent space won't be uniform anymore so the surface will probably have shading seams.

Modeling the Low-Poly Mesh

The in-game mesh usually needs to be carefully optimized to create a good silhouette, define edge-loops for better deformation, and minimize extreme changes between the vertex normals for better shading (see Smoothing Groups & Hard Edges).

In order to create an optimized in-game mesh including a good silhouette and loops for deforming in animation, you can start with the 2nd subdivision level of your digital sculpt, or in some cases with the base mesh itself. Then you can just collapse edge loops or cut in new edges to add/remove detail as necessary. Or you can re-toplogize from scratch if that works better for you.

UV Coordinates

Normal map baking tools only capture normals within the 0-1 UV square, any UV bits outside this area are ignored.

Only one copy of the forward-facing UVs should remain in the 0-1 UV square at baking time. If the mesh uses overlapping UVs, this will likely cause artifacts to appear in the baked map, since the baker will try render each UV shell into the map. Before baking, it's best to move all the overlaps and mirrored bits outside the 0-1 sqaure.

If you move all the overlaps and mirrored bits exactly 1 UV unit (any whole number will do), then you can leave them there after the bake and they will still be mapped correctly. You can move them back if you want, it doesn't matter to most game engines. Be aware that ZBrush does use UV offsets to manage mesh visibility, however this usually doesn't matter because the ZBrush cage mesh is often a different mesh than the in-game mesh used for baking.

You should avoid changing the UVs after baking the normal map, because rotating or mirroring UVs after baking will cause the normal map not to match the tangent basis anymore, which will likely cause lighting problems.

In 3ds Max, W is a third texture coordinate. It's used for 3D procedural textures and for storing vertex color in UV channels (you need 3 axes for RGB, so UVW can store vertex color). Bake problems can be avoided by moving any overlapping UVs to -1 on the W axis, with the same results as moving them 1 unit on the U or V axes. The tool Render To Texture will always bake whatever UVs are the highest along the W axis. However using W can be messy... it's generally hidden unless you purposefully look for it (bad for team work), doesn't get preserved on export to other apps, and high W values can prevent selecting and/or welding UVs.

Mirroring

Normal maps can be mirrored across a model to create symmetrical details, and save UV space, which allows more detail in the normal map since the texture pixels are smaller on the model.

With object-space maps, mirroring requires specific shader support. For tangent-space maps, mirroring typically creates a shading seam, but this can be reduced or hidden altogether, depending on the method used.

Typical Mirroring Workflow

  1. Delete the mesh half that will be mirrored.
  2. Arrange the UVs for the remaining model, filling the UV square.
  3. Mirror the model to create a "whole" mesh, welding the mesh vertices along the seam.
  4. Move the mirrored UVs exactly 1 unit (or any whole number) out of the 0-1 UV square.
  5. Bake the normal map.

Sometimes an artist will decide to delete half of a symmetrical model before baking.

This is a mistake however because often the vertex normals along the hole will bend towards the hole a bit; there are no faces on the other side to average the normals with. This will create a strong lighting seam in the normal map.

It's typically best to use the complete mirrored model to bake the normal map, not just the unique half.

To prevent the mirrored UVs from causing overlaps or baking errors, move the mirrored UVs out of the 0-1 UV space, so only one copy of the non-mirrored UVs is left within the 0-1 square.

To avoid texel "leaks" between the UV shells, make sure there's enough Edge Padding around each shell, including along the edges of the normal map. None of the UV shells should be touching the edge of the 0-1 UV square, unless they're meant to tile with the other side of the map.

Center Mirroring

If the mirror seam runs along the surface of a continuous mesh, like down the center of a human face for example, then it will probably create a lighting seam.

In Epic Games' Unreal Engine 3 (UE3) their symmetrical models commonly use centered mirroring. Epic uses materials that mix a DetailMap with the normal maps; these seem to scatter the diffuse/specular lighting and help minimize the obviousness of the mirror seams. For their Light Mapped models they use a technique that can almost completely hide the mirror seam.

GOW2 normal map seams, UDK normal map seams

Offset Mirroring

Offset mirroring is a method where you move the mirror seam off to one side of the model, so the seam doesn't run exactly down the center. For example with a character's head, the UV seam can go down along the side of the head in front of the ear. The UV shell for the nearest ear can then be mirrored to use the area on the other side of the head.

This avoids the "Rorschach" effect and allows non-symmetrical details, but it still saves texture space because the two sides of the head can be mirrored (they're never seen at the same time anyhow).

Offset mirroring doesn't get rid of the seam, but it does move it off to a place where it can either be less obvious, or where it can be hidden in a natural seam on the model.

Flat Color Mirroring

Tutorial for painting out seams on mirrored tangent-space normal maps by warby solves seams by painting a flat set of normals along the seam, using neutral blue (128,128,255). However it only works along horizontal or vertical UV seams, not across any angled UVs. It also removes any details along the mirror seam, creating blank areas.

Element Mirroring

The mirror seam can be avoided completely when it doesn't run directly through any mesh. For example if there's a detached mesh element that runs down the center of the model, this can be uniquely mapped, while the meshes on either side can be mirrors of each other. Whenever the mirrored parts don't share any vertex normals with the non-mirrored parts, there won't be any seams.

<<Anchor(SmoothingGroupsAndHardEdges)>>

Smoothing Groups & Hard Edges

Each vertex in a mesh has at least one vertex normal. Vertex normals are used to control the direction a triangle will be lit from; if the normal is facing the light the triangle will be fully lit, if facing away from the light the triangle won't be lit.

Each vertex however can have more than one vertex normal. When two triangles have different vertex normals along their shared edge, this creates a shading seam, called a hard edge in most modeling tools. 3ds Max uses Smoothing Groups to create hard/soft edges, Maya uses Harden Edge and Soften Edge. These tools create hard and soft edges by splitting and combining the vertex normals.

When a mesh uses all soft normals (a single smoothing group) the lighting has to be interpolated across the extreme differences between the vertex normals. If your renderer doesn't support the same tangent basis that the baker uses, this can produce extreme shading differences across the model, which creates shading artifacts. It is generally best to reduce these extremes when you can because a mismatched renderer can only do so much to counteract it.

Hard edges are usually best where the model already has a natural seam. For example, you can add a hard edge along the rim of a car's wheel well, to prevent the inside of the wheel well from distorting the shading for the outside of the car body. Mechanical models usually need hard edges where ever the surface bends more than about 45 degrees.

For most meshes, the best results usually come from adding hard edges where ever there are UV seams. There are no hard rules however, you must experiment with different approaches to find what works best in your game.

When you use object-space normal maps the vertex normal problem goes away since you're no longer relying on the crude vertex normals of the mesh. An object-space normal map completely ignores vertex normals. Object-space mapping allows you to use all soft edges and no bevels on the low-res mesh, without showing lighting errors.

Hard Edge Discussions & Tutorials

Using Bevels

Bevels/chamfers generally improve the silhouette of the model, and can also help reflect specular highlights better.

However bevels tend to produce long thin triangles, which slow down the in-game rendering of your model. Real-time renderers have trouble rendering long thin triangles because they create a lot of sub-pixel areas to render.

Bevels also balloon the vertex count, which can increase the transform cost and memory usage. Hard edges increase the vertex count too, but not when the edge also shares a seam in UV space. For a good explanation of the vertex count issue, see Beautiful, Yet Friendly.

Using hard edges with matching UV shells tends to give better performance and better cosmetic results than using bevels. However there are differing opinions on this, see the Polycount thread "Maya transfer maps help".

Edited Vertex Normals

If you use bevels the shading will be improved by editing the vertex normals so the larger flat surfaces have perpendicular normals. The vertex normals are then forced to blend across the smaller bevel faces, instead of across the larger faces. See the Polycount thread Superspecular soft edges tutorial chapter 1.

<<Anchor(ModelingTheHighPolyMesh)>>

Modeling The High-Poly Mesh

Subdivision Surface Modeling and DigitalSculpting are the techniques most often used for modeling a normal map.

Some artists prefer to model the in-game mesh first, other artists prefer to model the high-res mesh first, and others start somewhere in the middle. The modeling order is ultimately a personal choice though, all three methods can produce excellent results:

  • Build the in-game model, then up-res it and sculpt it.
  • Build and sculpt a high resolution model, then build a new in-game model around that.
  • Build a basemesh model, up-res and sculpt it, then step down a few levels of detail and use that as a base for building a better in-game mesh.

If the in-game mesh is started from one of the subdivision levels of the basemesh sculpt, various edge loops can be collapsed or new edges can be cut to add/remove detail as necessary.

Sloped Extrusions

See also Modeling High/Low Poly Models for Next Gen Games by João "Masakari" Costa

mental ray Round Corners Bump

The mental ray renderer offers an automatic bevel rendering effect called Round Corners Bump that can be baked into a normal map. This is available in 3ds Max, Maya, and XSI. See Zero Effort Beveling for normal maps - by Robert "r_fletch_r" Fletcher.

Jeff Patton posted about how to expose Round Corners Bump in 3ds Max so you can use it in other materials.

Michael "cryrid" Taylor posted a tutorial about how to use Round Corners in XSI.

XSI is able to bake a good normal map with it, but 3ds Max seems to bake it incorrectly, and Maya isn't able to bake the effect at all. Maybe Max might be able to bake it correctly, if the .mi shader is edited to use the correct coordinate space?

Baking

The process of transferring normals from the high-res model to the in-game model is often called baking. The baking tool usually starts projecting a certain numerical distance out from the low-poly mesh, and sends rays inwards towards the high-poly mesh. When a ray intersects the high-poly mesh, it records the mesh's surface normal and saves it in the normal map.

To get an understanding of how all the options affect your normal map, do some test bakes on simple meshes like boxes. They generate quickly so you can experiment with UV mirroring, smoothing groups, etc. This helps you learn the settings that really matter.

Baking sub-sections: 

# Anti-Aliasing
# Baking Transparency
# Edge Padding
# High Poly Materials
# Reset Transforms
# Solving Intersections
# Solving Pixel Artifacts
# Solving Wavy Lines
# Triangulating
# Working with Cages

<<Anchor(AntiAliasing)>>

Anti-Aliasing

Turning on super-sampling or anti-aliasing (or whatever multi-ray casting is called in your normal map baking tool) will help to fix any jagged edges where the high-res model overlaps itself within the UV borders of the low-poly mesh, or wherever the background shows through holes in the mesh. Unfortunately this tends to render much much slower, and takes more memory.

One trick to speed this up is to render 2x the intended image size then scale the normal map down 1/2 in a paint program like Photoshop. The reduction's pixel resampling will add anti-aliasing for you in a very quick process. After scaling, make sure to re-normalize the map if your game doesn't do that already, because the un-normalized pixels in your normalmap may cause pixelly artifacts in your specular highlights. Re-normalizing can be done with NVIDIA's normal map filter for Photoshop.

3ds Max's supersampling doesn't work nicely with edge padding, it produces dark streaks in the padded pixels. If so then turn off padding and re-do the padding later, either by re-baking without supersampling or by using a Photoshop filter like the one that comes with Xnormal.

<<Anchor(BakingTransparency)>>

Baking Transparency

Sometimes you need to bake a normal map from an object that uses opacity maps, like a branch with opacity-mapped leaves. Unfortunately baking apps often completely ignore any transparency mapping on your high-poly mesh.

File:NormalMap$JoeWilson ivynormals error.jpg
3ds Max's RTT baker causes transparency errors.<
>image byJoe "EarthQuake" Wilson

To solve this, render a Top view of the mesh. This only works if you're using a planar UV projection for your low-poly mesh and you're baking a tangent-space normal map.

<<Anchor(EdgePadding)>>

Edge Padding

If a normal map doesn't have enough Edge Padding, this will create shading seams on the UV borders.

High Poly Materials

3ds Max will not bake a normal map properly if the high-res model has a mental ray Arch & Design material applied. If your normal map comes out mostly blank, either use a Standard material or none at all. For an example see the Polycount thread Render to Texture >:O.

Reset Transforms

Before baking, make sure your low-poly model's transforms have been reset. This is very important! Often during the modeling process a model will be rotated and scaled, but these compounded transforms can create a messy local "space" for the model, which in turn often creates rendering errors for normal maps.

In 3ds Max, use the Reset Xforms utility then Collapse the Modifier Stack. In Maya use Freeze Transformation. In XSI use the Freeze button.

<<Anchor(SolvingIntersections)>>

Solving Intersections

The projection process often causes problems like misses, or overlaps, or intersections. It can be difficult generating a clean normal map in areas where the high-poly mesh intersects or nearly intersects itself, like in between the fingers of a hand. Setting the ray distance too large will make the baker pick the other finger as the source normal, while setting the ray distance too small will lead to problems at other places on the mesh where the distances between in-game mesh and high-poly mesh are greater.

Fortunately there are several methods for solving these problems.

  1. Change the shape of the cage. Manually edit points on the projection cage to help solve tight bits like the gaps between fingers.
  2. Limit the projection to matching materials, or matching UVs.
  3. Explode the meshes. See the polycount thread Explode script needed (for baking purposes).
  4. Bake two or more times using different cage sizes, and combine them in Photoshop.

<<Anchor(SolvingPixelArtifacts)>>

Solving Pixel Artifacts

If you are using 3ds Max's Render To Texture to bake from one UV layout to another, you may see stray pixels scattered across the bake. This only happens if you are using a copy of the original mesh in the Projection, and that mesh is using a different UV channel than the original mesh.

There are two solutions for this:

  • Add a Push modifier to the copied mesh, and set it to a low value like 0.01.

- or -

  • Turn off Filter Maps in the render settings (Rendering menu > Render Setup > Renderer tab > uncheck Filter Maps). To prevent aliasing you may want to enable the Global Supersampler in Render Setup.

See also #Anti-Aliasing.

<<Anchor(SolvingWavyLines)>>

Solving Wavy Lines

When capturing from a cylindrical shape, often the differences between the low-poly mesh and the high-poly mesh will create a wavy edge in the normal map. There are a couple ways to avoid this:

  1. Adjust the shape of the cage to influence the directions the rays will be cast. At the bottom of this page of his normal map tutorial, Ben "poopinmymouth" Mathis shows how to do this in 3ds Max. Same method can be seen in the image below.
  2. Subdivide the low-res mesh so it more closely matches the high-res mesh. Jeff "airbrush" Ross has a video tutorial that shows how to do this in Maya.
  3. Paint out the wavy line. The normal map process tutorial by Ben "poopinmymouth" Mathis includes an example of painting out wavy lines in a baked normal map.
  4. Use a separate planar-projected mesh for the details that wrap around the barrel area, so the ray-casting is more even. For example to add tread around a tire, the tread can be baked from a tread model that is laid out flat, then that bake can layered onto the bake from the cylindrical tire mesh in a paint program.
  5. The polycount thread "approach to techy stuff" has some good tips for normal-mapping cylindrical shapes.

Triangulating

Before baking, it is usually best to triangulate the low-poly model, converting it from polygons into pure triangles. This prevents the vertex normals from being changed later on, which can create specular artifacts.

Sometimes a baking tool or a mesh exporter/importer will re-triangulate the polygons. A quad polygon is actually treated as two triangles, and the internal edge between them is often switched diagonally during modeling operations. When the vertices of the quad are moved around in certain shapes, the software's algorithm for polygon models tries to keep the quad surface in a "rational" non-overlapping shape. It does this by switching the internal edge between its triangles.

<<Anchor(WorkingWithCages)>>

Working with Cages

Cage has two meanings in the normal-mapping process: a low-poly base for subdivision surface modeling (usually called the basemesh), or a ray-casting mesh used for normal map baking. This section covers the ray-casting cage.

Most normal map baking tools allow you to use a distance-based raycast. A ray is sent outwards along each vertex normal, then at the distance you set a ray is cast back inwards. Where ever that ray intersects the high poly mesh, it will sample the normals from it.

File:NormalMap$Normalmap raycasting 1.jpg
Hard edges and a distance-based raycast (gray areas) cause ray misses (yellow) and ray overlaps (cyan).<
>image byDiego Castaño

Unfortunately with a distance-based raycast, split vertex normals will cause the bake to miss parts of the high-res mesh, causing errors and seams.

Some software allows you to use cage mesh option instead, which basically inflates a copy of the low-poly mesh, then raycasts inwards from each vertex. This ballooned-out mesh is the cage.

In 3ds Max the cage controls both the distance and the direction of the raycasting.

In Maya the cage only controls the distance; the ray direction matches the vertex normals (inverted).

This may have been fixed in the latest release...<
> In Xnormal the cage is split everywhere the model has hard edges, causing ray misses in the bake. You can fix the hard edge split problem but it involves an overly complex workflow. You must also repeat the whole process any time you change your mesh:

  1. Load the 3d viewer.
  2. Turn on the cage editing tools.
  3. Select all of the vertices.
  4. Weld all vertices.
  5. Expand the cage as you normally would.
  6. Save out your mesh using the Xnormal format.
  7. Make sure Xnormal is loading the correct mesh.

Painting

Don't be afraid to edit normal maps in Photoshop. After all it is just a texture, so you can clone, blur, copy, blend all you want... as long as it looks good of course. Some understanding of the way colors work in normal maps will go a long way in helping you paint effectively.

A normal map sampled from a high-poly mesh will nearly always be better than one sampled from a texture, since you're actually grabbing "proper" normals from an accurate, highly detailed surface. That means your normal map's pixels will basically be recreating the surface angles of your high-poly mesh, resulting in a very believable look.

If you only convert an image into a normal-map, it can look very flat, and in some cases it can be completely wrong unless you're very careful about your value ranges. Most image conversion tools assume the input is a heightmap, where black is low and white is high. If you try to convert a diffuse texture that you've painted, the results are often very poor. Often the best results are obtained by baking the large and mid-level details from a high-poly mesh, and then combined with photo-sourced "fine detail" normals for surface details such as fabric weave, scratches and grain.

Sometimes creating a high poly surface takes more time than your budget allows. For character or significant environment assets then that is the best route, but for less significant environment surfaces working from a heightmap-based texture will provide a good enough result for a much less commitment in time.

Flat Color

The color (128,128,255) creates normals that are completely perpendicular to the polygon, as long as the vertex normals are also perpendicular. Remember a normal map's per-pixel normals create offsets from the vertex normals. If you want an area in the normal map to be flat, so it creates no offsets from the vertex normals, then use the color (128,128,255).

This becomes especially obvious when mirroring a normal map and using a shader with a reflection ingredient. Reflection tends to accentuate the angles between the normals, so any errors become much more apparent.

[[attachment:normalmap_127seam.jpg|Media:NormalMap/attachments/normalmap_127seam.jpg

In a purely logical way, 127 seems like it would be the halfway point between 0 and 255. However 128 is the color that actually works in practice. When a test is done comparing (127,127,255) versus (128,128,255) it becomes obvious that 127 creates a slightly bent normal, and 128 creates a flat one.

This is because most game pipelines use unsigned normal maps. For details see the Polycount forum thread tutorial: fixing mirrored normal map seams.

<<Anchor(BlendingNormalMapsTogether)>>

Blending Normal Maps Together

Blending normal maps together is a quick way to add high-frequency detail like wrinkles, cracks, and the like. Fine details can be painted as a height map, then it can be converted into a normal map using one of the normal map tools. Then this "details" normal map can be blended with a geometry-derived normal map using one of the methods below.

Here is a comparison of four of the blending methods. Note that in these examples the default values were used for CrazyBump (Intensity 50, Strength 33, Strength 33), but the tool allows each layer's strength to be adjusted individually for stronger or milder results. Each of the normal maps below were re-normalized after blending.

[[Image:NormalMap$nrmlmap_blending_methods_Maps.png}}
The blended normal maps.<
>image byEric Chadwick

The four blending methods used above:

  1. CrazyBump by Ryan Clark blends normal maps together using calculations in 3D space rather than just in 2D. This does probably the best job at preserving details, and each layer's strength settings can be tweaked individually.
  2. Combining Normal Maps in Photoshop by Rod Green blends normal maps together using Linear Dodge mode for the positive values and Difference mode for the negative values, along with a Photoshop Action to simplify the process. It's free, but the results may be less accurate than CrazyBump.
  3. Making of Varga by Paul "paultosca" Tosca blends normal maps together using Overlay mode for the red and green channels and Multiply mode for the blue channel. This gives a slightly stronger bump than the Overlay-only method. Leo "chronic" Covarrubias has a step-by-step tutorial for this method in CG Bootcamp Combine Normal Maps.
  4. Normalmap Deepening by Ben "poopinmymouth" Mathis shows how to blend normal maps together using Overlay mode. CGTextures tutorial for the NVIDIA Photoshop filter by Scott Warren also shows how to create normalmaps using multiple layers (Note: to work with the Overlay blend mode each layer's Output Level should be 128 instead of 255, you can use the Levels tool for this).

The Getting good height from Nvidia-filter normalizing grayscale height thread on the Polycount forum has a discussion of different painting/blending options. Also see the 2D Tools section for painting and conversion tools.

Pre-Created Templates

A library of shapes can be developed and stored for later use, to save creation time for future normal maps. Things like screws, ports, pipes, and other doo-dads. These shapes can be stored as bitmaps with transparency so they can be layered into baked normal maps.

Re-normalizing

Re-normalizing means resetting the length of each normal in the map to 1.

A normal mapping shader takes the three color channels of a normal map and combines them to create the direction and length of each pixel's normal. These normals are then used to apply the scene lighting to the mesh. However if you edit normal maps by hand or if you blend multiple normal maps together this can cause those lengths to change. Most shaders expect the length of the normals to always be 1 (normalized), but some are written to re-normalize the normal map dynamically (for example, 3ds Max's Hardware Shaders do re-normalize).

If the normals in your normal map are not normalized, and your shader doesn't re-normalize them either, then you may see artifacts on the shaded surface... the specular highlight may speckle like crazy, the surface may get patches of odd shadowing, etc. To help you avoid this NVIDIA's normal map filter for Photoshop provides an easy way to re-normalize a map after editing; just use the Normalize Only option. Xnormal also comes with a Normalize filter for Photoshop.

Some shaders use compressed normal maps. Usually this means the blue channel is thrown away completely, so it's recalculated on-the-fly in the shader. However the shader has to re-normalize in order to recreate that data, so any custom normal lengths that were edited into the map will be ignored completely.

Ambient Occlusion into a Normal Map

If the shader doesn't re-normalize the normal map, an Ambient Occlusion Map can actually be baked into the normal map. This will shorten the normals in the crevices of the surface, causing the surface to receive less light there. This works with both diffuse and specular, or any other pass that uses the normal map, like reflection.

To bake the AO into a normal map, adjust the levels of the AO layer first so the darks only go as low as 128 gray, then set the AO layer to Darken mode. This will shorten the normals in the normalmap, causing the surface to receive less light in the darker areas.

This trick doesn't work with any shaders that re-normalize, like 3ds Max's Hardware Shaders. The shader must be altered to actually use the lengths of your custom normals; most shaders just assume all normals are 1 in length because this makes the shader code simpler. Also this trick will not work with most of the common normal map compression formats, which often discard the blue channel and recalculate it in the shader, which requires re-normalization.

For the best results it's usually best to multiply AO into the color map instead of editing it into the normal map.

<<Anchor(BacklightingExample)>>

Back Lighting Example

You can customize normal maps for some interesting effects. If you invert the blue channel of a tangent-space map, the normals will be pointing to the opposite side of the surface, which can simulate backlighting.

File:NormalMap$tree front.jpg
Tree simulating subsurface scattering (front view).<
>image byEric Chadwick

The tree leaves use a shader than adds together two diffuse maps, one using a regular tangent-space normal map, the other using the same normal map but with the blue channel inverted. This causes the diffuse map using the regular normal map to only get lit on the side facing the light (front view), while the diffuse map using the inverted normal map only gets lit on the opposite side of the leaves (back view). The leaf geometry is 2-sided but uses the same shader on both sides, so the effect works no matter the lighting angle. As an added bonus, because the tree is self-shadowing the leaves in shadow do not receive direct lighting, which means their backsides do not show the inverted normal map, so the fake subsurface scatter effect only appears where the light directly hits the leaves. This wouldn't work for a whole forest because of the computational cost of self-shadowing and double normal maps, but could be useful for a single "star" asset, or if LODs switched the distant trees to a model that uses a cheaper shader.

<<Anchor(ShadersAndSeams)>>

Shaders and Seams

You need to use the right kind of shader to avoid seeing seams where UV breaks occur. It must be written to use the same tangent basis that was used during baking. If the shader doesn't, the lighting will either be inconsistent across UV borders or it will show smoothing errors from the low-poly vertex normals.

Xnormal generates accurate normals when displayed in Xnormal, and the SDK includes a method to write your own custom tangent space generator for the tool.

3ds Max Shaders

The "Render To Texture" tool in 3ds Max 2011 and older generates tangent-space normal maps that render correctly in the offline renderer (scanline) but do not render correctly in the realtime viewport with the 3ds Max shaders. Max is using a different tangent basis for each. This is readily apparent when creating non-organic hard surface normalmaps; smoothing errors appear in the viewport that do not appear when rendered.

The errors can be fixed by using "Render To Texture" to bake a tangent-space or object-space map, and using the free "3Point Shader" by Christoph 'CrazyButcher' Kubisch and Per 'perna' Abrahamsen. The shader uses the same tangent basis as the baking tool, so it produces nearly flawless results. It also works with old bakes.

You can get OK results in the Max viewport using a tangent-space map baked in Maya, loading it in a Standard material, and enabling "Show Hardware Map in Viewport". Another method is to use Render To Texture to bake an object-space map then use Nspace to convert it into a tangent-space map then load that in a DirectX material and use the RTTNormalMap.fx shader.

Autodesk is aware of these issues, and plans to address them in an upcoming release. See these links for more information:

File:NormalMap$max2010 normalmap workarounds thumb.png
Viewport methods in 3ds Max 2010.<
>Actual size<
>image by Eric Chadwick

3ds Max Edit Normals Trick

After baking, if you add an Edit Normals modifier to your low-poly normalmapped model, this seems to "relax" the vertex normals for more accurate viewport shading. The modifier can be collapsed if desired.

Maya Shaders

Maya seems to correctly generate normals to view in realtime, with the correct tangent basis, with much less smoothing errors than 3ds Max.

  • BRDF shader by Brice Vandemoortele and Cedric Caillaud (more info in this Polycount thread) Update: New version here with many updates, including object-space normal maps, relief mapping, self-shadowing, etc. Make sure you enable cgFX shaders in the Maya plugin manager, then you can create them in the same way you create a Lambert, Phong etc. Switch OFF high quality rendering in the viewports to see them correctly too.
  • If you want to use the software renderer, use mental ray instead of Maya's software renderer because mental ray correctly interprets tangent space normals. The Maya renderer treats the normal map as a grayscale bump map, giving nasty results. Mental ray supports Maya's Phong shader just fine (amongst others), although it won't recognise a gloss map plugged into the "cosine power" slot. The slider still works though, if you don't mind having a uniform value for gloss. Spec maps work fine though. Just use the same set up as you would for viewport rendering. You'll need to have your textures saved as TGAs or similar for mental ray to work though. - from CheeseOnToast

<<Anchor(NormalMapCompression)>>

Normal Map Compression

Normal maps can take up a lot of memory. Compression can reduce the size of a map to 1/4 of what it was uncompressed, which means you can either increase the resolution or you can use more maps.

Usually the compression method is to throw away the Blue channel, because this can be re-computing at minimal cost in the shader code. Then the bitmap only has to store two color channels, instead of four (red, green, blue, and alpha).

DXT5nm Compression

DXT5nm is the same file format as DXT5 except before compression the red channel is moved into the alpha channel, the green channel is left as-is, and the red and blue channels are blanked with the same solid color. This re-arranging of the color channels is often called swizzling.

The Green and Alpha channels are used because in the DXT format they are compressed using somewhat higher bit depths than the Red and Blue channels. Red and Blue have to be filled with the same solid color because DXT uses a compression system that compares differences between the three color channels. If you try to store some kind of texture in Red and/or Blue (specular power, height map, etc.) then the compressor will create more compression artifacts because it has to compare all three channels.

There are some options in the NVIDIA DXT compressor that help reduce the artifacts if you want to add texture to the Red or Blue channels. The artifacts will be greater than if you keep Red and Blue empty, but it might be a tradeoff worth making. Some notes about this on the NVIDIA Developer Forums.

DXT1 Compression

DXT1 is also used sometimes for tangent-space normal maps, because it is half the size of a DXT5. The downside though is that it causes many more compression artifacts, so much so that most people end up not using it.

3Dc Compression

3Dc compression is also known as BC5 in DirectX 10. It works similar to DXT5nm, because it only stores the X and Y channels. The difference is it stores both the same way as the DXT5 Alpha channel, which is a slightly higher bit depth than DXT5nm's Green channel. 3Dc yields the best results of any listed algorithm for tangent space normal map compression, and requires no extra processing time or unique hardware. See 3Dc for more information.

A8L8 Compression

The DDS format !A8L8 isn't actually compressed, it's just two 8bit grayscale channels (256 grays each). It does save you from having to store all three color channels. Your shader has to recompute the blue channel for it to work. However, !A8L8 does not actually save any space in texture memory, it is typically converted to a four-channel 32bit texture when it's sent to the card. This format really only helps save disk space.

Links

3D Tools

See

2D Tools

See

Tutorials

Discussion

Discuss this page on the Polycount forums. Suggestions welcome.

Even though only one person has been editing this page so far, the information here was gathered from many different sources. We wish to thank all the contributors for their hard-earned knowledge. It is much appreciated! Template:Http://boards.polycount.net/images/smilies/pcount/icons/smokin.gif

Retrieved from "http://wiki.polycount.com/w/index.php?title=Normal_map&oldid=4679"
Personal tools
Namespaces

Variants
Actions
Navigation
Tools