Difference between revisions of "Texture Baking"

Revision as of 07:42, 23 June 2015

Texture baking is the process of transferring details from one model to another.

The baking tool starts a certain distance out from the model (usually a low-resolution model for game use), and casts rays inwards towards another model (usually a high-resolution sculpt). When a ray intersects the 2nd model, it records the surface detail and saves that into a texture map, using the model's Texture Coordinates.

Baking tools support multiple map types. High-resolution normals go into a Normal map, occluded surfaces go into an Ambient occlusion map, etc.

Baking Workflow

1. Re-Topo.
   • Use ReTopologyModeling to build the low-resolution in-game model.
   • Create good topology for baking. See Normal Map Modeling#Contiguous Meshes and Understanding averaged normals and ray projection/Who put waviness in my normal map?.
2. UV splits.
   • For tangent-space normal mapping, split the UVs for every hard edge (where vertex normals are split, or different smoothing groups are used, same thing). See You're making me hard. Making sense of hard edges, uvs, normal maps and vertex counts, and Earthquake on separating smoothing groups in the UV.
3. Mirror.
   • If creating a symmetrical model, mirror those parts. See Normal Map Modeling#Mirroring.
4. UV offset.
   • See JedTheKrampus on offsetting Mirrored UVs, Texture Baking#UV Coordinates.
5. Triangulate.
   • See Texture Baking#Triangulation, and Polygon Count#Polygons Vs. Triangles.
6. Optimize.
   • Optimize the high-resolution model to speed up bakes, to avoid running out of RAM while baking, and to keep 3d file sizes manageable. See Tools#Decimation_Software.
   • If it's a sculpt reduce the vertex count to a manageable file size before exporting. Sculpting tools like Zbrush create triangles smaller than the bake pixels, which will increase baking time significantly without actually improving the bake.
   • If it's a subdivision surface choose an appropriate resolution. Use just enough subdivisions to get a smooth surface at the baking resolution, and no more.
7. Explode.
   • Interpenetrating parts can cause baking artifacts, because neighboring surfaces will capture parts of each other. See Texture Baking#Solving Intersections.
   • Exploding means separating non-welded surfaces, mesh elements, Zbrush subtools, etc. so there is space between them. Use the same separation for the highpoly model and the lowpoly model. After the bake, move the pieces back together.
   • If the modeling software has animation, you can keyframe the explode to easily reverse it after baking. Tools are also available, see Explode script needed (for baking purposes).
8. Cage.
   • Create an inflated copy of the low-resolution model, which encompasses the high-resolution model. See Texture Baking#Cages. Some tools create this automatically.
9. Which baker?
   • Texture Baking#Baking Tools
10. Import & Test.
    • Import the high-resolution model, the low-resolution model, and the cage.
    • Test the bake settings using lower values for faster iteration. Smaller render sizes, lower anti-aliasing samples, etc.
    • Fix errors and re-bake until solved. Look for overlaps, gaps, skewed details. Adjust the bake settings, low-poly model, cage, etc. See Skew you buddy! Making sense of skewed normal map details., and Skewmesh Tutorial.
    • Some errors can be painted out later. However any painting must be repeated if the model is re-baked. Also, painting on a normal map can introduce more artifacts.
11. Render.
    • Increase the settings to full quality.
    • Use anti-aliasing or super-sampling. See Texture Baking#Anti-Aliasing.
    • Use 16-bit. See Of Bit Depths, Banding and Normal Maps.
    • Object space, if converting to a specific tangent space. See Synched Workflow, and Converting Between Spaces.
12. Convert.
    • Convert to a specific tangent space. See Synched Workflow, and Converting Between Spaces.
13. Edit.
    • Combine bakes. See Ambient occlusion map#EarthQuake's Baking Method
    • Add details from photos or other bakes. See RNM Normal Map Combiner, and Combine Normal.
    • Paint out errors. Do this only as a last resort though. Any painting must be repeated if the model is re-baked. Painting on a normal map can introduce artifacts.
    • Swizzle. See Normal Map Technical Details#Common Swizzle Coordinates.
    • Re-normalize.
    • Reduce to 8-bit. See Of Bit Depths, Banding and Normal Maps.
14. View & Check.
    • View the finished in-game model and textures in your game engine, and check for errors.

UV Coordinates

Normal map baking tools only capture details within the 0-1 UV square, any UV bits outside this area are ignored.

Only one copy of the forward-facing UVs should remain in the 0-1 UV square at baking time. If the mesh uses overlapping UVs, this will likely cause artifacts to appear in the baked map, since the baker will try render each UV shell into the map. Before baking, it's best to move all the overlaps and mirrored bits outside the 0-1 square.

The mirrored UVs (in red) are offset 1 unit before baking.
Image by Eric Chadwick.

If you move all the overlaps and mirrored bits exactly 1 UV unit (any whole number will do), then you can leave them there after the bake and they will still be mapped correctly. You can move them back if you want, it doesn't matter to most game engines. Be aware that ZBrush does use UV offsets to manage mesh visibility, however this usually doesn't matter because the ZBrush cage mesh is often a different mesh than the in-game mesh used for baking.

You should avoid changing the UVs after baking the normal map, because rotating or mirroring UVs after baking will cause the normal map not to match the tangent basis anymore, which will likely cause lighting problems.

In 3ds Max, W is a third texture coordinate. It's used for 3D procedural textures and for storing vertex color in UV channels (you need 3 axes for RGB, so UVW can store vertex color). Bake problems can be avoided by moving any overlapping UVs to -1 on the W axis, with the same results as moving them 1 unit on the U or V axes. The tool Render To Texture will always bake whatever UVs are the highest along the W axis. However using W can be messy... it's generally hidden unless you purposefully look for it (bad for team work), doesn't get preserved on export to other apps, and high W values can prevent selecting and/or welding UVs.

When shared edges are at different angles in UV space, different colors will show up along the seam. The tangent basis uses these colors to light the model properly.
Image by Eric Chadwick.

When you look at a tangent-space normal map for a character, you typically see different colors along the UV seams. This is because the UV shells are often oriented at different angles on the mesh, a necessary evil when translating the 3D mesh into 2D textures. The body might be mapped with a vertical shell, and the arm mapped with a horizontal one. This requires the normals in the normal map to be twisted for the different orientations of those UV shells. The UVs are twisted, so the normals must be twisted in order to compensate. The tangent basis helps reorient (twist) the lighting as it comes into the surface's local space, so the lighting will then look uniform across the normal mapped mesh.

When an artist tiles a tangent-space normal map across an arbitrary mesh, like a landscape, this tends to shade correctly because the mesh has a uniform direction in tangent space. If the mesh has discontinuous UV coordinates (UV seams), or the normal map has large directional gradients across it, the tangent space won't be uniform anymore so the surface will probably have shading seams.

Triangulation

Before baking, it is usually best to triangulate the low-poly model, converting it from polygons into pure triangles. This prevents the vertex normals from being changed later on, which can create specular artifacts.

When quads are triangulated in Modo, the internal edges are sometimes flipped, which causes shading differences. Image by James "Talon" O'Hare.

Sometimes a baking tool or a mesh exporter/importer will re-triangulate the polygons. A quad polygon is actually treated as two triangles, and the internal edge between them is often switched diagonally during modeling operations. When the vertices of the quad are moved around in certain shapes, the software's algorithm for polygon models tries to keep the quad surface in a "rational" non-overlapping shape. It does this by switching the internal edge between its triangles.

The specular highlight is affected by triangulation. Flip edges to fix skewing. See the Polycount thread Skewed Specular Highlight? for pictures and more info. Image by Robert "TychoVII" Kreps.

Cages

Cage has two meanings in the normal-mapping process: a low-poly base for Subdivision Surface Modeling (usually called the BaseMesh), or a ray-casting mesh used for normal map baking. This section covers the ray-casting cage.

Most normal map baking tools allow you to use a distance-based raycast. A ray is sent outwards along each vertex normal, then at the distance you set a ray is cast back inwards. Where ever that ray intersects the high poly mesh, it will sample the normals from it.

• 

  Hard edges and a distance-based raycast (gray areas) cause ray misses (yellow) and ray overlaps (cyan). Image by Diego Castaño.

• 

  The gray area shows that using all soft edges (or hard edges and a cage-based raycast) will avoid ray-casting errors from split normals. Image by Diego Castaño.

Unfortunately with a distance-based raycast, split vertex normals will cause the bake to miss parts of the high-res mesh, causing errors and seams.

Some software allows you to use cage mesh option instead, which basically inflates a copy of the low-poly mesh, then raycasts inwards from each vertex. This ballooned-out mesh is the cage.

In 3ds Max the cage controls both the distance and the direction of the raycasting.

In Maya the cage only controls the distance; the ray direction matches the vertex normals (inverted).

• The tutorial 3DTutorials/Modeling High-Low Poly Models for Next Gen Games by João "Masakari" Costa has more examples of ray-casting, plus how to get better results from the bake.

Anti-Aliasing

Aliasing errors can occur both during the bake (from overlaps in the high-poly model) and in-game (from overlaps in the low-poly model).

Contiguous low-poly meshes are better in general, as long as the triangle budget allows it. Otherwise interior parts will be aliased in-game where the edges overlap one another. See Normal Map Modeling#Contiguous Meshes.

For the high-poly model, turning on super-sampling or anti-aliasing (or whatever multi-ray casting is called in your normal map baking tool) will help to fix any jagged edges where the high-poly model overlaps itself within the UV borders of the low-poly mesh, or wherever the background shows through holes in the mesh. Unfortunately this tends to render much much slower, and takes more memory.

A bake without anti-aliasing shows artifacts where the high-poly mesh has overlaps.
Image by 'knak47'

One trick to speed this up is to render 2x the intended image size then scale the normal map down 1/2 in a paint program like Photoshop. The reduction's pixel resampling will add anti-aliasing for you in a very quick process. After scaling, make sure to re-normalize the map if your game doesn't do that already, because the un-normalized pixels in your normalmap may cause pixelly artifacts in your specular highlights. Re-normalizing can be done with NVIDIA's normal map filter for Photoshop.

3ds Max's supersampling doesn't work nicely with edge padding, it produces dark streaks in the padded pixels. If so then turn off padding and re-do the padding later, either by re-baking without supersampling or by using a Photoshop filter like the one that comes with Xnormal.

Transparency

Sometimes you need to bake a normal map from an object that uses opacity maps, like a branch with opacity-mapped leaves. Unfortunately baking apps often completely ignore any transparency mapping on your high-poly mesh.

• 

  3ds Max's RTT baker causes transparency errors. Image by Joe "EarthQuake" Wilson.

• 

  Use colored lighting to bake perfect transparency. Image by Joe "EarthQuake" Wilson.

To solve this, render a Top view of the mesh. This only works if you're using a planar UV projection for your low-poly mesh and you're baking a tangent-space normal map.

• Make sure the Top view matches the dimensions of the planar UV projection used by the low-poly mesh. It helps to use an orthographic camera for precise placement.
• On the high-poly mesh either use a specific lighting setup or a use special material shader:
• 1) The lighting setup is described in these tutorials:
• * Creating A Normal Map Right In Your 3D App by Ben Cloward
• *Generating High Fidelity Normal Maps with 3-D Software by Dave McCoy, Graphics Techniques Consultant, Xbox Content and Design Team
• 2) The material shader does the same thing, but doesn't require lights.
• * NormalTexMap scripted map for 3ds Max by Dave Locke.
• * InfoTexture map plugin for 3ds Max by John Burnett

• 

  The lighting setup for top-down rendering. Image by Ben Cloward.

Edge Padding

If a normal map doesn't have enough Edge Padding, this will create shading seams on the UV borders.

High Poly Materials

3ds Max will not bake a normal map properly if the high-res model has a mental ray Arch & Design material applied. If your normal map comes out mostly blank, either use a Standard material or none at all. For an example see the Polycount thread Render to Texture >:O.

Reset Transforms

Before baking, make sure your low-poly model's transforms have been reset. This is very important! Often during the modeling process a model will be rotated and scaled, but these compounded transforms can create a messy local "space" for the model, which in turn often creates rendering errors for normal maps.

In 3ds Max, use the Reset Xforms utility then Collapse the Modifier Stack. In Maya use Freeze Transformation. In XSI use the Freeze button.

Solving Intersections

The projection process often causes problems like misses, or overlaps, or intersections. It can be difficult generating a clean normal map in areas where the high-poly mesh intersects or nearly intersects itself, like in between the fingers of a hand. Setting the ray distance too large will make the baker pick the other finger as the source normal, while setting the ray distance too small will lead to problems at other places on the mesh where the distances between in-game mesh and high-poly mesh are greater.

Fortunately there are several methods for solving these problems.

1. Create a contiguous low-poly mesh. See Normal Map Modeling#Contiguous Meshes.
2. Change the shape of the cage. Manually edit points on the projection cage to help solve tight bits like the gaps between fingers.
3. Explode the meshes. See the polycount thread Explode script needed (for baking purposes).
4. Limit the projection to matching materials, or matching UVs.
5. Bake two or more times using different cage sizes, and combine them in Photoshop.

Solving Pixel Artifacts

Random pixel artifacts in the bake.
Image by Eric Chadwick

If you are using 3ds Max's Render To Texture to bake from one UV layout to another, you may see stray pixels scattered across the bake. This only happens if you are using a copy of the original mesh in the Projection, and that mesh is using a different UV channel than the original mesh.

There are two solutions for this:

• Add a Push modifier to the copied mesh, and set it to a low value like 0.01.

- or -

• Turn off Filter Maps in the render settings (Rendering menu > Render Setup > Renderer tab > uncheck Filter Maps). To prevent aliasing you may want to enable the Global Supersampler in Render Setup.

See also #Anti-Aliasing.

Solving Wavy Lines

When capturing from a cylindrical shape, often the differences between the low-poly mesh and the high-poly mesh will create a wavy edge in the normal map. There are a couple ways to avoid this:

1. The best way... create your lowpoly model with better supporting edges. See the Polycount threads Understanding averaged normals and ray projection/Who put waviness in my normal map?, approach to techy stuff, Any tips for normal mapping curved surface?.
2. Bake an object-space map using more edges, then convert to tangent space using the original mesh and Handplane. See the Polycount thread Question about continuous meshes for low poly bakes.
3. Adjust the shape of the cage to influence the directions the rays will be cast. Beware... this work will have to be re-done every time you edit the lowpoly mesh, as the cage will be invalidated. At the bottom of this page of his normal map tutorial, Ben "poopinmymouth" Mathis shows how to do this in 3ds Max. Same method can be seen in the image below.
4. Subdivide the low-res mesh so it more closely matches the high-res mesh. Beware... this will cause the normal map not to match your lowpoly vertex normals, probably causing shading errors. Jeff "airbrush" Ross has a video tutorial that shows how to do this in Maya.
5. Paint out the wavy line. Beware... this work will have to be re-done every time you re-bake the normal map. The normal map process tutorial by Ben "poopinmymouth" Mathis includes an example of painting out wavy lines in a baked normal map.
6. Use a separate planar-projected mesh for the details that wrap around the barrel area, so the ray-casting is more even. Beware... this will cause the normal map not to match your lowpoly vertex normals, probably causing shading errors. For example to add tread around a tire, the tread can be baked from a tread model that is laid out flat, then that bake can layered onto the bake from the cylindrical tire mesh in a paint program.

Adjusting the shape of the cage to remove distortion.
Image by Timothy "tpe" Evison

Baking Tools

• 3ds Max: Render To Texture
• Airborn Studios Tools - Backstube (Xnormal batcher), Packstube (map packer).
• Blender: Render Bake
• Handplane (tangent space conversion)
• Knald
• Maya: Transfer Maps
• MightyBake
• Modo: Baking
• Substance Designer: Bakers
• Xnormal
• Zbrush: Create Normal Map