Texture Baking

From polycount
Revision as of 15:10, 1 June 2015 by EricChadwick (Talk | contribs)

Jump to: navigation, search

Texture baking is the process of transferring details from one model to another.

The baking tool starts a certain distance out from the model (usually a low-resolution model for game use), and casts rays inwards towards another model (usually a high-resolution sculpt). When a ray intersects the 2nd model, it records the surface detail and saves that into a texture map.

Baking tools support multiple map types. High-resolution normals go into a Normal map, occluded surfaces go into an Ambient occlusion map, etc.

Baking Workflow

  1. Re-Topo.
  2. Triangulate.
  3. Optimize.
    • Optimize the high-resolution model to speed up bakes, to avoid running out of RAM while baking, and to keep 3d file sizes manageable.
    • If it's a sculpt reduce the vertex count to a manageable file size before exporting. See Tools#Decimation_Software. Sculpting tools like Zbrush create triangles smaller than the bake pixels, which will increase baking time significantly without actually improving the bake.
    • If it's a subdivision surface choose an appropriate resolution. Use just enough subdivisions to get a smooth surface at the baking resolution, and no more.
  4. Explode.
    • Interpenetrating parts can cause baking artifacts, because neighboring surfaces will capture parts of each other. See Texture Baking#Solving Intersections.
    • Separate non-welded surfaces, mesh elements, Zbrush subtools, etc. so there is space between them. Use the same separation for the highpoly model and the lowpoly model. After the bake, move the pieces back together.
    • If the modeling software has animation, keyframe the explode. Tools are also available, see Explode script needed (for baking purposes).
  5. Cage.
    • Create an inflated copy of the low-resolution model, which encompasses the high-resolution model. See Texture Baking#Cages.
  6. Which baker?
  7. Import & Test.
    • Import the high-resolution model, the low-resolution model, and the cage.
    • Test the bake settings using lower values for faster iteration. Smaller render sizes, lower anti-aliasing samples, etc.
    • Fix errors and re-bake until solved. Look for overlaps, gaps, skewed details. Adjust the bake settings, low-poly model, cage, etc. See Skew you buddy! Making sense of skewed normal map details., and Skewmesh Tutorial.
    • Some errors can be painted out later. However any painting must be repeated if the model is re-baked. Also, painting on a normal map can introduce more artifacts.
  8. Render.
  9. Convert.
  10. Edit.

UV Coordinates

Normal map baking tools only capture normals within the 0-1 UV square, any UV bits outside this area are ignored.

Only one copy of the forward-facing UVs should remain in the 0-1 UV square at baking time. If the mesh uses overlapping UVs, this will likely cause artifacts to appear in the baked map, since the baker will try render each UV shell into the map. Before baking, it's best to move all the overlaps and mirrored bits outside the 0-1 square.

The mirrored UVs (in red) are offset 1 unit before baking.
Image by Eric Chadwick.

If you move all the overlaps and mirrored bits exactly 1 UV unit (any whole number will do), then you can leave them there after the bake and they will still be mapped correctly. You can move them back if you want, it doesn't matter to most game engines. Be aware that ZBrush does use UV offsets to manage mesh visibility, however this usually doesn't matter because the ZBrush cage mesh is often a different mesh than the in-game mesh used for baking.

You should avoid changing the UVs after baking the normal map, because rotating or mirroring UVs after baking will cause the normal map not to match the tangent basis anymore, which will likely cause lighting problems.

In 3ds Max, W is a third texture coordinate. It's used for 3D procedural textures and for storing vertex color in UV channels (you need 3 axes for RGB, so UVW can store vertex color). Bake problems can be avoided by moving any overlapping UVs to -1 on the W axis, with the same results as moving them 1 unit on the U or V axes. The tool Render To Texture will always bake whatever UVs are the highest along the W axis. However using W can be messy... it's generally hidden unless you purposefully look for it (bad for team work), doesn't get preserved on export to other apps, and high W values can prevent selecting and/or welding UVs.

When shared edges are at different angles in UV space, different colors will show up along the seam. The tangent basis uses these colors to light the model properly.
Image by Eric Chadwick.

When you look at a tangent-space normal map for a character, you typically see different colors along the UV seams. This is because the UV shells are often oriented at different angles on the mesh, a necessary evil when translating the 3D mesh into 2D textures. The body might be mapped with a vertical shell, and the arm mapped with a horizontal one. This requires the normals in the normal map to be twisted for the different orientations of those UV shells. The UVs are twisted, so the normals must be twisted in order to compensate. The tangent basis helps reorient (twist) the lighting as it comes into the surface's local space, so the lighting will then look uniform across the normal mapped mesh.

When an artist tiles a tangent-space normal map across an arbitrary mesh, like a landscape, this tends to shade correctly because the mesh has a uniform direction in tangent space. If the mesh has discontinuous UV coordinates (UV seams), or the normal map has large directional gradients across it, the tangent space won't be uniform anymore so the surface will probably have shading seams.

Triangulation

Before baking, it is usually best to triangulate the low-poly model, converting it from polygons into pure triangles. This prevents the vertex normals from being changed later on, which can create specular artifacts.

When quads are triangulated in Modo, the internal edges are sometimes flipped, which causes shading differences.
Image by "Talon" O'Hare


Sometimes a baking tool or a mesh exporter/importer will re-triangulate the polygons. A quad polygon is actually treated as two triangles, and the internal edge between them is often switched diagonally during modeling operations. When the vertices of the quad are moved around in certain shapes, the software's algorithm for polygon models tries to keep the quad surface in a "rational" non-overlapping shape. It does this by switching the internal edge between its triangles.


The specular highlight is affected by triangulation. Flip edges to fix skewing. See the Polycount thread Skewed Specular Highlight? for pictures and more info.
Image by Robert "TychoVII" Kreps


Cages

Cage has two meanings in the normal-mapping process: a low-poly base for subdivision surface modeling (usually called the basemesh), or a ray-casting mesh used for normal map baking. This section covers the ray-casting cage.

Most normal map baking tools allow you to use a distance-based raycast. A ray is sent outwards along each vertex normal, then at the distance you set a ray is cast back inwards. Where ever that ray intersects the high poly mesh, it will sample the normals from it.

Normalmap raycasting 1.jpg Normalmap raycasting 2.jpg
Hard edges and a distance-based raycast (gray areas) cause ray misses (yellow) and ray overlaps (cyan).
Image by Diego Castaño 		The gray area shows that using all soft edges (or hard edges and a cage-based raycast) will avoid ray-casting errors from split normals.
Image by Diego Castaño

Unfortunately with a distance-based raycast, split vertex normals will cause the bake to miss parts of the high-res mesh, causing errors and seams.

Some software allows you to use cage mesh option instead, which basically inflates a copy of the low-poly mesh, then raycasts inwards from each vertex. This ballooned-out mesh is the cage.

In 3ds Max the cage controls both the distance and the direction of the raycasting.

In Maya the cage only controls the distance; the ray direction matches the vertex normals (inverted).

This may have been fixed in the latest release...<
> In Xnormal the cage is split everywhere the model has hard edges, causing ray misses in the bake. You can fix the hard edge split problem but it involves an overly complex workflow. You must also repeat the whole process any time you change your mesh:

  1. Load the 3d viewer.
  2. Turn on the cage editing tools.
  3. Select all of the vertices.
  4. Weld all vertices.
  5. Expand the cage as you normally would.
  6. Save out your mesh using the Xnormal format.
  7. Make sure Xnormal is loading the correct mesh.


Anti-Aliasing

Turning on super-sampling or anti-aliasing (or whatever multi-ray casting is called in your normal map baking tool) will help to fix any jagged edges where the high-res model overlaps itself within the UV borders of the low-poly mesh, or wherever the background shows through holes in the mesh. Unfortunately this tends to render much much slower, and takes more memory.

A bake without anti-aliasing shows artifacts where the high-poly mesh has overlaps.
Image by 'knak47'

One trick to speed this up is to render 2x the intended image size then scale the normal map down 1/2 in a paint program like Photoshop. The reduction's pixel resampling will add anti-aliasing for you in a very quick process. After scaling, make sure to re-normalize the map if your game doesn't do that already, because the un-normalized pixels in your normalmap may cause pixelly artifacts in your specular highlights. Re-normalizing can be done with NVIDIA's normal map filter for Photoshop.

3ds Max's supersampling doesn't work nicely with edge padding, it produces dark streaks in the padded pixels. If so then turn off padding and re-do the padding later, either by re-baking without supersampling or by using a Photoshop filter like the one that comes with Xnormal.


Transparency

Sometimes you need to bake a normal map from an object that uses opacity maps, like a branch with opacity-mapped leaves. Unfortunately baking apps often completely ignore any transparency mapping on your high-poly mesh.

JoeWilson ivynormals error.jpg JoeWilson ivynormals rendered.jpg
3ds Max's RTT baker causes transparency errors.
image by Joe "EarthQuake" Wilson 		The lighting method bakes perfect transparency.
image by Joe "EarthQuake" Wilson

To solve this, render a Top view of the mesh. This only works if you're using a planar UV projection for your low-poly mesh and you're baking a tangent-space normal map.

The lighting setup for top-down rendering.
Image by Ben Cloward


Edge Padding

If a normal map doesn't have enough Edge Padding, this will create shading seams on the UV borders.


High Poly Materials

3ds Max will not bake a normal map properly if the high-res model has a mental ray Arch & Design material applied. If your normal map comes out mostly blank, either use a Standard material or none at all. For an example see the Polycount thread Render to Texture >:O.


Reset Transforms

Before baking, make sure your low-poly model's transforms have been reset. This is very important! Often during the modeling process a model will be rotated and scaled, but these compounded transforms can create a messy local "space" for the model, which in turn often creates rendering errors for normal maps.

In 3ds Max, use the Reset Xforms utility then Collapse the Modifier Stack. In Maya use Freeze Transformation. In XSI use the Freeze button.


Solving Intersections

The projection process often causes problems like misses, or overlaps, or intersections. It can be difficult generating a clean normal map in areas where the high-poly mesh intersects or nearly intersects itself, like in between the fingers of a hand. Setting the ray distance too large will make the baker pick the other finger as the source normal, while setting the ray distance too small will lead to problems at other places on the mesh where the distances between in-game mesh and high-poly mesh are greater.

Fortunately there are several methods for solving these problems.

  1. Change the shape of the cage. Manually edit points on the projection cage to help solve tight bits like the gaps between fingers.
  2. Limit the projection to matching materials, or matching UVs.
  3. Explode the meshes. See the polycount thread Explode script needed (for baking purposes).
  4. Bake two or more times using different cage sizes, and combine them in Photoshop.


Solving Pixel Artifacts

Random pixel artifacts in the bake.
Image by Eric Chadwick

If you are using 3ds Max's Render To Texture to bake from one UV layout to another, you may see stray pixels scattered across the bake. This only happens if you are using a copy of the original mesh in the Projection, and that mesh is using a different UV channel than the original mesh.

There are two solutions for this:

  • Add a Push modifier to the copied mesh, and set it to a low value like 0.01.

- or -

  • Turn off Filter Maps in the render settings (Rendering menu > Render Setup > Renderer tab > uncheck Filter Maps). To prevent aliasing you may want to enable the Global Supersampler in Render Setup.

See also #Anti-Aliasing.


Solving Wavy Lines

When capturing from a cylindrical shape, often the differences between the low-poly mesh and the high-poly mesh will create a wavy edge in the normal map. There are a couple ways to avoid this:

  1. The best way... create your lowpoly model with better supporting edges. See the Polycount threads Understanding averaged normals and ray projection/Who put waviness in my normal map?, approach to techy stuff, Any tips for normal mapping curved surface?.
  2. Adjust the shape of the cage to influence the directions the rays will be cast. Beware... this work will have to be re-done every time you edit the lowpoly mesh, as the cage will be invalidated. At the bottom of this page of his normal map tutorial, Ben "poopinmymouth" Mathis shows how to do this in 3ds Max. Same method can be seen in the image below.
  3. Subdivide the low-res mesh so it more closely matches the high-res mesh. Beware... this will cause the normal map not to match your lowpoly vertex normals, probably causing shading errors. Jeff "airbrush" Ross has a video tutorial that shows how to do this in Maya.
  4. Paint out the wavy line. Beware... this work will have to be re-done every time you re-bake the normal map. The normal map process tutorial by Ben "poopinmymouth" Mathis includes an example of painting out wavy lines in a baked normal map.
  5. Use a separate planar-projected mesh for the details that wrap around the barrel area, so the ray-casting is more even. Beware... this will cause the normal map not to match your lowpoly vertex normals, probably causing shading errors. For example to add tread around a tire, the tread can be baked from a tread model that is laid out flat, then that bake can layered onto the bake from the cylindrical tire mesh in a paint program.
Adjusting the shape of the cage to remove distortion.
Image by Timothy "tpe" Evison

Baking Tools

Retrieved from "http://wiki.polycount.com/w/index.php?title=Texture_Baking&oldid=6419"
Personal tools
Namespaces

Variants
Actions
Navigation
Tools