Revision as of 13:10, 9 July 2010

Vertex Normal

For the definition of a vertex normal, see the MSDN article Face and Vertex Normal Vectors

Game artists can manually alter the vertex normals to change the way an in-game mesh is lit, how it renders a reflection, how the fresnel falloff looks, etc. Basically any lighting effect that uses vertex normals can be tweaked.

Some game engines do not import edited vertex normals. The next version of UDK will import custom vertex normals via FBX, though currently it does not.

Foliage Shading

Foliage meshes in game are usually made using flat planes with a transparent texture containing a bunch of leaves (or grass blades). These textures are used instead of modeling each leaf or blade of grass discretely, because they would slow the framerate too much. Also the increased vertex count uses much more memory.

Flat planes generally do not shade very well when lit. To counteract this, the vertex normals can be bent to influence the shading, helping to hide the flatness of the geometry.

#!wiki green/solid
 {i} Note: This technique only works in a game engine that supports edited vertex normals. You have to test your asset pipeline to see.

Editing Normals in 3ds Max

Vertex normals can be manually bent in 3ds Max by using the Edit Normals modifier. See the 3ds Max 2011 online help.

Edited vertex normals are somewhat fragile in 3ds Max, they can be lost easily:

• If you take an Editable Mesh model, activate Attach, and try to attach an Editable Poly to it, the vertex normals of the Poly model will be reset. However using Attach in an Editable Poly model will work fine.
• If you Attach multiple meshes together, make sure each has at least one edited normal. Meshes without edited normals will have their vertex normals jumbled.
• If you Attach models that have been rotated, make sure to use Reset Xforms on them first. Attaching a rotated model to another will skew all its edited normals.
• If you use Detach to make a new object, the detached object's vertex normals will be reset.
• If you use Detach to create a new element in the same object, the detached element's vertex normals will be reset.

Slide Normal Thief

SlideNormalThief by the studio Slide London is a 3ds Max MAXScript tool that projects vertex normals from one model to another, automating what would otherwise be a tedious manual process.

The script is based on a much-slower script called NormalThief.ms written by Mauricio B. Gehling.

The Reference Object is the model you wish to copy vertex normals from, and the Target Object is the model that will get its normals bent to match. The vertex counts of the two models don't need to match, the normals on the Target will be bent to match the nearest normals on the Reference. Both models need to have an Edit Normals modifier applied.

In the example above, the red shape was a box with a single smoothing group applied, some edges cuts into it, and the vertices moved around some. The tool seems to work well when modeling roughly the shape of the canopy, and using a single smoothing group... modeling a shape that looks like how you want the tree's normals to point.

Exporting Edited Normals

Edited vertex normals can be exported from 3ds Max with some file formats:

Editing Normals in Maya

Vertex normals can be adjusted in Maya by using the Normals tools in the Polygons menu (Maya 2011 online help).

Vertex normals can also be baked from one model to another by using Transfer Attributes.

Normal Mapping

Vertex normals have a strong influence on tangent-space normal maps. Some game engines require a bit more geometry near areas that have a lot of change in angle (hard corners, etc.) otherwise the vertex normals fight with the normal map and create shading errors. Careful topology can minimize or avoid these errors altogether.