VertexNormal

Vertex Normal

For the definition of a vertex normal, see the MSDN article Face and Vertex Normal Vectors

Game artists can manually alter the vertex normals to change the way an in-game mesh is lit, how it renders a reflection, how the fresnel falloff looks, etc. Basically any lighting effect that uses vertex normals can be tweaked.

Some game engines do not import edited vertex normals. The next version of UDK will import custom vertex normals via FBX, though currently it does not.

Foliage Shading

Foliage meshes in game are usually made using flat planes with a transparent texture containing a bunch of leaves (or grass blades). These textures are used instead of modeling each leaf or blade of grass discretely, because they would slow the framerate too much. Also the increased vertex count uses much more memory.

Flat planes generally do not shade very well when lit. To counteract this, the vertex normals can be bent to influence the shading, helping to hide the flatness of the geometry.

#!wiki green/solid
 {i} Note: This technique only works in a game engine that supports edited vertex normals. You have to test your asset pipeline to see.

Editing Normals in 3ds Max

Vertex normals can be manually bent in 3ds Max by using the Edit Normals modifier. See the 3ds Max 2011 online help.

Edited vertex normals are somewhat fragile in 3ds Max, they can be lost easily:

• If you take an Editable Mesh model, activate Attach, and try to attach an Editable Poly to it, the vertex normals of the Poly model will be reset. However using Attach in an Editable Poly model will work fine.
• If you Attach multiple meshes together, make sure each has at least one edited normal. Meshes without edited normals will have their vertex normals jumbled.
• If you Attach models that have been rotated, make sure to use Reset Xforms on them first. Attaching a rotated model to another will skew all its edited normals.
• If you use Detach to make a new object, the detached object's vertex normals will be reset.
• If you use Detach to create a new element in the same object, the detached element's vertex normals will be reset.

Normal Thief

NormalThief.ms is a MAXScript tool for 3ds Max by Mauricio B. Gehling. The script projects vertex normals from one model to another, automating what would be a tedious manual process. However the script is extremely slow... the rendering process can take 2 hours for a model with 2000 vertices.

There's no documentation, except what the author posted on Scriptspot.

The Reference is the model you wish to copy vertex normals from, and the Selected models are the ones that will get their normals bent to match it. The vertex counts of the two models don't need to match, the normals on the Selected objects will be bent to match the nearest normals on the Reference object. The Reference mesh needs to be collapsed into an Editable Mesh, but the Selected meshes do not.

In the example above, the red shape was a box with a single smoothing group applied, some edges cuts into it, and the vertices moved around some. The thinking behind it was that a simpler mesh would speed things up, but it actually didn't. Normal Thief seemed to work better when it was fed more of a blobby shape.

Normal Thief seems to work well when modeling roughly the shape of the canopy, and using a single smoothing group... modeling a shape that looks like how you want the tree's normals to point.

Exporting Edited Normals

Edited vertex normals can be exported from 3ds Max with some file formats:

File Format	Supports Edited Normals?
3DS	no
ASE	yes
CGF (CryEngine)	?
Collada	no
FBX	yes
OBJ	yes
PSK (Unreal Engine)	no
WRL (VRML)	no

Editing Normals in Maya

Vertex normals can be adjusted in Maya by using the Normals tools in the Polygons menu (Maya 2011 online help).

Vertex normals can also be baked from one model to another by using Transfer Attributes.

Normal Mapping

Vertex normals have a strong influence on tangent-space normal maps. Some game engines require a bit more geometry near areas that have a lot of change in angle (hard corners, etc.) otherwise the vertex normals fight with the normal map and create shading errors. Careful topology can minimize or avoid these errors altogether.

---