Difference between revisions of "Vertex skinning"

Revision as of 14:47, 6 August 2014

Vertex Skinning, also known as weighting or binding (and archaically known as 'enveloping'), is creating a relationship of vertices to bones. When bones transform, the vertices transform along with them proportionally to their weight to the bone.

Explanation

A good explanation is given by Jason Osippa in Stop Staring:

[Skinning] really doesn't understand motion; it only understands positions. If you have one point weighted 100% to a joint, it will indeed move in arcs, and follow rotations; it's followign the motions of the join 100%, and since the joint is rotating, the point seems like it is rotating. As soon as you're at less than 100% weighting, it becomes apparent that the point is calculating a distance, not a motion.

Think of it this way- if you were to take the original joint and vertex position, and the position of the transformed joint, the vert at 100% weighting would behave just like a child. If you draw a line from the original vert position to the 100% vert position, the vert position at less than 100% weighting will lie at the according distance along that line (it is, essentially, a LERP, hence the name 'Linear Skinning'). With one vert and one influence, this is easy enough to understand:

However, for multiple influences, it becomes more complex, but the above explanation does not change. It may be helpful to examine the provided images while reading the Sample Code provided below.

A note on Algorithms (Linear Skinning vs. Dual Quaternion Skinning)

The information here is applicable to Linear Skinning, by far the most common type. Since 2006, there has been a technique called Dual Quaternion Skinning (you can find the paper here and the website here). It will not be covered in this article other than this reference, but the workflow and concepts behind skinning remain the same, no matter the final algorithm used in the vertex shader.

The Process

This section details the technical aspects of setting skinning in your 3d program.

Creating the Skin

There are a variety of skinning techniques for each program that will not be covered here. It entails adding a Skin to a mesh and adding the bones to the system, and then adjusting the envelopes of the bones and/or the weights of individual points.

Bone System

Add the minimum number of bones necessary to the system. All bones in the Skin must be passed to the Vertex Shader- it is perfectly fine to have bones in the skeleton that are not part of the Skin (they must be transformed but incur no additional cost for skinning). You should aim for a maximum of 64 bones, preferably lower. Any mesh that has more than 64 bones will usually be broken up either by the artist, or dynamically.

Shader Model 2.0 and 3.0 support 256 vertex constant registers, and a bone is a 3x4 matrix, which is equivalent to 4 float3's. Therefore, 256/4 = 64, which is the max number of bones in a system. The lower the better, though.

Influences per Vertex

Hardware skinning supports a maximum of four influences per vertex (less influences will provide some performance savings, but this should not be a factor when you are skinning).

The reason for this is how this info is passed into the shader. The shader receives two inputs for each vertex (in addition to the constants mentioned above): BLENDWEIGHTS and BLENDINDICES. Each is a float 4, which means they are just 4 floats each. The BLENDINDICES contain the bone ID and the BLENDWEIGHTS contain the corresponding weight. To store more than 4 bones would take additional inputs, which would be possible with custom code, but will add additional overhead (4 bones should be sufficient in most cases, anyway).

Tips, tricks, and links

If you are a technical animator, it would behoove you to create a custom skinning tool for your workflow if you have no already. Default skinning tools in Max, Maya, and XSI, are decent, but you should build a custom skin tool to suit your workflow, since you will do A LOT of skinning, most likely. It will allow you to customize your tool to your preferred workflow- for example, I generally weight points with Add/Subtract weights, Grow/Shrink selection, Blend, etc. So I have a tool that allows me to customize every aspect of that (instead of being stuck adding/subtracting Max's default of .05). More importantly, though, the tool will allow (and encourage) you to build custom tools. My skin tools allow me to lock the weight of bones, have a Replace Weight function that will swap bone weights, mass copy-paste of verts, etc., whatever I find myself doing or wanting, I can add.

Hardware Skinning

The actual 'Hardware Skinning' comes right at the rendering step, as the data is being passed to the shader. To see where it fits into the general animation pipeline, check out the Skeletal animation article. Three important keywords here are Bone Matrix Array/Bone matrices, Blend Indices, and Blend Weights.

Bone matrices, as mentioned above, are 3x4 transforms. An array of the transforms is created on the CPU and passed for the bones that are part of the Skin system to the Vertex Shader. Each matrix is 4 float3's, and takes 4 constant registers, as mentioned above.

BLENDINDICES is a float4 the bones affecting vertex (and is a semantic in the vertex input struct). The vertex shader will use these numbers to look up in the Bone Matrices array to find the transform.

BLENDWEIGHTS is a float4 consisting of the weights corresponding to the BLENDINDICES above. After the bone transform is indexed in the Bone Matrix Array, the vertex is transformed by that, with a weight of the blendweight (the vertex LERPs between the old and new position based on the weight). It does the same for the Normal and the Tangent.

Extra Indices and Weights are padded with 0's (and if Dynamic Branching is supported, such as in ShaderModel3 are skipped over). The vertex shader iterates from the first Blend Index to the last, and the result is the transformed vertex. This vertex, at this point, is processed and lit just as it normally would be for a static model.

Sample Program

Below is some sample vertex skinning code, it is not a complete shader and is not functional as is. It will still need other things in the Vertex Shader to work correctly, but it will demonstrate the concepts in this article. The code was taken from OGRE's example shader.

//Bone matrix- You can call this whatever you want, this is your matrix of bones
//In this case, it is an array of 24 3x4 matrices, which is the max for shader model 1.1
//For shader model 2 or 3, it'd be 64
float3x4   worldMatrix3x4Array[24];	//This may have a semantic depending on your engine

//...

//Relevant struct inputs
float4 position : POSITION;
float3 normal   : NORMAL;
float4 blendIdx : BLENDINDICES;
float4 blendWgt : BLENDWEIGHT;
//...and so on

//...


//Vertex shader code
//...
// transform position by indexed matrix
float4 blendPos = float4(0,0,0,0);
int i;
for (i = 0; i < 4; ++i)
{
	blendPos += float4(mul(worldMatrix3x4Array[blendIdx[i]], position).xyz, 1.0) * blendWgt[i];
}

// transform normal
float3 norm = float3(0,0,0);
for (i = 0; i < 4; ++i)
{
	norm += mul((float3x3)worldMatrix3x4Array[blendIdx[i]], normal) * 
	blendWgt[i];
}
norm = normalize(norm);

//...
//You'd continue with the vertex shader as normal from here (lighting operations, etc).