Intro

Face tracking add-ons do not cover custom avatar setups. Understanding how the face tracking templates work on models is required. See template setup guide.

The Fundamentals of the template that need to be fulfilled

1. Face tracking blend shapes are on the model mesh and named correctly
2. Face tracking prefab (Face Tracking Animations)
3. Modifications of the FX controller to block hand expressions and any auto blink logic

Methods of Avatar Setup

1. Mesh Override
2. Pumpkin Avatar Tools
3. Mesh Data Transfer

⚠️Before proceeding make sure you are a creating backup of your avatar⚠️

Mesh Override

This method will work with most models as long the perquisites are fulfilled. This method can be applied to existing avatar setups without the need of redoing the avatar.

Perquisites

• Face tracking mesh has the same amount of polygons as base mesh
• Face tracking FBX has the same amount of bones as base FBX

Process

1. On your existing model click on the “Body” skinned mesh renderer
2. In the Project window navigate to the “[Avatar Name] - Jerry’s Face Tracking/FBX” folder.
3. Replace the “Mesh” with the modified FT mesh by dragging the “Body” mesh from modified FT FBX. Tip - Unity by default has project window icons huge which is not very helpful. Use the slider in the bottom right corner to change the view scale.
4. In the Project window navigate to “[Avatar Name] - Jerry’s Face Tracking/Setup'' and grab the corresponding "VF - VRCFT - [Avatar Name]" prefab and add it to the model. Note - This has face tracking prefab in the prefab and other extra controls for face tracking
5. On the “Avatar Descriptor” > “Playable Layers” replace FX controller and modified parameter list (Modified for taking advantage of VRCFury parameter compressor) with modified version in “AVATAR - Jerry’s Face Tracking/Setup''. Add VRCFury BlendShape Link to link face tracking to other mesh if needed.

Pumpkin Avatar Tools

Pumpkin avatar tools is an Unity plugin tool that can copy one avatar asset to another.

Perquisites

• No additional mesh added to the face
• Unity Knowledge
• Add Pumpkin Avatar Tools to Unity

Process

1. Add the face tracking FBX to the scene
2. Go top of Unity and open Tools > Pumpkin > Avatar Tools
3. Go to the Copy Components section of the Avatar Tools
4. Add target avatar FBX in Avatar and source to Copy From
5. Verify copy component options. VRCFury is off by default, check the VRCFury components to copy if the avatar includes VRCFury components
6. Click Copy Components button

Mesh Data Transfer

Mesh data transfer is a Blender add-on tool that allows the transfer of blendshapes (Shape Keys) from one mesh to another. There is some setup is required to transfer the blendshapes

Perquisites

• No additional mesh added to the face
• Blender Knowledge
• Unity Knowledge
• Add Mesh Data Transfer plugin to Blender
• Add Shape Keys Extra script to Blender
• Add CATS Plugin plugin to Blender

Process

1. Import both models into Blender
2. Line up the meshes. These need to be overlayed. If there are major differences mesh data transfer will not work.
3. On the Source Mesh uncheck the all the blendshapes besides the face tracking blendshapes.
4. On the target mesh go to Mesh Data Transfer. Selection Nearest, Shape Keys, Exclude Muted, and World Space. Click Transfer
5. Mesh will show up now on the target mesh. Verify that they move correctly.

Tip - If mesh is sticking together use CATS feature "Apply to Basis" with blendshape both meshes have that are the same to separate the mouth. Typically this would be AH shape. Remember to revert the shape when completing mesh data transfer.