How Luna's 3D portrait moves its body, how she decides when to move it, and how to write your own gestures in a few lines of JavaScript.

Everything here is about VRM (3D) models. Live2D models have their own motions and don't use any of this.

• What a gesture is
• From her reply to her body
• How she picks one
• What happens every frame
• The skeleton and its numbers
• Anatomy of a gesture file
• Writing one, step by step
• Common mistakes
• Code or a .vrma clip?

A gesture is a short list of key poses in time. A pose says where some of her bones point: "right upper arm raised, forearm bent up, hand turned so the palm faces you". The portrait eases smoothly from one key to the next and blends the whole thing over whatever she's already doing (breathing, looking around, talking), so a gesture never snaps on or off.

Each gesture is one file in portrait/gestures/:

portrait/
├── viewer/gestures.js   the player: loads the files, blends them, plays them
└── gestures/
    ├── _pose.js       shared helpers (rest pose, mirroring, fingers)
    ├── wave.js
    ├── hug.js
    ├── wink.js
    └── ...            one file per gesture

The file name is the gesture's name: wink.js is wink. Files starting with _ are helpers or notes and are never loaded as gestures.

 you: "I missed you"
        │
        ▼
 Luna's prompt offers her the tags     (gestures.py reads every gesture
 [wave] [hug] [wink] ...                file with an "about" line)
        │
        ▼
 she replies "I missed you too [hug]"
        │
        ├── the screen, the voice and her history get "I missed you too"
        │   (the tag is cut out before anything is shown or spoken)
        │
        └── the live monitor sends {"t": "gesture", "name": "hug"}
                │
                ▼
         the portrait page finds hug.js (already loaded) and plays it

1. The portrait asks the server for gestures/index.json: a list of every .js file in portrait/gestures/ except the _ ones.
2. It loads each file and reads its settings. left: true makes a mirrored copy called <name>_left, idle: true adds it to her idle moves, private: true keeps it away from stream chat.
3. The viewer (./viewer.sh) makes a button for every gesture it found. A file with a mistake in it gets a pink line in the panel saying which file and what's wrong, instead of a missing button.

Files are read fresh on every page load, so after editing a gesture, F5 is enough.

/portrait gestures <mode>, or the gestures row in the tools pane, switches between tags, meaning, keywords and off. See Portrait for the details of each.

There are three ways, and you choose which with /portrait gestures <mode> or the gestures row in the tools pane. Every gesture file and clip takes part in all three; the same two lines in a file feed them:

about: "sultry come-hither, teasing, inviting",          // tags and meaning
keywords: ["beckon", "come hither", "crooks a finger"],  // keyword fallback

Her system prompt gets one line listing every gesture that has an about:

Body: you have an animated 3D body on screen. When a gesture fits what you say, put one tag where it happens: [wave] (hello, goodbye), [hug] (affection, comfort), [sultry_beckon] (sultry come-hither, teasing, inviting), ... At most one per reply, most replies need none.

The language model reads the conversation and, when it fits, writes the tag into its reply like any other word. Nothing else chooses: it's the same reasoning that picks her words, so it understands the moment, not just the vocabulary. That's why it suits role-play: "come a little closer..." in a teasing scene gets [sultry_beckon] with no keyword in sight.

As the reply streams in, the tag is cut out before anything is shown, spoken or saved, and the portrait is told to play it. Private gestures are left out of the line in chat rooms, so she can't pick them there.

The about text is the whole interface: write it as "when to use this", short and concrete. A vague about gets picked at odd times; no about means she never picks it herself.

Cost: about 60 tokens of prompt (cached by llama.cpp, so read once) plus a few output tokens when she uses one.

After she answers, the embedding model (the one that recalls her memories) turns two things into vectors, lists of numbers that capture meaning:

• the exchange: "They said: ... She answered: ...";
• a short description of each gesture. The built-ins have hand-written ones; every other gesture or clip with an about gets "A moment for the gesture sultry beckon: sultry come-hither, teasing, inviting."

It compares the exchange with every description (cosine similarity, 0 to

1. and also with a few "ordinary reply" descriptions (facts, a finished task, an explanation). A gesture plays only if it beats the best "ordinary" score by portrait.meaning_margin (0.04 by default), so most replies get nothing. /portrait meaning <text> shows the scores for any line, and the log records them for every reply.

It runs after the reply, off the turn's path (10 to 30 ms), needs the embedding server, and is fuzzier than tags. Private gestures only take part in your own conversations.

Underneath both modes (and alone in keywords mode), the portrait watches what she writes. A word from a gesture's keywords inside an action (*beckons*, *puts a finger to her lips*) plays it; an emoji in the list (🤫) plays it anywhere. A keyword matches the start of a word, so hug catches hugs and hugging. A tag she wrote herself always wins over a keyword, and the earliest match in the reply wins over later ones.

Keywords only look inside *actions*, never at ordinary sentences, so "I hugged a tree today" doesn't make her hug.

For a .vrma clip, put about and keywords in clips.json.

About 60 times a second the portrait builds her pose in layers:

1. Her own body language. portrait.js sets her head and neck (where she's looking), chest and shoulders (breathing), spine (sway).
2. Rest pose. Arms down, elbows soft, fingers a little curled, legs straight. Any other bone a gesture has touched goes back to straight.
3. Gestures on top. For each playing gesture, the player:
   • works out a weight from 0 to 1: rising over the fade-in, 1 while it plays, falling over the fade-out;
   • finds the two keys around the current time and eases between them (smoothstep, so motion starts and stops gently);
   • turns each bone towards that pose by the weight (a quaternion slerp), or, for + bones, adds the rotation on top of what's there.
4. Expressions. The gesture's expr values are applied: smiles, winks, blush, tears.
5. Physics. three-vrm runs the spring bones (hair, ears, tail), so they swing from whatever the body just did.

Two consequences worth knowing:

• A gesture never has to return to rest itself. The fade-out blends it back into her normal pose.
• Starting a new gesture fades the old one out while the new one fades in, so you can interrupt freely.

Gestures run on real time, not frame count, so they last the same on a slow machine and a fast one.

Every VRM model has the same named bones (the VRM humanoid standard): hips, spine, chest, upperChest, neck, head, leftShoulder, leftUpperArm, leftLowerArm, leftHand, the finger bones, and the legs, each with a left and a right. That's why one gesture works on every VRM model.

Rotations are [x, y, z] in radians (π ≈ 3.14 is half a turn, 1.57 a quarter turn, 0.5 is about 30°). They're measured from the T-pose: all zeros means arms straight out to the sides, palms down.

Use the right side's numbers with y and z flipped:

rightUpperArm: [0, 0.8, 1.15]    // right hand to her chest
leftUpperArm:  [0, -0.8, -1.15]  // the same thing, left hand

You rarely need to type the left side: both({...}) mirrors a right-side pose onto both arms, and left: true makes a whole left-handed copy.

rightLowerArm: [0, 2.0, -0.55, -0.4]: the 4th number turns the forearm about its own length, like turning a doorknob, before the bend. It's how a palm comes to face the right way without the wrist bending unnaturally. Combine it with the hand's x for the rest of the turn.

• Plain (rightUpperArm): "point this bone here". Used for arms, hands, fingers and legs.
• With + ('+head', '+spine', '+chest'): "add this to what she's already doing". Used for bones the portrait animates itself (her head follows her gaze, her chest breathes), so the gesture adds a nod or a lean without stopping the rest.

hips: [x, y, z] is a position in metres, not a rotation: hips: [0, 0.15, 0] lifts her 15 cm (the jump uses it), [-0.025, 0, 0] shifts her weight onto one leg.

Here's wink.js, a whole gesture:

// wink: a playful wink with a little head tilt and a smile
export default {
  about: "playful, teasing, a little flirty",
  dur: 1.6, fade: [0.25, 0.4],
  keys: [
    [0.0, { '+head': [0.02, 0, 0] }],
    [0.35, { '+head': [0.04, -0.06, -0.16] }],
    [1.1, { '+head': [0.04, -0.06, -0.17] }],
    [1.6, { '+head': [0.01, 0, -0.04] }],
  ],
  expr: { blinkRight: [[0, 0], [0.3, 1], [0.95, 1], [1.2, 0]],
          relaxed: [[0, 0], [0.3, 0.55], [1.6, 0.4]], blush: 0.25 },
};

You don't have to repeat every bone in every key. For each bone, the player looks for the nearest keys that mention that bone on either side of the current time and eases between them. A bone missing from the later keys simply holds its last value.

So: put the full pose on the first and the last key, and in between list only what changes. sultryBeckon.js does exactly that: the arms are in the first and last keys, and the keys in between only curl the fingers.

A number holds an expression for the whole gesture; a list of [time, value] pairs animates it:

expr: { happy: 0.6 }                                   // smile throughout
expr: { blush: [[0, 0], [0.8, 1], [3, 1], [3.6, 0]] }  // blush rises, holds, fades

A wink's eye opens again by itself when the gesture ends.

import { both, fingers, open } from './_pose.js';

For one finger different from the rest (pointing, shh), set its three bones yourself after the helper:

...fingers('right', 0.9),                      // curl them all
rightIndexProximal: [0, 0, 0], rightIndexIntermediate: [0, 0, 0], rightIndexDistal: [0, 0, 0],

Say you want her to put a hand on her heart.

1. Start with one bone. Make portrait/gestures/hand_on_heart.js:

export default {
  dur: 3, keys: [[0, { rightUpperArm: [0, 0.8, 1.15] }], [3, { rightUpperArm: [0, 0.8, 1.15] }]],
};

Run ./viewer.sh, click hand on heart, drag round to look from the side. The upper arm should come down and forward.

2. Add the forearm. Bend it towards her chest:

rightLowerArm: [0, 2.15, -0.15],

F5, click, look. Nudge one number at a time by about 0.1 to 0.3 until the hand sits where you want.

3. Fix the hand. If the palm faces the wrong way, change rightHand's x, or add a 4th number to rightLowerArm. Check from the side.

4. Give it life. A second key a little different from the first, so she isn't frozen, a head tilt, a face:

import { fingers } from './_pose.js';

const HAND = { rightUpperArm: [0, 0.8, 1.15], rightLowerArm: [0, 2.15, -0.15], rightHand: [0, 0.2, 0] };

export default {
  about: "touched, grateful",
  dur: 3.0, fade: [0.5, 0.6],
  keys: [
    [0.0, { ...HAND, ...fingers('right', 0.2), '+head': [0.06, 0, 0.08] }],
    [1.5, { '+head': [0.09, 0, 0.12] }],
    [3.0, { ...HAND, ...fingers('right', 0.25), '+head': [0.06, 0, 0.1] }],
  ],
  expr: { relaxed: 0.4, blush: [[0, 0], [1, 0.5]] },
};

5. Let her use it. The about line is all it takes. Within about 20 seconds she has [hand_on_heart] in her prompt, and /portrait hand_on_heart works.

Both play through the same blend, both can carry expr (blush, tears, smiles), and a clip named like a gesture (wave.vrma) replaces it. See Portrait for clips.