features - NeocortexVT/VNyanDocFork GitHub Wiki
The immersive features are special features in VNyan that allow you to add objects and behaviors to your model from within VNyan that improves immersion. The menus for these features can be found in the list on the right of VNyan. Currently these include Props, Stickers, Pendulums and Stretch Bones.
VNyan has a system for placing props on your model that you can access from the side menu. There are a few types of props you can add: vnyan props, .png images, .gif animated images, Spout2 sources, Text props, browser sources, and NDI sources.
Users can make custom props for VNyan in Unity using the VNyan SDK. VNyan props can be recognized by their .vnprop
extensions.
VNyan Prop objects can be stored anywhere on your computer, but a dedicated location can be found in VNyan's installation folder, under Item\Props
. This folder also contains a number of default props that are packaged with VNyan.
To load a prop object into VNyan, open the Props menu, and under Prop to Add in the bottom left select Load from file. Clicking Add Prop will open a dialogue box that allows you to navigate to and select the prop you wish to add. This only needs to be done once per profile and will be done automatically when starting up VNyan in subsequent sessions. The Load from file options also allows you to load in .png images and .gif animated images as props (however, see Stickers.
Prop Objects can have different animators such as Dynamic Bones, particle effects, custom shaders and more. See this tutorial for instructions on how to make custom props (as well as other objects for VNyan). VNyan Prop objects can be linked to different parameters to control or change them by adding VNyan SDK's components to them in Unity. For example, you can link animations in the prop to different parameter values using the Anim Param Link component. Setting or changing the specified parameter value in VNyan will then control the prop.
You can load Spout2 sources as props, allowing you to bring in any source that can send Spout2. This can be other instances of VNyan, or a Live2D model from VTube Studio. In order to load Spout2 sources as props, you first need to create a Spout2 prop. Open the Props menu, and under Prop to Add in the bottom left select Spout2. Click Add Prop, and a new Spout2 prop will be created. This only needs to be done once per profile and the prop will be created automatically when starting up VNyan in subsequent sessions. You can then specify the specific Spout2 source in the input field at the bottom of the prop settings window.
You can use the Spout2 Plugin for OBS to send sources from OBS to VNyan. For example, you can add the FugiTech Reactive discord app to VNyan by first bringing it into OBS, and then sending it to VNyan as a Spout2 prop.
Through the prop system, you can add simple texts linked to your model or the environment. In order to add text props, open the Props menu, and under Prop to Add in the bottom left select Text. Click Add Prop, and a new text prop will be created. This only needs to be done once per profile and the prop will be created automatically when starting up VNyan in subsequent sessions. You can then specify the specific text in the input field at the bottom of the prop settings window.
Text prop texts can be linked to a text parameter by entering the parameter name between <>
like so: <example-parameter>. The text prop text will change automatically with the text parameter value. Properties of the text prop cannot be altered; if you wish to use a different font, font size, background, etc., you will need to create your own text VNyan prop in Unity using the VNyan DSK.
You can load a website or local html file as a browser prop. This can be used to bring twitch chat or different overlays into your VNyan environment. Browser interaction is not supported, so you will not be able to click or navigate within the website itself. Websites that require login or authentication do not work.
In order to add browser props, open the Props menu, and under Prop to Add in the bottom left select Browser. Click Add Prop, and a new browser prop will be created. This only needs to be done once per profile and the prop will be created automatically when starting up VNyan in subsequent sessions. You can then specify the specific URL in the input field at the bottom of the prop settings window.
The Props system can be used to display NDI data in VNyan directly. This way, images sent via the NDI protocol can be linked to parts of your model or world. In order to load NDI sources as props, you first need to create an NDI prop. Open the Props menu, and under Prop to Add in the bottom left select NDI. Click Add Prop, and a new NDI prop will be created. This only needs to be done once per profile and the prop will be created automatically when starting up VNyan in subsequent sessions. You can then specify the specific NDI source in the input field at the bottom of the prop settings window.
Once a prop has successfully been added, a button will appear in the list above the Prop to Add option. This will also add an entry for the prop in the Toggle Prop node, which will allow you to toggle the visibility of the prop through the node graph system. The red/green indicator on the button indicates whether the prop is currently hidden or in view respectively. Props are initially visible when added to VNyan the first time. Clicking the button opens a settings menu to the right that allows you to configure the placement of the prop.
Click here for a detailed overview of the configuration options for the prop
Prop Settings:
Name | Description |
---|---|
Prop Name | The name of the prop. This is the name that will appear in the Toggle Prop node. |
Linked Bone | The bone that the prop will be linked to. The prop will copy all transformations of the linked bone. Options are:
|
Linked Game Object | This field is only visible when the Avatar GameObject option is selected in Linked Bone. The name of the GameObject on your avatar the prop will be linked to. Click the magnifying glass button to select the GameObject from a list. |
Position X/Y/Z | Adjust the local position transform of the prop. |
Rotation X/Y/Z | Adjust the local rotation transform of the prop. |
Scale X/Y/Z | Adjust the local scale transform of the prop. |
Always face camera | Force the prop to always face the camera directly, regardless of the rotation of the Linked Bone. Does not override Rotation X/Y/Z adjustments. |
Spout2 Source | This field is only visible when the prop is a Spout2 prop. The name of the Spout2 source for the prop. |
Text |
This field is only visible when the prop is a Text prop. The text that will be displayed as the prop. The Text value can be a text parameter by entering the parameter name between <> . The text prop text will change automatically with the text parameter value. |
Browser URL | This field is only visible when the prop is a Browser prop. The URL of the webpage that will be displayed as a prop. Can also be a local path to an HTML file. |
NDI Source | This field is only visible when the prop is an NDI prop. The name of the NDI source for the prop. |
Additional controls:
Name | Description |
---|---|
Delete | Remove the selected prop from the currently active VNyan profile. Settings will be lost, and can only be recovered from a [[backup |
Clone | Add a copy of the selected prop to the currently active VNyan profile. Copies over all settings. |
Show/Hide | Toggle the visibility of the selected prop. |
Transform Mode: Can be enabled/disabled at the bottom left of the props window. Enabling Transform Mode shows a gizmo that will allow you to adjust the selected prop's position, rotation or scale by clicking and dragging the mouse. Transformations made with the gizmo are translated to local space, and may not reflect the mouse movement directly. Commonly the left-to-right transforms are mirrored.
It is possible to use VMC Trackers as prop trackers in VNyan. First, the VMC tracker you want to use as prop tracker must be mapped in the VMC Tracking Settings, by clicking the VMC Tracker Mapping button, and map a tracker slot to a VMC tracker. Once the tracker slot has been mapped, you can link a prop to the tracker position by selecting the mapped tracker slot as the Linked Bone of the prop. Your prop will now track to the VMC tracker.
VNyan features a Pendulum chain physics system. Such systems are commonly used in 2D Vtuber models to simulate physics or create certain effects, and can be used in VNyan to the same end on 3D Vtuber models. It can be used to create eye wobbles, flowy arm or rig movements, and more! The Pendulum chain window can be opened from the side-menu on the right of the app.
To create a Pendulum chain in VNyan, first set the Amount of Pendulums value in the bottom left of the Pendulums menu. This sets how many bones the new pendulum chain will have (up to 10 bones). Then press Add Chain, and a new entry will be created above it. Note that the chain length cannot be changed once created, so make sure you make it as long as you need. Generally, it is recommended to make your chain a bit longer than the length you will actually use, as more joints will help stabilize the chain's movement. Click the new entry to configure its setting in the window to the right of it.
Import/Export
You can export your chain with the Export Chain button at the bottom of that specific pendulum's window when selected. Chains can be imported with the Import Chain button in the bottom left. Chains are stored as a .vnchain
file. This allows you to create backups and share pendulum chains with the VNyan community. Importing a new chain will create a new entry, and will not replace existing chains.
Pendulum chains are moved or driven by either parameters, blendshapes, or a combination of both. These values will move the first bone of the chain, pulling or twisting the rest of the chain in return. The amount that the bone is moved is equal to the sum of Input value * Multiplier
. In the Preview at the bottom of the window, you can see how your chain responds to input value changes in real-time. The way that the rest of the chain responds to this movement can be adjusted under the Pendulum settings next to the preview window. You can set multiple inputs to drive the chain by pressing the + icon.
Click here for an overview of the input settings for Pendulum chains.
Name | Description |
---|---|
Type | Specify whether the input value is a blendshape or a parameter. |
Value Name | Specify the name of the input blendshape or parameter. |
Multiplier | Scale how much the input value moves the chain. Setting a negative scale number will reverse the driven motion. Multiplier must be set to a value not equal to 0 in order for the pendulum to be affected by the input! |
Effect | Set whether the input will move the top pendulum bone horizontally, or rotate the angle of the top pendulum bone. When using Pos X, the whole chain will shift left-to-right and come to rest in a vertical line. When using angle, the whole chain will rotate and sway, and come to rest at that new rotation. This might be useful if you want parameter changes to maintain some change in movement. |
You can set multiple outputs for your pendulum chain to affect either blendshapes, parameters, the rotation of a game object attached to your model, or a combination of all three. Output values are determined by the left-to-right positions of the Pendulum bones (the blue dots), the multiplier value and the offset value. Each output value is linked to one specific Pendulum bone in the Bone # field. You can specify multiple output effects for the chain by pressing the + icon.
Click here for an overview of the output settings for Pendulum chains.
Name | Description |
---|---|
Bone # | Set which pendulum bone the output is linked to. Bone # 1 corresponds to the first bone or point in the chain. As this effectively copies the input value to the output with a multiplier, this will likely not give useful output. Bone # 2 or higher will provide output that is more characteristic of Pendulum chains. |
Blendshape | Which blendshape should be driven by the specified bone if it is in a positive position (right half of the preview window). |
Negative Blendshape | Which blendshape should be driven by the specified bone if it is in a negative position (left half of the preview window). Specifying both Blendshape and Negative Blendshape can be used create back-and-forth movement between blendshapes. |
Parameter | Output the pendulum bone position value into a named parameter, allowing you to use it in other graphs. |
Offset | Additively shift the named parameter by the specified amount. For example, if you want the value of this parameter to be 5 when the pendulum is in rest, you would set offset to 5. (This setting does not work for blendshape or Avatar GameObject values.) |
Avatar GameObject | Pendulum bone movement will rotate the specified game object attached to your avatar. This could be your model's main skeleton bone, additional bones like ear or tail bones, or any mesh that is part of you model. |
Transform | Specify whether the pendulum bone should rotate the selected GameObject in the X, Y or Z rotation direction. |
Multiplier | Scale pendulum bone output by the specified value. This can be used to effect larger/smaller changes. Multiplier must be set in order for the pendulum to have any effect on the output! |
The sliders to the right of the Preview control the physical motion of your pendulum chain. In other words, these settings change how the bones in the pendulum respond to the movement initiated by the input values.
Click here for an overview of the Pendulum behavior settings.
Name | Description |
---|---|
Damping | Determine the drag on the pendulum motion, reducing oscillations or for how long the pendulum swings. Higher damping results in slower movement with fewer swings, while lower damping results in more chaotic movement with more swings. |
Elasticity | How much force each bone transfers to the next bone. High elasticity will result in the chain moving in a straight line, while low elasticity will result pendulum bones largely unable to move their next bone. |
Stiffness | How stiff the links are between each bone. Higher stiffness results in the chain bones moving all together, while lower stiffness causes pendulum bones to have more swing. |
Inert | How resistant each bone is to movement. Higher inertia means that more force/movement from each bone will be required to swing the next bone, resulting in more rigid left-to-right movement. |
Stretch bones are a feature in VNyan that allow parts of your model to react to other parts of the model, based on the distance between the two. This can be used to create muscle flex effects, enhance jiggle physics, control blendshapes, and more! The Stretch Bone window can be opened from the side-menu on the right of the app.
A Stretch Bone system consists of an Anchor object, a Target Object, and a Stretch Bone Object. The distance between the Anchor Object and Target Object drives a transformation in the Stretch Bone Object. The position of the Anchor Object is based on the root of the object, and so is typically, though not necessarily, a non-tracked bone. The position of the Target Object is based on a value somewhere in the local space of the object. This value can be adjusted, for example halfway along the object, and is therefor more suited to be attached to moving bones, like tracking bones. The Stretch Bone Object is the bone that will be transformed based on the stretch values. This can be either of the other two bones (typically the Anchor Object rather than the Target Object), or a third, separate bone.
To create a Stretch Bone in VNyan, press Add Bone in the bottom left of the window, and a new entry will be created above it. Click the new entry to configure its setting in the window to the right of it.
Clicking on the name of a Stretch Bone system on the left opens a window that will let you configure the settings of that system. In this window, you can specify the Anchor, Target and Stretch Bone Objects, adjust the Target Object offsets, set limitations for the output values of the Stretch Bone system and link a blendshape to the output values.
Click here for a detailed explanation of all input options.
Name | Description |
---|
Stretch bones were added in VNyan 1.4.5.
Community Note: This section requires expansion, including a detailed overview of the Stretch Bone settings values.
Stickers are a quick and easy way to stick images and animated gifs to you model. It offers simple and intuitive drag-and-drop functionality, but has minimal control options compared to the Props system.
Stickers can be added to VNyan from files or directly from the clipboard. To add a sticker from a file, open the stickers window from the side-menu and click the Add from File button. This will open a dialogue window where you can navigate to and select your file. To add a sticker from the clipboard, right-clock and copy, or ctrl+c
the image or gif you want to add as a sticker, storing it in the Windows clipboard. Then simply click the Add from Clipboard button and the image or gif that was copied last will be added to VNyan.
Once a sticker has been added to VNyan, it can be stuck to your model by opening the Stickers window from the side-menu, and dragging the sticker onto the model. The sticker will attach to the surface directly behind it, and follow the movement of your model when successfully attached. Stickers can be moved by dragging any existing stickers around when the Stickers window is open. Stickers can be scaled by scrolling the mouse wheel while the sticker is held by the mouse. Stickers can be deleted by holding the sticker and pressing the delete
key. When using Spout2 for window capture, the Stickers window will not be visible in the capture.
Note: Due to their simplicity, stickers have a number of limitations. If any of these limitations need to be avoided, please use the Props system. Stickers are always rendered on top of the model. A sticker behind the model will appear as if it is on top. The position or visibility of stickers cannot be controlled by nodes.
Model settings are special settings that let you tweak how you control your model and how your model interacts with effects in VNyan. These settings can be found on the top of the screen when the menu button is selected. These settings currently include Expressions, Colliders and Gestures.
VNyan's Expression menu allows you to create Blendshape rulesets or conditions to trigger or control other blendshapes. For instance, if you want a "laughter" expression to activate when both your eyes are closed and your mouth is smiling and open, you could set an expression that does this.
Expressions are evaluated after tracking. This means that the input values for the Expressions are retrieved before expressions are applied, and tracking blendshape values are then overwritten. For example, when using the multiplier output to reduce the size of the A
blendshape if the model is smiling, the original A
value will be based on the raw tracking value, not the A
blendshape value from last frame. This ensures that the blendshape value is modified as intended each frame.
You can set multiple blendshapes to control whether the expression is turned on or off. When multiple Blendshape Conditions are set, all must be met to get an output.
Name | Description |
---|---|
Blendshape Name | The name of the blendshape whose value will be check to determine output. |
Min Value | The minimum blendshape value on the named blendshape required in order to activate the output. |
Max Value | The maximum blendshape value on the named blendshape allowed to activate an output. |
Your expression can either set a static value of different blendshapes as long as the Blendshape Condition is met (for instance, if browInnerUp >70, set Surprised blendshape to 100), or it can scale dynamically active blendshapes by a multiplier (for example, if Smile blendshape > 70, A blendshape should be scaled by 0.5). Output Values have the following settings:
Name | Description |
---|---|
Type | Is either set to Set Value or Multiply. Set Value assigns a static value when the Blendshape Condition is met. Multiply scales the output value by a specified multiplier when the Blendshape Condition is met. |
Blendshape | The name of the target blendshape that should be altered when the Blendshape Condition is met. |
Value | Used for Set Value. Specify the value that the target blendshape should be set to when the Blendshape Condition is met. |
Multiplier | Used for Multiply. Specify the factor that unmodified blendshape values should be multiplied by. |
Min/Max Value | Used for Multiply. Set the minimum and maximum blendshape values that are assigned to the output blendshape while the Blendshape Condition is met. For example, if Min and Max value are set to 30 and 70 respectively, output blendshape values after multiplication lower than 30 will be set to 30, and higher than 70 will be set to 70, so long as the Blendshape Condition is true. If the Blendshape Condition is not true, output blendshape values can exceed this range. |
Smooth Seconds | Used by both Set Value and Multiply. Set how fast the output blendshape value transitions to the target value when the Blendshape Condition first changes from false to true or vice versa. |
VNyan has a few built-in colliders that are mainly used for interactions within VNyan. Many physics effects like the water and slime nodes, or item interactions like for the drop node or food node rely on these colliders. You can adjust the size and placement of these colliders to your avatar in the Colliders etc menu.
These colliders might impact other elements on your model if they interact with other colliders in the world, such as particle emitters. For particle emitters, if they are set to collide with only the default setting and nothing else, then they will not interact with VNyan's built-in colliders.
You will find the following options in the Colliders etc menu:
Window name | Setting Name | Description |
---|---|---|
Avatar Colliders | Show Colliders | Activates a green sphere that indicates the collider boundary of you avatar's head and torso. It is recommended not to adjust collider settings while this setting is off. |
Head Size | Adjust the size of your avatar's head collider. | |
Head Offset | Adjust the height of your avatar's head collider. | |
Torso Size | Adjust the size of your avatar's torso collider. | |
Torso Offset | Adjust the height of your avatar's torso collider. | |
Mouth Collider | Show Collider | Activates a green sphere that indicates the collider boundary of you avatar's mouth. This collider detects when edible objects are treated as being eaten. It is recommended not to adjust collider settings while this setting is off. |
Size | Adjust the size of your avatar's mouth collider. | |
Depth | Adjust your avatar's mouth collider forwards or backwards. | |
Height | Adjust the height of your avatar's mouth collider. | |
Right/Left Food Offsets | Pos X/Y/Z | Adjust the position of food items that are spawned in your right/left hand. |
Right/Left Food Offsets | Rot X/Y/Z | Adjust the rotiation of food items that are spawned in your right/left hand. |
The gestures system offers a way to activate VNyan node graphs similar to hotkeys, but using your hand shape as read by hand tracking instead of button input. This way, VNyan nodes can be activated without being directly within reach of a keyboard or stream deck, affording you more freedom while vtubing. The implementations can be as simple as toggling a facial expression when making a heart with your hands, playing a sound effect when making finger guns, or controlling animations by holding your fingers up, to name a few examples.
The gesture system includes two nodes that offer different control over node graphs. The Gesture Node will send a new trigger signal whenever the specified gesture is made. The Gesture Filter Node will instead filter existing trigger signals based on whether the specified gesture is held at the time the Gesture Filter Node receives a trigger signal. These two nodes can be combined to create two-handed gesture controls. Gestures must first be set up before they can be used in these nodes.
To set up a gesture in VNyan, open the top menu in the app, and click the Gestures button. This will open the Gestures window. To add a new gesture configuration, click the Add Gesture button in the bottom left. This will add an entry above. Click the entry, and a settings menu will open to the right. Here you can rename the gesture, specify the hand of the gesture, record a gesture, configure the sensitivity, and delete and export gestures. Gestures can be imported by clicking the Import Gesture in the bottom left of the Gestures window, and selecting a .vngest
file.
A gesture must be recorded with the Record Gesture button. Simply hold the desired hand shape and click the button. Hand tracking must be enabled for this. The sensitivity of the hand gesture can be adjusted with the sliders below. The sliders are used to adjust the range of angles that fingers can be at to trigger the gesture. Higher values mean a wider range and the gesture will trigger more easily. After these values are set up, if your fingers are within the specified range of angles compared to the recorded gesture, VNyan will automatically register the gesture for any node graphs using gesture nodes.
Click here for a detailed description of the Gesture parameters.
Name | Description |
---|---|
Gesture Name | The name that the gesture is registered as. This name will show up in the dropdown menus in gesture-related nodes. |
Avatar Hand | The hand that the gesture will be read from. The gestures system will read the avatar's hand, not the user's hand, so mirroring hand tracking will swap what hand is monitored. This also means that if you are blending tracking methods, the gesture is based on the final result! |
Finger Angle Threshold | The range of angles that the held finger position can be off from the recorded finger position where the gesture is still registered as the gesture in question. |
Held Milliseconds | The number of milliseconds the gesture must be held at in order for the gesture to be registered as present. |