Asset Packs

Synthetic Humans requires a lot of separate assets to generate people. In addition, each of these individual assets must be paired with metadata that lets the system know how it can be used. For example, each article of clothing needs metadata explaining what sexes / ages it's appropriate for, what materials it's compatible with, etc. This metadata is stored in .taginfo files. Assets and .taginfos have matching names and are stored in the same directory (much like unity .meta files). Finally, all of these assets need to be catalogued somewhere so that the system can find them. Together, all of these components make up a Synthetic Humans Asset Pack.

Synthetic Humans ships with a default asset pack, which requires no modification to work. However, the purpose of this document is to describe some of the technical details of asset packs so that advanced users can modify existing asset packs, or even create entirely new ones.

Customizing Asset Packs for Human Generation

As a user of Synthetic Humans, you'll most commonly interact with Asset Packs not by modifying them directly, but by using them in human generation. If you take a look at a HumanGenerationConfig, you'll notice that it does not directly reference Asset Packs. Rather, it references something called a SyntheticHumanAssetPool.

SyntheticHumanAssetPools provide a layer between Asset Packs and human generation. Within a SyntheticHumanAssetPool, you can configure it to pull assets from multiple packs, or even ignore specific assets that you don't want.

To ignore specific assets, go to the asset pool's "Filters" section and add a new IgnoreAssetFilter. Then, drag the .taginfo of the asset you want to ignore into this filter. Note that for clothing items you just need to ignore the tag of the base clothing item itself, not any of its materials or VATs. Though you can ignore those too if you so choose.

Default Asset Pack Structure

The default asset pack can be found in the Synthetic Humans package's Resources/_Assets folder. At the root level, you'll find the actual AssetPackManifest which is used to reference the asset pack in config files. Additionally, you'll find that the actual assets are split into two folders: system and user. The system folder contains mostly the base meshes and VATs for all assets in the pack. See the "VAT Format" section for more info on VATs. The user folder contains mostly materials, as well as some other hand-authored assets.

The default asset pack is split in half like this because of the way our asset export pipeline works. We generate our base meshes and VATs in SideFX Houdini and auto-export them straight to the system folder, hence why it's separate from the user folder. Asset packs do not actually require any specific folder structure - they just exist to make it easier for humans to browse.

VAT Format

We use VATs (vertex animation textures ) to deform meshes to fit differently sized humans. All meshes that need VATs will have a UV2 that maps each vertex to exactly one pixel. At runtime, this is used with the corresponding VAT to read offset data from pixel values and apply that to the mesh. Eg. if a person is generated with their height set to 0.8, we read from the body mesh's "height VAT" and scale each vertex offset by 0.8 (not exactly how it's implemented, but that's a good approximation.)

.taginfo Format

.taginfo files are just JSON files in a simple key-value format. During asset import into Unity, a special importer converts these files to the proper tag class based on the value of the tagtype field (the only required field for .taginfo). As such, the actual data included in a given .taginfo file depends on what type of tag it is.

To see all possible tag types and the fields they use, look for any classes that derive from the SyntheticHumanTag class in the Synthetic Humans runtime assembly.

Creating or Modifying Asset Packs

For those who would like to add new assets to an asset pack, there are two main things standing in the way - .taginfo generation and VAT generation.

.taginfo Generation

The first are the .taginfo files. Every asset needs one, which can be a pain if you're hand-authoring a lot of content. Internally, we have automated pipelines for generating these, but you can also make use of FolderTags in order to accelerate tagging of hand-authored content. FolderTags automatically generate identical .taginfo files for all assets in a folder, which is often all that you need (eg. if you have a folder of all shirts or all male body VATs).

To make a FolderTag, first create a folder and group all of the assets you want to tag. Then, in that folder, right click and go to Create -> Synthetic Humans -> Tags. Select the type of tag that you need (eg. Clothing for clothing meshes, Material for materials, etc.) Finally, name this tag FolderTag (case and spelling sensitive) and set any settings that you want copied over to your individual asset tags. When you go through the Asset Pack generation process described below, individual .taginfo files bearing these exact settings will be created for each asset in the folder.

If your assets are not being generated the way you expect them to, there is a chance that you are missing some settings in your .taginfo files. The HumanGenerator class has all the logic used for actually selecting assets based on their tag data, so it is recommended that you read through that to understand what settings are needed in your particular case.

VAT Generation

As described above, VATs are needed for any mesh that is expected to deform based on body size. This includes both clothing and the human body meshes themselves. This is by far the most complex part of the entire process. We have an entire internal pipeline built out in SideFX Houdini to generate these and appropriately resize assets, and that is complex enough that it requires its own documentation.

If you have interest in doing this for yourself, please reach out to us directly.

The Final Product

Once you've made all the changes to your Asset Pack, there is one final step. We must update the AssetPackManifest so that it contains all of the new assets.

To do this, an AssetPackManifest must first exist in the root directory of your asset pack. You can create one by right clicking on your root directory and selecting Create -> Synthetic Humans -> Tags -> AssetPackManifest. If you are modifying an existing Asset Pack, you can skip this step.

To update the AssetPackManifest with the latest content changes, open the Window -> Synthetic Humans -> Synthetic Humans Editor Tools window. At the top of this window, you should see a section for selecting an AssetPackManifest. Drag your manifest into this slot, and click the Update AssetPackManifest From Root Folder button.