Advanced Placement Tutorial - Unity-Technologies/com.unity.cv.synthetichumans GitHub Wiki
In this tutorial, we will continue working on the Scene we built in the Introduction to Synthetic Humans Tutorial, in which we covered the basics of generating humans and randomizing their animations and positions, as well as generating a computer vision dataset.
Please make sure you follow the Introduction to Synthetic Humans Tutorial up to at least Step 5 before starting this tutorial.
The advanced placement system in Synthetic Humans comprises a few parts. To place humans on navigable surfaces in the Scene, the Synthetic Humans package offers the NavMesh Placement Randomizer . This Randomizer can use several placement methods (called Placers) to situate the humans in the Scene. The choice of which Placer to use is controlled by the user and is mapped to the animation of the generated human at the time of placement. These mappings are defined in Animation Placement Groups. Thus, animation randomization needs to happen before placement. For example, if a generated human has a standing animation, the code responsible for placing this human needs to use a Placer that targets ground areas.
Throughout the tutorial, lines starting with :green_circle: denote the individual actions you will need to perform in order to progress through the tutorial. The rest of the text will provide additional context and explanation around the actions. If in a hurry, you can just follow the actions!
Step 1: Randomizer Setup and Animation Placement Groups
At the end of the Introduction to Synthetic Humans Tutorial, we have a Scene with a Scenario object inside. At the start of the Scenario 50 humans are generated. In each iteration a random subset is activated and randomly positioned with random animations, and every 50 Iterations a new set of humans is generated.
Summary of pre-requisites, done in Introduction to Synthetic Humans Tutorial:
- Create an HDRP project and install the Synthetic Humans and Perception packages
- Import all samples from the Synthetic Humans package
- Open Tutorial Scene from the samples and add a Scenario.
- Add the Human Generation Randomizer and assign one or more Human Generation Configs to it.
- Add the Synthetic Human Animation Randomizer
:green_circle: If you came here after the introductory tutorial, your Scenario has an active Point List Placement Randomizer. Deactivate it.
:green_circle: Add the NavMesh Placement Randomizer to your Scenario and add the Scene's camera to the Cameras list in the Randomizer.
This list of cameras is used to decide whether the placed humans are visible in one of the specified cameras. If not, the placement is retried.
Previously, we looked at the base Prefab assigned to our Human Generation Configs. We will now revisit this Prefab and study it a bit more. The asset is called HumanBase_AllPlacementMethods.
:green_circle: Open the Prefab asset named HumanBase_AllPlacementMethods. You can either search for it in the Project tab or find it in the Base Prefab field of one of your Human Generation Configs that are added to the Human Generation Randomizer.
A few things are of note here. As discussed before, the Synthetic Human Animation Randomizer Tag holds a list of animation tags to use on the generated humans. Animation tags are assets that point to animation clips and contain some additional metadata. The Randomizer that actually assigns an animation to the humans is named Synthetic Human Animation Randomizer and you have already added it to your Scenario.
Note that the base Prefab also has a Nav Mesh Placement Randomizer Tag. This component speaks with the NavMesh Placement Randomizer which we just added a couple of paragraphs above. This component has a field named Animation Placement Groups. These specify your desired mappings between animations and the available placement algorithms. We have added two sample groups here:
The first one maps a number of animations to something called the Bound-Based Ground Placement algorithm, and the second one maps animations to the Sitting Placement algorithm.
:green_circle: Click on SampleGroundPlacementGroup to open it. It looks like this:
At the top a Placer is selected and at the bottom we have a list of animation tags. What this means is the Placer selected at the top will be used if any of the animations added to the list are found on a human that is targeted by the NavMesh Placement Randomizer.
If at runtime, the system finds that more than one Placer has been mapped to the same animation, it will choose one randomly.
:information_source: To learn more about Placers, we recommend checking out the related documentation page.
Step 2: Ground Placement
The Tutorial Scene contains an object named PlacementRooms which is currently deactivated. We will use this to try out advanced placement.
:green_circle: Enable the PlacementRooms object in the Scene Hierarchy tab.
The room will appear around the camera. We will now zoom out a bit to have a full view of the room.
:green_circle: Select the object named SecondCameraPosition in the Scene. We want to copy the transform properties of this object to the camera.
:green_circle: In the Inspector tab, click on the context menu (three dots icon) on the Transform component and click Copy -> Component.
:green_circle: Select Main Camera in the Scene. Open the context menu for the Transform component and choose Paste -> Component Values. You should now see the complete room.
:green_circle: Add a NavMeshSurface component to the PlacementRooms object.
:green_circle: Make sure the Collect Objects field is set to All. This makes the NavMeshSurface encompass all meshes in the scene and not just its own child objects. So you will not need to add more NavMeshSurface components to your Scene even if you add more rooms.
:green_circle: Unfold the Advanced section and set Default Area to Not Walkable. This will cause the NavMesh to only be generated on surfaces that have NavMesh Modifier components, giving us more control.
The Inspector UI for PlacementRooms now looks like this:
:green_circle: Within PlacementRooms, find the Floor object and select it.
This is the object on which humans will be placed using the Bound-based Ground Placer as we discussed above. You need to add two components to this object.
:green_circle: Add a NavMesh Modifier component to the Floor object.
:green_circle: Enable Override Area and set the Area Type to Walkable.
:green_circle: Add a Ground Placer Tag component to the Floor object and make sure Area is set to NavMesh Modifier.
:green_circle: You can now bake a test NavMesh to confirm that the correct settings have been applied.
Select PlacementRooms and click Bake. You should see the NavMesh on the floor, in the Scene view:
The Scene is now ready to be used with the Bound-based Ground Placer. Other Placers will require more components to be added to Scene objects. We will add the Sitting Placer soon.
:green_circle: Click ▷ to see the humans placed on the ground inside the room.
Step 3: Sitting Placement
We will now use the Sitting Placer. This Placer will handle the task of placing humans on areas that have been defined as seats. The human base Prefab we are using already has an Animation Placement Group attached that instructs the placement system to use the Sitting Placer for one of the four animations that our humans will have. To see this, find and open the asset named SampleSittingPlacementGroup.
The first step is to add a new navigable Area for sitting.
:green_circle: From the Unity editor menus select Window -> AI -> Navigation and select the Areas tab.
:green_circle: Type "Sitting" in the the first empty field. This defines a new area.
We now need to add the necessary components to the Scene objects. Select all the surfaces that could act as seats.
:green_circle: Select the surfaces that can act as seats in the Scene. These are the sofas and the stools. The specific objects are (Scene hierarchy):
- PlacementRooms/Sofas/Sofa/Cube
- PlacementRooms/Sofas/Sofa (1)/Cube
- PlacementRooms/Stools/Stool1/Cylinder (1)
- PlacementRooms/Stools/Stool2/Cylinder (1)
- PlacementRooms/Stools/Stool3/Cylinder (1)
:green_circle: Add a NavMesh Modifier component to the selected objects. Enable Override Area and set the Area Type to Sitting.
:green_circle: Add a Sitting Placer Tag component to selected objects. This instructs the Sitting Placer to use these surfaces.
Finally, we need to specify which sides of the selected objects are viable for humans to sit on.
:green_circle: Select PlacementRooms/Sofas/Sofa/Cube and in the UI for Sitting Placer, set the Min and Max for Enabled Edges in Polar Coordinate to 190 and 350 respectively.
You'll see a visualization of these settings in the Scene view:
:green_circle: Repeat the above for the other sofa (PlacementRooms/Sofas/Sofa (1)/Cube).
To separate the NavMeshes on the sofas, the ground, and the stools we need to set a proper Step Height and Max Slope in the NavMesh Agent settings. This is needed because we do not want the humans to be placed in-between these surfaces as though they are traversing between them. We also need to make sure NavMeshes encompass the surfaces of the stools which are quite small. This means the agents (humans) need to have a small enough radius.
:green_circle: From the Unity editor menus select Window -> AI -> Navigation and select the Agent tab.
:green_circle: Set both Step Height and Max Slope to 0, and set Radius to 0.1 as shown below.
The settings on the NavMesh Surface also need to be modified to include small areas.
:green_circle: Find the NavMeshSurface component attached to the PlacementRooms object. In the Advanced section of the component, set Minimum Region Area to 0.1.
:green_circle: Click ▷ to see the humans placed on the ground as well as on the sofa inside the room.