Allocator Tagging Guide - o3de/o3de GitHub Wiki

O3DE Allocators are setup to allow using a hierarchy structure for tagging memory from specific base allocators.

The main base allocators of O3DE are

1. The OS Allocator which delegates to the operating System malloc call.
2. The System Allocator which uses the memory High Performance Allocator Schema under the hood for allocating memory.
3. The Pool Allocator which uses a pool of memory for allocating specific block sizes of memory in O(1)
4. The Thread Pool Allocator which is a thread safe implementation of the pool allocator, where separate pools are allocated on different threads.

This guide provides examples on how to inherit an O3DE Allocator and order to track memory for a specific system(such as Animation or Prefabs)

The guide mentioned here requires the following commits to be integrated into the users branch.

To piggyback off of an existing allocator, The base allocator class can be wrapped in the AZ::ChildAllocatorSchema template.
An inherited class should then be added that implements the AZ_RTTI macro which allows providing a user readable name for the new allocator itself. The AZ_CHILD_ALLOCATOR macro that encapsulates these steps

AZ_CHILD_ALLOCATOR(PrefabSystemAllocator, "{CD8443AE-BC56-4C46-AF3A-6633C2C0E694}", AZ::SystemAllocator);

The above macro creates the PrefabSystemAllocator, which inherits from the ChildAllocatorSchema class template and standard container allocator alias of PrefabSystemAllocator_for_std_t.
The PrefabSystemAllocator can be used to specify the allocator for class using the AZ_CLASS_ALLOCATOR macro.
The PrefabSystemAllocator_for_std_t is for use with the AZStd container types such as an AZStd::vector, AZStd::basic_string, AZStd::unordered_map, etc... for use as a standard container allocator.

Side Note: A random UUID should be generated for the new allocator. This can be done on Windows using the Create GUID tool which is available in the Visual Studio Tools -> Create Guid menu.

Alternatively a random UUID can be created using the Python UUID module using the following one liner

python -c "import uuid; print(f'{{{str(uuid.uuid4()).upper()}}}');"

PrefabSystemComponent.h

class PrefabLoader final
    : public PrefabLoaderInterface
{
    public:
        AZ_TYPE_INFO_WITH_NAME_DECL(PrefabLoader);
        AZ_RTTI_NO_TYPE_INFO_DECL();
        AZ_CLASS_ALLOCATOR_DECL;
};

class PrefabSystemComponent
    : public AZ::Component
{
public:
    AZ_COMPONENT_DECL(PrefabSystemComponent);
private:
    using ContainerUsingCustomAllocator = AZStd::vector<int, PrefabSystemAllocator_for_std_t>;
    ContainerUsingCustomAllocator m_customContainer;
};

PrefabSystemComponent.cpp

AZ_TYPE_INFO_WITH_NAME_IMPL(PrefabLoader, "PrefabLoader", "{A302B072-4DC4-4B7E-9188-226F56A3429C}");
AZ_RTTI_NO_TYPE_INFO_IMPL(PrefabLoader, PrefabLoaderInterface);
AZ_CLASS_ALLOCATOR_IMPL(PrefabLoader, PrefabSystemAllocator)

AZ_COMPONENT_IMPL_WITH_ALLOCATOR(PrefabSystemComponent, "{27203AE6-A398-4614-881B-4EEB5E9B34E9}", PrefabSystemAllocator);

The sys_DumpAllocator Console Command can be used via the AZ Console system to dump the memory allocations of any active allocations.

Sample output for the user of the Console Variable is below. It is setup to be able to placed into a CSV file to aid in data analysis.

[CONSOLE] Executing console command 'sys_DumpAllocators'
Index,Name,Used KiB,Reserved KiB,Consumed KiB,Parent Allocator
0,OSAllocator,1383.67,2097152.00,2097152.00,
1,SystemAllocator,346025.41,2097152.00,2097152.00,
2,RapidJSONAllocator,1345.22,2097152.00,2097152.00,OSAllocator
3,EntityAllocator,2.93,2097152.00,2097152.00,SystemAllocator
4,ComponentAllocator,123.48,2097152.00,2097152.00,SystemAllocator
5,PhysXAllocator,321.34,2097152.00,2097152.00,SystemAllocator
6,ThreadPoolAllocator,1039.00,2097152.00,2097152.00,
7,LuaSystemAllocator,3714.58,2097152.00,2097152.00,SystemAllocator
8,AudioSystemAllocator,148.19,2097152.00,2097152.00,SystemAllocator
9,LuaSystemAllocator,264.08,2097152.00,2097152.00,SystemAllocator
10,RHISystemAllocator,0.66,2097152.00,2097152.00,SystemAllocator
11,AzClothAllocator,0.14,2097152.00,2097152.00,SystemAllocator
12,AWSNativeSDKAllocator,99.48,2097152.00,2097152.00,SystemAllocator
13,EMotionFXAllocator,487.15,2097152.00,2097152.00,SystemAllocator
14,EMotionFX::AttributeAllocator,0.34,2097152.00,2097152.00,
15,EMotionFXManagerAllocator,0.27,2097152.00,2097152.00,SystemAllocator
16,ImporterAllocator,0.80,2097152.00,2097152.00,SystemAllocator
17,ActorManagerAllocator,0.20,2097152.00,2097152.00,SystemAllocator
18,ActorUpdateAllocator,0.12,2097152.00,2097152.00,SystemAllocator
19,MotionManagerAllocator,0.16,2097152.00,2097152.00,SystemAllocator
20,MotionAllocator,0.82,2097152.00,2097152.00,SystemAllocator
21,MotionEventManagerAllocator,0.16,2097152.00,2097152.00,SystemAllocator
22,SoftSkinManagerAllocator,0.02,2097152.00,2097152.00,SystemAllocator
23,AnimGraphManagerAllocator,0.16,2097152.00,2097152.00,SystemAllocator
24,BlendSpaceManagerAllocator,0.05,2097152.00,2097152.00,SystemAllocator
25,AnimGraphAllocator,0.14,2097152.00,2097152.00,SystemAllocator
26,RecorderAllocator,0.58,2097152.00,2097152.00,SystemAllocator
27,PoseAllocator,0.12,2097152.00,2097152.00,SystemAllocator
28,ThreadDataAllocator,6.07,2097152.00,2097152.00,SystemAllocator
29,EditorAllocator,2.88,2097152.00,2097152.00,SystemAllocator
30,PropertyWidgetAllocator,0.09,2097152.00,2097152.00,SystemAllocator
31,CommandAllocator,6.72,2097152.00,2097152.00,SystemAllocator
32,PoolAllocator,0.00,2097152.00,2097152.00,
-,Totals,354975.03,69206016.00,69206016.00,
33 allocators active

NOTE: There is currently a max limit of 100 registered allocators.

In O3DE Allocator have support for recording the address, size and callstack of each allocation that occurs when using a specific allocator.
This can be done by turning on allocation record tracking as well as profiling for the allocator itself.
By default this is only enabled by default in the C++ Unit Test framework when using the LeakDectionFixture for Googletest fixtures.

In order to enable allocator recording in other runtime applications such as the GameLauncher and Editor, support for a startup configuration file has been added to O3DE that can read a Windows-style INI file that specifies the name of allocator that is registered with the Allocation system and a value for level of allocation recording that should occur.

Support for loading of the startup configuration file can be overridden by the O3DE_STARTUP_CFG_FILE_CHECK_OVERRIDES CMake Cache variable when configuring.
By default the startup config file would only be loaded in debug and profile configurations and not in release configurations, but can overridden by specifying the O3DE_STARTUP_CFG_FILE_CHECK_OVERRIDES CMake cache option above.

The path to the startup config file is searched in the following locations:

1. ~/.o3de/Registry/startup.cfg (~ stands for the user home directory. i.e /home/<user> on Linux, C:/Users/<user> on Windows, /Users/<user> on MacOS.
2. <executable-directory>/Registry/startup.cfg.
3. The value of the O3DE_STARTUP_CFG_FILE environment variable if set.
4. The value of the --startup-cfg-file command line argument.

The specifics of enabling allocation tracking for an allocator involves adding a setting to the startup configuration file of the form:
allocator_tracking_<allocator-name> = <AllocationRecordMode>

The "<AllocationRecordMode>" value must be match one of the options of the AllocationRecordMode enum.

When the value of the allocator_tracking_<allocator-name> key is set to a value above the RECORD_NO_RECORDS enum, allocation record tracking becomes enabled.

The following is a sample startup.cfg file that illustrates how to turn on Allocation tracking.

The names of allocators can be found at runtime by using the sys_DumpAllocators command in the previous section.

allocator_tracking_PoolAllocator = RECORD_NO_RECORDS
allocator_tracking_PoolAllocator = 0 ; // Same as the RECORD_NO_RECORDS enum
allocator_tracking_ThreadPoolAllocator = RECORD_STACK_NEVER
allocator_tracking_ThreadPoolAllocator = 1 ; // Same as the RECORD_STACK_NEVER enum
allocator_tracking_OSAllocator = RECORD_STACK_IF_NO_FILE_LINE
allocator_tracking_OSAllocator = 2 ; // Same as the RECORD_STACK_IF_NO_FILE_LINE enum
allocator_tracking_SystemAllocator = RECORD_FULL
allocator_tracking_SystemAllocator = 3 ; // Same as the RECORD_FULL enum

There are also several Console Commands, that allows users to

The console output could looks as follows

sys_DumpAllocationRecordsToDevWriteStorage OSAllocator
[CONSOLE] Executing console command 'sys_DumpAllocationRecordsToDevWriteStorage OSAllocator'
(mem) - Printing allocation records for allocator OSAllocator. Estimated time to print all records is 1 seconds
(mem) - Printed 142 allocations in 1 seconds for allocator "OSAllocator" (71 records per seconds)

Running the console command would result in a log file being output to the <project-root>/user/allocation_records directory.

Here is a sample of a dumped record file: records.2023-11-03T205715Z.51804.log