Data

The data associated with a Training Set or a trained Model is stored in JSON text format. In order for this to play nicely with the Unreal assets that represent these types it is necessary to store it outside the normal Content folder of a project. This has a few benefits:

• Values are human readable and editable
• Allows for scripted processing of data
• Allows importing and sharing of data
• Standalone applications can create and import data (you can't create assets in the project at runtime)

There are a few considerations due to using this approach though:

• Files are associated with asset by a unique ID included in the filename
• Custom nodes are needed to load new data into standalone applications (since there will be no Assets associated with the data file)

Location

The data is all stored under one directory (and can have sub-directories) in a path configured in the project ini file (typically Config/DefaultGame.ini) using the following section/key:

[InteractML]
DataPath=../Custom/Data/Path/Relative/To/Content/Directory

The default value for this (if you don't specify it) is ../Data i.e. a directory alongside the projects Content directory called "Data".

Asset Files

Any Unreal managed Training Set or Model assets you add will create a corresponding data file in the DataPath directory refered to above. Once this has happened you are free to move the file and rename the asset as you wish, within reason:

• Moving the data file to a new folder in the hierarchy inside the DataPath directory is fine.
• Renaming the Asset is fine, the data file will have it's name updated to match.

Files created to back assets have the following naming convention:

YOUR_ASSET_NAME.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.THE_ASSET_TYPE.json

YOUR_ASSET_NAME comes from the asset name in Unreal XXX... is the 32 hex digit unique ID THE_ASSET_TYPE is one of the following depending on the asset type/use of the file:

• training
• classification.model
• regression.model
• dtw.model

A real example might be something like this:

Data/ClassificationDemo_Labels_Model.9BEF612D480BC31DB2840F997CC443E9.classification.model.json

💡 When adding new (or editing existing) Training Set and Model assets to source control, make sure you add/update the corresponding data file for each one in source control too. For example, the Classification demo here has new Training Set data:
images/ui/Example_SourceControlOfDataFiles.png

Accessing New Data

If you are providing training and running functionality at run-time in a standalone application you will need to refer to it directly by filename. Filenames are whatever you want but the path is always within, and relative to, the above DataPath directory location.

External Training Set Data

To use a Training Set data file directly you need to add a Training Set (External) Blueprint node to your graph.

images/ui/Example_ExternalTrainingSet.png

Inputs:

• images/ui/Pin_String.png Data Path - String of path (optional) and filename, without extensions (they are implied), e.g. Ext/NewExamples1

Outputs:

• Training Set images/ui/Pin_Object.png - A training set object that can be used with Example Recorder or Teach The Machine nodes
• Has Data? images/ui/Pin_Bool.png - Does the Training Set have any examples in it?

See Recording Examples and Training Models for more 👉

External Model Data

To use a Model data file directly you need to add a Model (External) Blueprint node to your graph.

images/ui/Example_ExternalModel.png

Inputs:

• images/ui/Pin_String.png Data Path - String of path (optional) and filename, without extensions (they are implied), e.g. Ext/NewExamples1
• images/ui/Pin_Wild.png Model Type - Select the type of model you are expecting in the drop-down box.

💡 Changing the Model Type affects the expected file extension of the data file.

Outputs:

• Model images/ui/Pin_Object.png - A model object that can be used with Teach The Machine or Machine Learning Robot nodes
• Trained? images/ui/Pin_Bool.png - Has the model been trained and can be run?

See Training Models and Running Models for more 👉

Save and Load

The temporary Training Set and Model objects that these nodes yield are maintained in memory from any Play-in-Editor sessions like a normal asset, however, as they don't have any actual project asset presence, the save and load process differs from normal slightly.

Load is performed when first used or re-used after being discarded (unreferenced and no unsaved data).

Save is performed when the currently loaded Level is saved (this is marked dirty by changes to the temp assets as a way of ensuring a save is performed at some point).

💡 Save can be performed explicitly if desired using the Save function node, for example in a standalone application where there is no editor functionality. See Utility Blueprints for information 👉

---

👈 [Labels]] ](/Interactml/iml-unreal/wiki/🏠-[[Home) | Demo Project 👉