Exporting Data - npruehs/tome-editor GitHub Wiki

Exporting your data in any arbitrary format is what makes Tome really useful compared to similar tools. No matter what game engine you use, or which data format you prefer, you can automatically write and format your data in any way you like.

Tome needs to know how to export your data. You specify your custom output format by creating a bunch of template files that will be automatically injected and filled where appropriate by Tome. Special placeholder strings will be filled with your actual data.

We are using multiple file templates instead of just one, because we've had a bad experience with separating templates from each other in text-based formats. For example, Resharper code templates are stored in XML files, which require the templates themselves to be escaped in strange manners, turning these template files hard to read and edit. Also, splitting templates up into multiple files supports one of the goals of Tome, improved collaboration.

Because this is a rather abstract topic, we'll walk through a JSON export template as example. The following sections will describe each of the template files in detail.

Tome will create a data file of your custom output format. It will write your file template (.texportf), and replace the special placeholder string $RECORDS$ with the actual record data (see below).

An example JSON export file template might look like this:

{
  "records": [
$RECORDS$
  ]
}

Upon encountering the $RECORDS$ placeholder in the file template (see above), Tome will inject all of your records by applying your record template.

Example:

    {
        "id": "$RECORD_ID$",
        "attributes": {
    $RECORD_FIELDS$
        },
        "components": [
    $RECORD_COMPONENTS$
        ],
        "parent_id": "$RECORD_PARENT$"
    }

In some cases, you want a handful of fields to be handled differently from others. For example, some XML data formats might require you to export a field as XML attribute, while all other data should be exported as XML elements.

Tome allows you to use the $FIELD_VALUE:yourfieldname$ placeholder to insert the value of yourfieldname at the specified place in your record template. The respective field will automatically be skipped when evaluating the $RECORD_FIELDS$ placeholder.

As most text-based data formats require records to be separated from each other by one or more delimiter characters, you may specify your delimiter string in this additional file. This will be inserted after each record except for the last one. In our JSON example, this file contains just a single comma and a line break.

,

Next, Tome will write the actual values of all of your record fields by applying your field value template.

Example:

      "$FIELD_ID$": "$FIELD_VALUE$"

Whenever encountering a field of a derived type with the Localized facet, Tome will apply your localized field value template instead. This allows you to export localization keys here, and creating another export template for exporting the actual localization data. Then, you can import the data with the localization keys once in your game, and the localized strings multiple times, once for each language.

Example:

      "$FIELD_ID$": "$RECORD_ROOT$.$RECORD_ID$.$FIELD_ID$"

Just as for records, field values often require to be separated from each other by one or more delimiter characters. This will be inserted after each field value of the same record, except for the last one. Again, in our JSON example, this file contains just a single comma and a line break.

,

In case you are exporting record components, Tome will apply this template for each component that contains at least one field used by the record.

Example:

            "$COMPONENT_NAME$"

In most cases, components require to be separated from each other by one or more delimiter characters as well. This will be inserted after each component of the same record, except for the last one. Again, in our JSON example, this file contains just a single comma and a line break.

,

If your field is of a list type, three different field value templates are applied. This allows you to specify that lists should be exported differently than other field values. List templates allow the same placeholders as other values (.texportv).

Example:

      "$FIELD_ID$": [$FIELD_VALUE$]

For each item in the list, the list item template is applied.

Example:

"$LIST_ITEM$"

Tome inserts the list item delimiter template between each pair of list items:

,

If your field is of a map type, three different field value templates are applied. This allows you to specify that maps should be exported differently than other field values. Map templates allow the same placeholders as other values (.texportv).

            "$FIELD_ID$": {
    $FIELD_VALUE$
            }

For each key-value pair in the map, the map item template is applied.

Example:

            "$FIELD_KEY$": "$FIELD_VALUE$"

Tome inserts the map item delimiter template between the key-value pairs:

,

In order to make the above export template available in Tome, you need create template files and add references to them in your Project Overview window.

Export template files (.texport) specify the name of your template, as it should be shown in the editor menu bar, as well as the default file extension of the exported files:

<?xml version="1.0" encoding="UTF-8"?>
<Template Version="2" ExportLeafs="true">
    <Name>JSON</Name>
    <FileExtension>.json</FileExtension>
    <TypeMap/>
    <IgnoredRecords/>
    <IgnoredFields/>
</Template>

When using an export template, Tome will look for all of the above other files with the same filename as your .texport (but with the above extensions).

You need to explicitly state which types of nodes you want to be exported. Each template has ExportRoots, ExportInnerNodes and ExportLeafs attributes for this. Note that you can exploit this feature for creating export templates that actually behave as code generators!

Because almost every game engine has their own name for the basic data types, you may specify a type map for automatically replacing Tome data type names by the ones of your engine.

<?xml version="1.0" encoding="UTF-8"?>
<Template Version="2" ExportLeafs="true">
    <Name>Slash XML</Name>
    <FileExtension>.blueprints.xml</FileExtension>
    <TypeMap>
        <Mapping TomeType="Armor Type" ExportedType="FantasyGame.Data.ArmorType"/>
        <Mapping TomeType="Boolean" ExportedType="System.Boolean"/>
        <Mapping TomeType="Color" ExportedType="System.String"/>
        <Mapping TomeType="Damage Type" ExportedType="FantasyGame.Data.DamageType"/>
        <Mapping TomeType="Integer" ExportedType="System.Int32"/>
        <Mapping TomeType="Movement Type" ExportedType="FantasyGame.Data.MovementType"/>
        <Mapping TomeType="Real" ExportedType="System.Single"/>
        <Mapping TomeType="Reference" ExportedType="System.String"/>
        <Mapping TomeType="String" ExportedType="System.String"/>
    </TypeMap>
    <IgnoredRecords/>
    <IgnoredFields/>
</Template>

Sometimes, you want to export the data in some table format (such as CSV or HTML). Setting the ExportAsTable attribute of the Template element will cause Tome to add empty field entries for any fields not used by a record.

<?xml version="1.0" encoding="UTF-8"?>
<Template Version="2" ExportAsTable="true" ExportLeafs="true">
    <Name>HTML</Name>
    <FileExtension>.html</FileExtension>
    <TypeMap/>
    <IgnoredRecords/>
    <IgnoredFields/>
</Template>

You may also exclude specific records and/or fields from being exported. Inside the IgnoredFields and IgnoredRecords elements, add Id elements specifying the ids of the records and/or fields you want to ignore. If an ignored record has children, these children will be ignored as well.

Note that records without any fields are not exported. Consequently, records whose parents don't have any fields will have their parent id reset for the export.

Tome allows you to specify the ExportLocalizedFieldsOnly attribute to restrict data export to custom types with the Localized facet. This way, you can create export templates for exporting localization data to send to your localization team and import in your game.

Since Tome 1.1, you can also add an element StringReplacementMap with two element children String and ReplaceWith.

You can always select File > Reload Project to reload all export templates. This allows for faster iteration.

After having added the export template to your project file, it will be available from the Export menu:

1. In the main window menu bar, click Export.
2. Select the template to export the data with.
3. Specify an output file.
4. Click Save.

Next: Importing Data