KotOR Modding 101: Introduction to Modding for KotOR and TSL - lachjames/KotOR-Modding-Guide GitHub Wiki

Introduction

KotOR/TSL both use (almost) the same game engine: the Odyssey engine, developed by BioWare. This engine is an extension of the Aurora engine used for the Neverwinter Nights games, and is very similar to Aurora in a lot of ways - so much so that I named this project after Aurora. Similarities include file structure, most file types (although there are new fields for the KotOR games), and just the way the games work in general.

Note: This guide is written for a wide audience, including those who have never modded KotOR/TSL before, those who would like a refresher, and modders who just want to see if they have any gaps in their knowledge. I do assume enough familiarity that you will understand when I talk about things like dialog, combat, inventory, etc. I will endeavor to avoid spoilers, but please finish the games before modding them (mainly because they're great, but also because you'll be able to better understand how the various systems work).

What we'll learn

In this guide, I will discuss the following key concepts, which should be well-understood by any successful KotOR modder:

  1. File structure: The game files for each game are, generally, stored in a collection of "archives" (a sort-of zip file). We'll discuss how these archives are organized, what they contain, and how files in one folder/archive can override a file in another folder/archive.
  2. Formats: There are many different file formats used within the game files (most of them custom-built for either the KotOR games or, more commonly, the games that preceded them). Some of these are "archive" formats, which are used like zip files and contain other files within them, and others are "data" formats (with GFF, a custom, XML-like structure, being the most common of these).
  3. Modules:
  4. GFF Files (Templates):
  5. GFF Files (Non-Templates):
  6. Scripting:
  7. Dialogs:

1. File structure

The KotOR games use custom archive formats (as many games do) to compress/simplify the file structure behind each game. We will consider the contents of these archives later on, in the formats section - for now,

The override folder

The utility of this folder is probably questionable to people new to KotOR modding, as the folder comes empty with the base games. However, it's very important - as the name indicates, files placed in this folder will override correspondingly named files found elsewhere in the game files. Essentially, the game looks here for files and, if it doesn't find them, then it goes and looks in the rest of the game files.

Modules

The file structure of modules is discussed in section 3 ("Modules").

Global files

Each module has its own templates, scripts, dialogs, etc., but there are also "global files", which are stored in the "data" folder (in BIF archives). Generally (with some exceptions), your mods will not touch these files; instead, you will place files in the override folder if you want to override a file globally.

There are many BIF archives in the data folder, including:

  • 2da.bif: Contains all the 2DA (two-dimensional arrays). These are very important, so we'll discuss them in their own section further down.
  • gui.bif: Containing templates for the GUI definition (in GFF format)
  • items.bif:
  • layouts.bif:
  • legacy.bif:
  • lightmaps.bif:
  • models.bif: Contains all the models used in the game (in both KotOR and TSL, all models are either stored here or in the override folder, and are not module-specific)
  • party.bif:
  • player.bif:
  • scripts.bif: Contains all the scripts which can be run by any module
  • sounds.bif: Contains all the sounds which can be accessed by any module
  • templates.bif: Contains all the templates which can be instantiated by any module
  • textures.bif: Contains all the textures used in the game (similar to models, to override or add new textures you put them in the override folder, and not in module files).

2DA files

2DA files are kind of like CSV files, except each column has a header. For example, a simple 2DA file might look like:

Row Name Weapon Level Model
0 Bastila Lightsaber 16 p_bastila
1 Carth Blaster pistol 12 p_carth
2 HK-47 Blaster rifle 17 p_hk47

(Note: this is not taken from a real 2DA file, and the values are all made-up and nonsensical. This is for illustration purposes only, so please don't use this as a reference for anything).

To access data from a 2DA file, we need to reference both a row (the first column) and a column (such as "Name", "Weapon", ...). We'll see how to do this through scripting later, and you can have a try at editing the 2DA file for yourself when you create your first mod in "KotOR Modding 102", but for now just be aware that the game engine relies heavily on the values in these tables for many things, such as appearance, class balancing, etc., and we are still learning new things about the 2DA files even nearly 20 years after KotOR's initial release.

Override priority

2. Formats

Data formats

GFF

The GFF format is used for the vast majority of the game files in both games. At a high level, it is very similar to a language such as XML/JSON/YAML (in purpose, at least). It is a flexible file format designed to hold key-value pairs.

A GFF file contains a single, high level struct.

Structs

A struct is a collection of key-value pairs, where the keys are names in string format and each value is either another struct, a list of structs, or just a value (such as a string, an integer, etc.).

Lists

In GFF, a list is an ordered collection of structs (never of values or other lists, as far as I have ever seen at least).

Values

A value is some piece of raw data, such as a string ("Hello World!"), an integer (123), a float (0.42), etc. These values can be of the following types:

Raw data types

Most data types in GFF are quite straightforward. These include:

  • Resource reference (resref): A string that gives the name of a resource (usually a GFF template of some kind). The game will look for the resref using the rules outlined in the "Override Priority" section found in part 1 of this guide.
  • Integers: These are whole numbers, such as -5, 42, ..., which are contained within four bytes and either signed (meaning they can be positive or negative), or unsigned (meaning they can only be >= 0).
  • Shorts: Shorts are like integers, but they take only two bytes (so they take less space, but have a smaller range of values they can take on).
  • Longs: Longs are like shorts and integers, but they take eight bytes (so they take up more space, but have a larger range of values they can take on).
  • Byte: Bytes are like integers, but they are only take up a single byte (and are always unsigned in GFF). So they can only take a value from 0-255. Bytes are often used for boolean (true/false) values in GFF files, but can be used for other things too.
  • Floats: Floats are how computers express any number (not just integers), such as 42.7, 0.12, etc. Floats are four bytes in GFF. GFF floats are always signed.
  • Doubles: Doubles are like floats, except they take twice as much space (hence "double", so 8 bytes) and can express a wider range of values. GFF doubles, like GFF floats, are always signed.
Text types

In GFF, text is handled specially (because, very often, text in GFF files is actually just a pointer to a line in the TLK file - discussed later - rather than the actual text data itself). There are, therefore, several special value types which deal with text:

  • Localized string (locstring):

Archive formats

The Odyssey engine mainly uses three different types of archives: RIM, ERF, and BIF archives. At a high level, these are both very similar to, say, a zip file, and they are simply a way to store a bunch of files as a single file (and possibly compress the collection of files too).

RIM and ERF files

RIM and ERF files, from a high level perspective, are quite similar to one another. They contain a table of contents, which is a list of pointers from a resource reference "resref" to the location of that resource in the archive, and then the resources themselves stored as binary data.

You will probably want to edit RIM/ERF files at some point while creating mods - the KLE can automate this process for you.

BIF

BIF archives are a little different to RIM and ERF archives, because they're keyed. Each game has a special file, called "chitin.key", which essentially is a list of pointers from resource references (resrefs) to a location in a BIF archive where that resource can be found.

You probably won't have to work with BIF files too much, as most things in the BIF archives can be overridden through the Override folder. However, if your mod requires editing BIF files (for example, editing a 2DA file and not just overriding it - likely for compatibility reasons), you can use TLKPatcher to make this process automatic and much more straightforward.

MOD

MOD files are saved in ERF format.

SAV

SAV files are saved in ERF format.

3. Modules

In both KotOR and TSL, the games are defined as a collection of modules. Modules contain an area (and only one area, as far as we can see, although in other games such as NWN modules can contain multiple areas), and this area contains both a set of static models (the scenery, which is things like the floor, walls, static furniture, etc.) and a set of instantiated templates (things like creatures, doors, placeables, etc.).

Modules also contain templates, scripts, and dialogs which are used only by that module (and are therefore not included in the global data).

File structure of modules

For the PC version of KotOR 1, modules consist of two files (both archives in RIM format). For the PC version of TSL, modules consist of up to four files (all archives, with some in RIM format and others in ERF format). The Xbox versions of both games use other files for modules too, but we won't consider that here.

Let's take a look at the files defining a particular module (using the module "101PER" as an example, although the structure is shared by all modules in both games unless otherwise stated):

101PER.rim

This archive contains only three files (all of which are in GFF format):

  • 101per.are: The ARE file, which includes some metadata about the module's area (please note, especially for modders of other games using a similar engine, such as NWN, that in KotOR/TSL every module only has one area). Includes information such as area scripts (on enter, on user defined, etc.), music, etc., as well as a list of rooms in the area. When loaded from a save (SAV) archive, ARE files also include information about the state of the area.
  • 101per.git: The GIT file, consisting of a list of the template instances in the level (e.g. creatures, doors, encounters, ...), their positions, and any other metadata specific to the particular instance of the template (rather than information about every instance of that template, which is generally stored in the template itself).
  • module.ifo: The IFO file, which contains some metadata about the module (such as scripts to run on load, on heartbeat, etc.), as well as the list of module areas (although all modules in KotOR/TSL only have one area).

101PER_s.rim

This file contains all the content from the base game for a particular module. In both games, this includes all templates specific to the module (that is, templates which are not already present in the "data/templates.bif" archive, which I call "global templates"). It also contains all scripts specific to the module, and (in KotOR 1 only) the dialogs specific to that module as well (in TSL, the dialogs get their own ERF archive, discussed below).

101PER.mod

This is an archive in ERF format, which contains overrides for the files in the other three archives. These .mod files don't come with the base game, but they are used by both official game patches and mods (so, depending on which version of the game you get, there may or may not be .mod files for some modules). These .mod files act as a "personal override folder" for a particular module, and will override any items in the other archives with the same name (but it is still superseded by the override folder; TODO: check this?).

101PER_dlg.erf (I’ve only ever seen this used in TSL)

This is an archive containing all the DLG files used by a given module, archived in ERF format.

Layout files

Modules also have a layout file associated with them, which contains the names and positions of the rooms in a given module. These layouts are stored separately to the other files for a given module (aka not in the modules folder, but rather in the BIF archive Layouts.bif, found in the data folder).

Layout files are written in plain-text, and are therefore relatively straightforward to read and edit (at least compared to GFF files, which require a tool to read/edit).

4. Template files

In this section, we discuss the main template file types (all of which are stored in GFF format). Templates are, as the name implies, a sort of "blueprint", which can be instantiated within the game (perhaps once, or as many times as you would like). So far, we have called these instances "entities".

Templates contain starter values for almost all the describing variables of an entity. However, once the game is run, a particular instance of a template might (and probably will) have some values change; this does not change the whole template, but only the individual instance (as an analogy, if you have the same house blueprint as a neighbour, and they paint their door bright red, you'd be upset if your door suddenly changed to bright red too).

Creatures (UTC)

Creatures (sometimes called "characters") are, as the name suggests,

Doors (UTD)

Doors, like in real life, connect two regions of space together. Some doors connect two regions within the same area (e.g. rooms), and others will move the player to a new module when using them (allowing transitions between modules).

Encounters (UTE)

Items (UTI)

Merchants (UTM)

Merchants, sometimes called "stores", sell items to, and buy items from, the PC. Some merchants have a fixed inventory (although some items in that inventory might have unlimited supply), while others - especially in TSL- might also randomly augment this fixed inventory with new gear.

Placeables (UTP)

Placeables are three-dimensional objects placed in an area. Types of placeables include, but are not limited to:

  • Simple static models like chairs, speeders, etc.
  • Static models that might be created or deleted with scripts
  • Containers, such as footlockers and crates
  • Interactable objects (such as the "fishy-fishy guy" from Manaan - he's not actually a creature at all, but just a placeable!)

Sounds (UTS)

Triggers (UTT)

Waypoints (UTW)

5. Non-template GFF files

These files are not GFF templates, but rather are entities of their own. These GFF files can be modified by the game engine at runtime, to update the information stored within them (this is, for example, how save games are implemented).

Dialog files (DLG)

Area definitions (ARE)

GIT files (GIT)

IFO files (IFO)

Factions (FAC)

GUI files (GUI)

Journal files (JRL)

Path files (PTH)

Resource(?) files (RES)

Not much is known about these files, as they're only found within save-games for KotOR/TSL. I believe RES files might be generic GFF files (generic in that they can contain any structure, although the game engine will surely expect a certain structure for a given RES file). It's very unlikely you'll have to deal with these for any mods you create.

6. Scripting

Scripting is a crucial part of creating nearly any non-cosmetic mod for the KotOR games. It can be daunting for newer modders, and especially for those who have never written code before, but I promise it's not as bad as it looks.

The NSS scripting language

The KotOR games both use the NSS scripting language, which is almost exactly the same as the implementation used for the first Neverwinter Nights game. NSS, sometimes caller Neverwinter Nights Script, NWScript, or some other similar name, is a programming language with the following properties:

  • It's a C-like language (meaning)
  • It's quite simple, has no notion of classes, and has rudimentary struct support (although, to my knowledge, there is only one struct in each game's NSS codebase). Structs will probably prove unnecessary for almost anything you want to do in NSS, and the NCS compiler (discussed later) actually converts structs to a bunch of variables anyway. Although structs are great in most programming languages, I tend to avoid them because they are not commonly used and, therefore, might not be properly supported by the compilers used to convert our NSS code into NCS code that the games can run.

Here is an example script, which calculates the first five Fibonacci numbers and prints them out, one by one:

void main() {
    int n;
    for (n = 0; n < 15; n++) {
        int a = 0;
        int b = 1;
        int tmp;
        int i;

        for (i = 0; i < n; i++) {
            tmp = b;
            b += a;
            a = tmp;
        }
        PrintInteger(b);
    }
}

Let's step through this script step-by-step and see exactly what it does:

int n;

This line defines a new integer variable, called n, and does not give it a value.

for (n = 0; n < 15; n++) {
    ...
}

This is a for-loop construct, discussed in further detail below. The start of a for loop contains three elements:

  • An initialization, which sets the initial value of some variable
  • A condition, such that when this condition is no longer met the loop will stop iterating
  • An increment, which changes the counting variable

Note that you can do more complicated things with for loops, but this can get confusing (and is therefore not recommended, as a general rule).

Let's step "inside" the for-loop now, and see what happens at every iteration.

        int a = 0;
        int b = 1;
        int tmp;
        int i;

This block of four lines of code defines four integer variables. The variable a is set to 0, b is set to 1, and tmp and i are not given any initial value (because they will be set later).

        for (i = 0; i < n; i++) {
            ...
        }

Here we have a for-loop nested inside another for loop - using nested loops is not something you're likely to need for most of your scripts, but it's useful for calculating Fibonacci numbers, as we're doing here.

This for-loop does three things, just like the one before it:

  • An initialization: we set the initial value of i to 0
  • A condition for continuing the loop: in this case, we continue if i is less than n
  • An increment: we increment i by 1 every iteration of the loop

Let's go into this second for-loop now, and see what happens at every iteration.

            tmp = b;
            b += a;
            a = tmp;

To calculate the next Fibonacci number, we need to set b = a + b, and a = b, simultaneously. However, we run into a problem: if we change b first, the line "a = b" won't work, because b has already updated, and if we change a first, the line "b = a + b" won't work, because a has already updated! So, we need to store one of these values in a "temporary variable", then modify the original variable we saved a copy of, and then finally use that temporary variable to update the other variable.

In this case, we do the following:

  • Save b to the temporary variable "tmp"
  • Add a to b using the "+=" operation
  • Sets a to the value of "tmp", which was set to the old value of "b" before we added "a" to it in the previous line of code

So, finally, we have updated both the variables properly!

Variables

Conditional structures

void main () {
    int x = 0;
    int y = 1;
    if (x < 2) {
        x -= 1;
    } else {
        x += 1;
    }
}

Control structures

void main () {
    int i = 0;
    while (i < 10) {
        PrintInteger(i);
        i++;
    }
}

Engine Actions

Calculating Fibonacci numbers is fun, but it's also not very useful if you're trying to mod a game! To interact with either KotOR or TSL, you will probably need to use engine actions, which are essentially named pieces of functionality that you can ask the engine to do. We've already used an action above, when we called PrintInteger(value) and put a variable

For each game, you can find the list of actions (slightly hidden) in the file "nwscript.nss", contained in the archive "data/scripts.bif". There are hundreds of actions for each game, and far too many to discuss here, so for now we'll just discuss the ones we need as we go along.

You call an engine action using the same syntax you use for subroutines; essentially, you state the name of the action, and then you pass parameters "into" the action (within a set of brackets) after the action's name. For example, we might call the action

Subroutines

Subroutines are blocks of code which we might want to give a name, so we can call them many times without writing them out many times. Subroutines are defined with the following format:

type name (args) {
    body
}

Let's look at each element of the subroutine definition:

  • type: The type of value returned by a subroutine - a subroutine might not return anything (in which case we set the type to "void"), or it might return, for example, a "string" value (so we set the type to "string"). When we call the subroutine, we can assign the result to a variable, or even use it in another subroutine call!

Below is a toy example of how subroutines can be defined and used in code:

int mult2 (int x) {
    return 2 * x;
}

void main () {
    int i = 0;

    while (i < 10) {
        PrintInteger(mult2(i));
        i++;
    }
}

Delegate actions

Some actions are special, because you can take another action or subroutine call and pass the call itself into the action as a parameter. For example, the DelayCommand action takes two actions:

  • fSeconds: a float number
  • aActionToDelay: a nested action or subroutine call (despite the name, you can use subroutine calls here too)

For example, we might ask the engine to print the number 42 in five seconds, like follows:

DelayCommand(5.0, PrintInteger(42));

Normally, when we call PrintInteger(42), the number 42 would be instantly printed to the log file. However, because we have wrapped it in a DelayCommand, this action will instead be performed in five seconds (long after the script itself has finished executing).

Writing scripts

Compiling scripts

Neither game is able to read the text-based NSS files that we write our code in. We need to "compile" our scripts, converting them into a format the game engine can understand. To do this, we will use a very useful tool called "nwnnsscomp", which was originally developed for Neverwinter Nights (hence the name). There are many versions of nwnnsscomp, some of which work better than others. TODO: link to a good version of nwnnsscomp.

We have two options for using nwnnsscomp:

  • We can use command line to run it
  • We can use an open source GUI interface wrapper (developed by yours truly) TODO: add link

When you're getting started, if you have no command line experience, I'd recommend using the GUI tool. However, there are benefits to using the command line tool (such as learning about the command line, which can be very useful while modding, as well as the ability to compile more than one script at a time via command line batching).

Using scripts in game

You can call scripts through a variety of mechanisms, including:

  • When a given dialog line is played
  • When a given dialog line is being considered (in this case, we use a StartingConditional script, discussed below)
  • When the player enters an area
  • When the heartbeat for an entity is triggered by the engine
  • And dozens more - if you want a script to be triggered when something happens in the game, you probably can do so

Most of the triggers for scripts are fairly self-explanatory (e.g. OnEnter for areas/triggers/..., OnDamaged, etc.), but a couple of them aren't so obvious (and these ones tend to be the important ones!).

OnHeartbeat scripts

Most entities in the game has an OnHeartbeat script, from modules, to creatures, to doors. The OnHeartbeat script is run at regular intervals by the game engine; in both games, the engine defines the length of a heartbeat to be six seconds, but it's not guaranteed that the OnHeartbeat script will run exactly every six seconds.

Generally, these scripts are useful for regularly executing a command on an entity (such as a call to the k_ai_master, which manages the AI for most creatures in the games, and expects to be called regularly by OnHeartbeat, as well as by other events).

It's unlikely that you'll have to deal with OnHeartbeat scripts for smaller mods, but they can be very useful when trying to make entities do complex things over time. We'll see an example of this in "KotOR Modding 201".

OnUserDefined scripts

The base games come with many scripts, which govern things like the friendly/enemy AI, idle movement for creatures, loot tables, etc. It would be painful (and certainly error-prone) to be forced to duplicate an entire script if you just want to make an addition, for example, to what happens when a creature notices the player.

Fortunately, the game's scripts are setup to trigger the OnUserDefined script at many different points (if you've ever heard of or used "event-driven programming", that's exactly what the OnUserDefined script allows). When the OnUserDefined script is called (we'll discuss how to call OnUserDefined in "KotoR Modding 201"), it is passed a "script variable" or "ScriptVar", which can be retrieved within the OnUserDefined script using the engine action "GetRunScriptVar()". There's nothing special about the script variables passed into OnUserDefined - we assign different values different meaning by convention. For example, if the script variable is set to the integer 1002, this indicates that an OnPerception event has occurred (and that the script should act accordingly).

The allocation of different integers to different events might seem quite arbitrary at first (and, to some extent, it is arbitrary). Fortunately, this information is well documented in the NSS scripts included with the game (for an example, take a look at the script k_ai_master.nss, located in the "data/scripts.bif" archive).

Conditional scripts

Most scripts are used to make the game engine do something, but sometimes they can be used for another purpose (mainly within dialog files). When looking through the included NSS scripts in either game, you might notice that some of the scripts use a main() function, and others use a StartingConditional() function. While the former has been explained in detail above, the StartingConditional() function is slightly different. These scripts are attached to links in DLG files between either a reply and an entry, or vice versa. These links are discussed in the DLG section, so for now we will focus on the point of StartingConditional() scripts, which is simply to determine whether the player is eligible for a given line of dialog to be played (we return 1 if they are, and 0 if they're not).

int StartingConditional() {
    
}

NCS and how the scripting language actually works

This is a significant point of discussion in "KotOR Modding 301", which I suggest reading after this tutorial, and maybe after creating some simple mods first. It goes into much more detail about how the game engine works behind the scenes. For now, we are going to basically consider NSS scripts "magic".

7. Dialog

KotOR and TSL are both RPGs with a very heavy story element, so it's no surprise that one of the main focuses of many larger mods is to create stories. These stories are generally conveyed to the player through the use of dialog, and KotOR/TSL therefore have quite powerful dialog engines.

Dialogs, DLGs and TLKs

A dialog, in KotOR/TSL, is essentially a set of entries (things that NPCs can say) and replies (things that the PC can say), all linked together to create a dialog tree. Newer modders often find the way dialog works in KotOR/TSL confusing, because dialogs are split over two files: the structure file (DLG) and the localization file (TLK).

The DLG file

The DLG file contains the structural information for a given dialog (i.e. after the NPC says x, the player can say a, b, or c). This structural information is language-independent.

The TLK file

The TLK file, on the other hand, contains all the language-dependent information about a dialog, such as the text of the line, the name of the sound and lip sync files, etc. The DLG file refers to lines in the TLK file using "resrefs", or resource references. Essentially, the TLK file consists of thousands of different lines of dialog, and each line of dialog in a DLG file will point to a line in the TLK file (unless it doesn't, which is often the case, as we'll see below).

You can actually hard-code text into the DLG file, rather than using resrefs to point to lines in the TLK file. This is very common in English-only mods (which are very common). You can actually do localisation in the DLG file itself too; however, if you plan to release your mod in multiple languages, using TSLPatcher to dynamically edit the TLK file is probably easier in the long run.

Creating and editing a DLG

DLG files are stored in GFF format, and contain structural information about a given dialog tree. There are several custom tools for editing DLG files, including:

  • DLG Editor, which has been the tool used for editing dialogs for years.
  • KotOR Tool comes with a dialog editor, but it's generally better to use one of the other tools if you can.
  • The KotOR Dialog Editor (a visual editor developed by me, but I recommend using the KLE instead as it's much better).
  • The KLE's built-in dialog editor, which is the tool I recommend you use if you are building your mod with the KLE.

Editing the TLK

The TLK file (notice I said "the" TLK file, as there is only one per game) contains a list of all dialog lines spoken in the game, containing the text spoken, the sound and lip sync files, etc. The file is stored in GFF format, just like DLG files.

Each language the game is developed for has its own TLK file. Sometimes, for certain languages, there can be more than one TLK file to accommodate differences in how the language works (I only speak English so I'm not entirely sure about how this works, to be honest). For now, let's assume there's only one TLK file.

You can use TSLPatcher to dynamically insert dialog lines into the TLK file while installing a mod. Eventually, KLE will support generating TSLPatcher definitions, but unfortunately this is a manual process for now.

Changing the game state through dialogs

Using the KotOR Level Editor's Built-in Dialog Editor

The above steps for creating and editing dialogs can be a bit overwhelming for newer modders, and even for very experienced modders it can take some time (as well as a bit of trial and error) to get dialogs working properly. Fortunately, the KotOR Level Editor (KLE) has a built-in visual dialog editor that can abstract away much of the complexities of creating and editing dialogs.

You can see information on how to use the KLE dialog editor on the "Using the Level Editor" page on this wiki.