Learning the TOS API - meldavy/ipf-documentation GitHub Wiki

Even as an experienced developer, coding without API documentation is tough. It's a repetitive trial and error process, reverse engineering, a ton of searching around for interesting pieces of code, and then realizing it doesn't work as expected. It's a tough and painful process, so I wanted to use this opportunity as a way to share my findings and document everything as I go.

When I started addon development, I spent a ton of time just trying to figure out how to get even the most basic of functions working, until I hit a hard stop. At that point, I decided that I'm not an efficient developer without the necessary documentation resources. So I started on my first big attempt at doing a code analysis.

https://github.com/meldavy/tos-ipf

First things first is doing a source dump. I created a simple application which loops through all IPFs and extracts the lua and xml file using IPFUnpacker. This gives a good starting point of being able to open up the source dump in an IDE and being doing a bunch of ctrl+F to search for interesting lines of code.

https://github.com/meldavy/ipf-documentation/blob/main/global_dump.txt

Globals are the closest thing you can get to an API documentation, as it contains all callable lua functions within the scope of the game. So the next step was to dump globals. I created a script that loops through the _G global table and dump everything. I learned a lot from table dumps done previously by other community members:

• https://github.com/JTosAddon/JTosAddonWiki/wiki/sessionFuncList
• https://github.com/NoctisCepheus/ToS-Addons-Cepheus/blob/master/function_afterRebuild.lua But I decided that this information might be out of date and I wanted to build a full dump for myself, resulting in the above global dump.

I'm fairly new to lua, and my background is majorly in OOP. Thus understanding the difference between a table and userdata initially took some time.

A table is like a map in OOP. It has a key and a value. It can also be used as an array, which is fairly straightforward. You can retrieve many data from the global table, especially from places like the session table within _G. The tables contain many getter functions that usually return you values you expect. However, when handling more complex types (basically almost everything that is not the an object's ID), you will start working with userdata. userdata is an arbitrary data type that is passed between C/C++ code and lua. And considering that low-level languages like C and C++ work with memory addresses rather than objects, types, and references, userdata is basically just a pointer to memory. Thus, it becomes impossible to know what fields and values are exposed by userdata objects, and you are basically stuck with a blackbox. And the best way to learn how these blackbox objects work is by looking at IMC's code!

A TOS Addon consists primarily of a few parts:

• Initialization
• Functions (our code)
  • Entry points:
    • Messages
    • Hooks
    • Other forms of invocation
• XML
• UI

Inside this particular page of the wiki, I will be going over only the core fundamental parts: Initialization, Messages, and Hooks. UI and XML will be explained separately in its own page (and I'm also learning addon UI apis so I don't currently have the expertise for a writeup), and I will share other commonly used functions in separate cookbook page.

Most addons are developed with two different approaches: Messages and Hooks, and they serve as the primary entry point of your addon.

https://github.com/meldavy/ipf-documentation/wiki/Message-(Event)-dump

Messages are synonymous to a typical event driven architecture. When an event happens, a message is broadcasted. And all functions that are listening to that message will get invoked.

    addon:RegisterMsg('TARGET_SET', 'MYFUNC_ON_TARGET_SET');
    addon:RegisterMsg('TARGET_UPDATE', 'MYFUNC_ON_TARGET_UPDATE');

Tree of Savior broadcasts the TARGET_SET message whenever the user locks on to a new target, and TARGET_UPDATE is broadcasted when the current target changes. We utilize the addon API which looks like follows (snippet from global dump)

[imcAddOn]:  table {
    [BroadMsg]: function()
    [CAddOn]:  table {
        [__name]: string = imcAddOn::CAddOn
        [RegisterOpenOnlyMsg]: function()
        [tolua_ubox]:  table {
        } // [tolua_ubox]
        [RegisterMsg]: function()
    } // [CAddOn]
} // [imcAddOn]

We can use this system to broadcast custom messages using addon:BroadMsg(args), and also listen to messages by addon:RegisterMsg(args) or addon:RegisterOpenOnlyMsg(args). The difference between RegisterMsg and RegisterOpenOnlyMsg is not known.

To figure out what messages are available, I also had to do a dump. I wrote a quick script that reads through all lua code, and uses some simple regex to extract all messages and their usages (link to the dump above)

Hooks are more of a hacky workaround. I'm not sure if this is how lua works in general, but the way it works in TOS is that all lua functions are stored in the global table. Let's say that there is a function that IMC created, called FUNCTION_DEVELOPED_BY_IMC, and it is internally invoked, let's say when your character receives damage. When your character receives damage, the invocation code will check the global table for a function named FUNCTION_DEVELOPED_BY_IMC, and tell it to run.

As mentioned earlier, a table is like a Map in OOP. In this case, the key is FUNCTION_DEVELOPED_BY_IMC function name, and the value is the actual function. However, as with all maps, you can replace a value of an entry in a map as long as you know the key. Thus, we can replace the function behind FUNCTION_DEVELOPED_BY_IMC function name to whatever function you want.

The addon community utilizes hooks by

   local originalFunction = _G["NAME_OF_FUNCTION"];
   _G["NAME_OF_FUNCTION"] = myNewFunction;
   _G["NAME_OF_FUNCTION_OLD"] = originalFunction;

The convention is to move the original function to a new key in _G by appending "_OLD" at the end of the original key, and insert our hook function into the original key. We still need a reference to the original function because the original function contains important code for the game. For instance, 'QUICKSLOTNEXPBAR_SLOT_USE' is a global function that is invoked when a user presses a skill hotkey. If we simply replace the function with our own, every time a user presses a skill hotkey, our own function will run instead. However, the original function no longer runs - thus our character no longer uses skills because we suppressed the actually logic behind a hotkey key press.

Thus, when using hooks, it is important that we callback the original function to preserve the original action of the function.

init code ...
   local originalFunction = _G["NAME_OF_FUNCTION"];
   _G["NAME_OF_FUNCTION"] = myNewFunction;
   _G["NAME_OF_FUNCTION_OLD"] = originalFunction;

function myNewFunction()
    -- do stuff
    NAME_OF_FUNCTION_OLD(); -- invoke original function
end

Without going too deep into the lua language and variable scopes etc, the last bit of an addon anatomy that I would like to go over in this page is the initialization. The best explanation (which barely contains any explanation) that I could find was from a Japanese blog I found through google. But basically, you want an init method that follows this addon convention:

-- third party addon conventional boilerplate
local author = 'yourname'
local addonName = 'youraddon'
_G['ADDONS'] = _G['ADDONS'] or {}
_G['ADDONS'][author] = _G['ADDONS'][author] or {}
_G['ADDONS'][author][addonName] = _G['ADDONS'][author][addonName] or {}
local g = _G['ADDONS'][author][addonName]

function <ADDON_NAME>_ON_INIT(addon, frame)
    g.addon = addon;
    g.frame = frame;
    -- register hooks and messages here
end

I'm pretty sure you can register hooks and messages outside of an ON_INIT method, but there might be some danger there where resource loading is not properly guaranteed (for instance the addon userdata used to register to a message might not be available at the time of that scope), so it is typical to put addon initialization code within ON_INIT. I am not sure how TOS's internal loader searches for ON_INIT functions, but I guess that's just how it works.