The Persistent Data Service (PDS) is a typed column-store database and ORM system for structured game data persistence. It tracks individual column-level changes with timestamps, enabling audit trails and GM investigation queries via LAS. PDS ran in the original Nevrax production deployment but the delta logging pipeline was broken during the Shard Unifier transition and needs to be reconnected.

PDS replaces manual PDR serialization with a code-generated ORM. The .pds schema language supports typed columns, primary keys, parent/child foreign key relationships, onChange callbacks, onInit blocks, class inheritance, and automatic container management. The pd_parser tool (ryzom/tools/pd_parser/) generates complete C++ classes from these schemas.

So far, guilds, fame, and missions have been migrated to .pds schemas. Characters, items, inventories, skills, friends lists, pets, and respawn points still use the manual PDR path — the PERSISTENT_DATA macro blocks in persistent_player_data.cpp. A pd_logs.pds entry (commented out in egs_pd.pds) indicates structured logging through PDS is also planned.

The PD classes for guilds, fame, and missions are currently bridged back to PDR serialization via proxy classes (CFameContainerProxy, etc.) in persistent_player_data.cpp, so that the existing PDR/BS save path handles their persistence while PDS delta logging is offline.

Schemas are defined in .pds files under entities_game_service/pd_scripts/. Example from guild.pds:

file "entities_game_service/guild_pd"
{
    derived class CGuildPD key(Id)
    {
        TGuildId        Id;
        uint64          Money;
        uint32          CreationDate;
        TPeople         Race;
        uint64          Icon;
        uint32          Building;
        uint32          Version;

        CGuildMemberPD:Guild<>  Members
        {
            onChange()
            @[
                IGuildManager::getInstance().guildMemberListChanged(this);
            ]@
        }
    }
}

The pd_parser code generator produces *_pd.h and *_pd.cpp files from these schemas. Each generated class has typed setters that automatically queue column updates to CPDSLib.

These classes are used in the current EGS as in-memory data structures. The generated code compiles and runs — the class mappings in egs_pd.cpp match the .pds schemas exactly.

Every setter call (e.g. CGuildPD::setMoney()) queues a column update in CPDSLib. When the delta logging pipeline is connected, these updates are written as timestamped delta logs which LAS can query for GM investigation.

Currently, the delta logging is disconnected (see Current Status), so the update queue is cleared every 5 seconds without being written. Persistence of the PD classes goes through PDR instead:

• Fame and missions are serialized into the character's PDR file via proxy classes in persistent_player_data.cpp (e.g. CFameContainerProxy wraps CFameContainerPD)
• Guilds are saved separately by CGuildManager as individual files through BS

Each character save overwrites the previous file — there is no change history.

PDS uses a reference + delta storage model with no SQL database — all data is flat binary files on the filesystem (via BS).

CTableBuffer holds the live data as a hash map of row index to flat byte array (CHashMap<TRowIndex, uint8*>). Each row is a fixed-size byte buffer with a header (allocation/dirty flags, dirty timestamp, optional mapped key for keyed tables) followed by the column data packed at fixed offsets.

One file per table chunk, containing the full state of every row as flat binary. These are point-in-time snapshots. RBS (Reference Builder Service) builds new references by applying all accumulated deltas to the previous reference. The file format includes a header with base/end row indices, timestamp, and delta ID range.

Timestamped log files containing CUpdateLog entries — individual column changes with start/end timestamps. Each entry records which table, row, and column changed and what the new value is. These are the files that LAS queries for audit investigation. Written periodically (every PDLogUpdate seconds, default 5) by pd_lib.

The XML database description, generated from the .pds schemas and stored alongside the reference/delta files. Contains type definitions, table structures, column names and types. LAS loads this to decode binary column data into human-readable output.

Raw deltas accumulate quickly, so a three-tier compaction system keeps the storage manageable:

1. pd_lib writes raw deltas every few seconds
2. RBS (pd_reference_builder/) compacts fine-grained deltas into coarser ones (seconds → minutes → hours)
3. RBS builds new references by applying accumulated hour/minute deltas to the previous reference, then prunes old deltas

PDS sends two message types to RBS:

• RB_GEN_DELTA — Merge fine-grained deltas into a coarser delta file (e.g. all second-level deltas for a minute period into one minute-level file). After merging, the finer-grained source files are deleted.
• RB_GEN_REF — Build a new reference snapshot. Loads the previous .dbref files, applies all deltas in the specified timestamp range, writes new .dbref files. After success, deltas older than a configurable threshold are pruned.

Tasks run in background threads, queued and executed one at a time. If the requesting PDS goes down, pending tasks are cancelled. RBS reports success or failure back to PDS via RB_TASK_SUCCESS / RB_TASK_FAILED.

save_shard/<shard>/pds/
├── description.xml          # Schema for decoding
├── ref/
│   ├── 0000_0000.dbref      # Reference snapshot for table 0, chunk 0
│   ├── 0001_0000.dbref      # Reference snapshot for table 1, chunk 0
│   └── ...
├── seconds/                 # Raw deltas (written by pd_lib, compacted by RBS)
│   └── ...
├── minutes/                 # Compacted minute-level deltas
│   └── ...
└── hours/                   # Compacted hour-level deltas
    ├── 2007.05.04.14.00_0000.pd_log
    └── ...

CPDSLib is linked into EGS and IOS. It supports two modes:

• PDS mode (_UsePDS = true) — Sends PD_UPDATE messages to the PDS server over the network
• Local mode (_UsePDS = false) — Writes delta logs locally via a backup service interface

Key methods: init(xmlDescription), set(table, row, column, value), allocateRow() / deallocateRow(), update() (called each tick).

Receives PD_INIT (schema registration), PD_UPDATE (row-level changes), and PD_SHEETID_MAPPING messages. Stores data in typed column buffers, writes timestamped delta files, and coordinates with RBS for snapshots.

LAS queries PDS delta logs. It opens a web TCP socket for queries from the admin interface, runs them in background threads, and returns paginated, filterable results.

Date format: YYYY.MM.DD.hh.mm[.ss] or relative like -24h, -7d, +1h.

At Nevrax, LAS ran as master/slave pairs on the PD backup machines, with per-shard PDRootDirectory paths.

The PDS delta logging pipeline is disconnected. During the Shard Unifier transition, the backup service interface was refactored and the dedicated PD backup interface (_PDBsi) was removed from _backup_service_interface_singleton.cpp, along with the PDBSHost/SlavePDBSHost config variables. This broke compilation of the delta-writing code in pd_lib.cpp, which was resolved by commenting it out (lines 620-703). The update queue is now silently cleared every 5 seconds.

Character and guild persistence is unaffected — the PDR/BS save path works independently.

• Change history: Every column update logged with a timestamp. GMs can answer "when exactly did this guild lose money?" or "what changed on this player's missions in the last hour?" Currently each save overwrites the previous file with no change history.
• LAS investigation: Structured queries across delta logs by entity ID, time range, column path, or string search.
• Incremental I/O: Only changed columns written each tick, rather than re-serializing entire character blobs periodically.

Reconnect delta logging: Rewire the commented-out logging code in pd_lib.cpp (lines 620-703) to use BsiGlobal or the shard-dependent Bsi instead of the removed _PDBsi. Since BS is now domain-wide with per-shard subdirectories via SaveFilesDirectory, a dedicated PD backup pair is no longer needed.

Enable PDS server mode: Call usePDS() from EGS/IOS init (gated by a config variable) and configure a PDS service instance at the domain level.

Continue the migration: Move characters, items, and inventories to .pds schemas. The pd_parser tool and schema language are ready — the work is in writing the .pds schemas and replacing the manual PERSISTENT_DATA macro blocks.

• Server Reference — All shard services
• PDSS — PD Support Service — Batch analytics that scans PDR character saves
• Backup Service — The BS-based persistence path