content manager extending - GoldhawkInteractive/X2-Modding GitHub Wiki

ContentManager – Architecture & Extensibility

Overview: The ContentManager is the game engine’s asset loading and caching hub. It orchestrates how game content packs (both core and mods) are loaded, processed, and saved, providing a unified pipeline for assets. Modders with access to the source (via add-on DLLs or Harmony patches) can extend this pipeline by adding custom loaders and processors, without modifying base game code. Below we break down the ContentManager’s design, focusing on extensible components (loaders, processors, resolvers) and how a modder can plug into them. We’ll also cover the asset load/save workflow (with a diagram), debugging tips, and notes on performance.

ContentManager Architecture & Workflow

Core Responsibilities: The ContentManager handles locating asset files, reading them from different sources (disk files, memory, or Unity asset bundles), converting raw data into game objects, and caching them for reuse. It maintains an Asset Cache mapping each asset’s descriptor to its loaded object and status. Content is organized into content packs (the base game and each mod) each with a manifest and possibly asset bundles. During initialization, the ContentManager loads each pack’s manifest and asset-bundle manifest, so it knows what assets exist in that pack.

Asset Descriptors: Assets are identified by Descriptor objects that encode the asset’s type, location (content pack, path), and source (FileSystem, AssetBundle, or Memory). The ContentManager’s resolver will map a Descriptor to an actual file or memory handle (FileHandle) before loading. Descriptors can be created via the API – for example, AssetReference.Asset<T>(name, …) constructs an AssetDescriptor for type T at a given path. Internally, there is also a parsing mechanism to infer asset type from file path using configured type mappings in the AssetConfiguration. Modders should ensure new asset files either follow existing directory conventions or use explicit type hints when constructing descriptors (to avoid resolution errors).

Asset Loading Workflow: When you request an asset, e.g. ContentManager.Load(descriptor), the process flows through several stages. Figure 1 illustrates this pipeline:

Figure 1: Simplified asset loading flow in ContentManager. The system resolves the asset’s descriptor, selects an appropriate loader, reads raw data (from file, bundle, or memory), then applies any post-processors to produce the final object, which is cached and returned.

  1. Resolve Descriptor: The ContentManager first resolves the descriptor to a file handle. This ensures the path is correct and the file exists in the expected content pack. If resolution fails or file is missing, the load is aborted with an error. The resolver may also apply redirects (if configured) to handle moved/renamed assets – for example, MovedAssetRedirector can point an old path to a new one. Modders typically won’t need custom resolvers unless introducing entirely new descriptor types or storage schemes.

  2. Check Cache: Once resolved, the system checks if the asset is already loaded. If so, and if it meets the required dependency level, it will skip duplicate loading. (Assets can be loaded with or without all dependencies, as controlled by a loading rule flag, and the ContentManager ensures an asset needing full dependencies is fully reloaded if previously loaded without them.) Cached assets are returned immediately to avoid redundant work.

  3. Select Loader: If not cached, ContentManager chooses an appropriate Loader. Loaders are stateless classes (implementing IContentManager.ILoader) marked with [AssetLoader] that know how to read a particular type of asset from a source. The manager scans its registered loaders and picks the first that CanLoad the given descriptor, considering loader priority. For example, it has loaders for plain files, asset bundles, memory assets, Unity scenes, etc., each with logic to handle certain descriptor types or sources.

  4. Load Raw Data: The selected ILoader executes and reads the asset data. This could be synchronous or asynchronous – some loaders implement IAsyncLoader for background loading. For instance:

    • The FileSystemLoader reads a file from disk and returns either a byte[] or string depending on extension. By default, text files (like .json, .txt) yield a string, while binary files yield byte data.

    • The AssetBundleLoader (for Unity .assetbundle files) loads the bundle via Unity’s API. It caches loaded AssetBundle objects and can share in-progress requests.

    • Other loaders exist for special cases (e.g. UnitySceneLoader loads a scene asset, MemoryLoader handles assets already in memory, etc.).

At this stage, the asset is in a raw form (bytes, string, Unity Object, etc.) which may not yet be the final game object. The ContentManager wraps the result and moves to post-processing.

  1. Post-Processors Chain: After raw load, the ContentManager applies Load Post-Processors – stateless transformers marked with [LoadPostProcessor]. Each processor checks if it can handle the asset (via CanProcess) and, if so, processes it, possibly outputting a new object. The manager runs these in sequence (by ascending priority) until the asset’s type matches the requested descriptor’s Type, or no more processors apply. This is where most custom data parsing or object construction happens. Key points:

    • Processors are discovered and registered at ContentManager initialization. The code automatically finds all classes with the LoadPostProcessor attribute and adds them to a list. Modders can simply drop in a new class (in their DLL) with this attribute – once the mod’s assembly is loaded, the ContentManager will include it if the assembly was present before initialization. (Note: Mod assemblies are usually loaded as content during pack loading; we discuss implications below.)

    • Example – JSON Parsing: The game stores a lot of data in JSON. A built-in processor, JsonParserProcessor, catches string or byte[] assets whose target Type is a complex object (excluding certain cases). It parses the JSON text into an FS data structure (fsData) using the FullSerializer library, then wraps it in a JsonData object. Another processor, VersionedJsonDataUplifter, may run next to upgrade old version data to the current format (using the Uplift system). Finally, the pipeline must deserialize the JsonData into the actual game object. This is handled by a processor (not explicitly named here) that uses the ContentManager’s configured serializers. The ContentManager maintains one or more fsSerializer instances for different version ranges and a “Main” serializer for the latest version. When it encounters JsonData with an embedded object type, it uses the appropriate serializer to create the C# object. The end result is the fully constructed game asset (e.g. a ContentPackManifest object, a game config class, etc.) ready to use.

    • Example – Assembly Loading: Mod DLLs themselves are loaded via a post-processor. When a mod’s content pack includes a .dll file, the initial loader likely reads it as byte[]. The AssemblyLoadProcessor then detects if the asset descriptor’s Type is System.Reflection.Assembly with byte[] data. It will attempt to also load the matching .pdb (debug symbols) if present as a dependency, then call Assembly.Load(bytes, pdbBytes) to load the assembly into the runtime. This means the mod’s code becomes active in the game process. After this point, any classes in that assembly tagged as loaders or processors could participate in further content loading (though timing matters, as noted below). Important: Modders should ensure their assemblies are either loaded early (e.g. by having the content pack manifest list an asset that triggers the assembly load) so that their custom processors are registered in time for relevant assets.

    • Example – Custom Data Format: Modders can introduce new data formats by writing a custom LoadPostProcessor. For instance, if a mod uses an Ink story file (text script) to define dialogue, they might include .ink files. The base game has StringToInkStoryAssetConverter as a post-processor that triggers when an asset is a raw string and the target Type is InkStoryAsset. It compiles the ink script text into a runtime story object using the Ink engine, populates an InkStoryAsset object, and returns it. This illustrates how modders can convert custom formats: you could mark a processor for your own asset class, have it read or compile the raw data, and output the final object. The ContentManager will then cache that object for use in game.

  2. Finalize & Cache: Once the asset’s type matches the descriptor’s expected Type, the ContentManager deems it fully loaded. It then stores the object in the asset cache and marks it as Loaded (and with full dependencies loaded, if applicable). At this point, an AssetLoaded event is fired for any listeners. The loaded asset can now be retrieved (for example via ContentManager.Get(descriptor) or through an AssetReference<T> wrapper). If the load process failed at any step, the state is set to Failure and errors are logged.

Asset Unloading: ContentManager also tracks assets for unloading. Mod content or large assets may need explicit unloading to free memory. Unloading also uses an extensible system of Unloaders (classes with [AssetUnloader]) to handle cleanup if simply removing from memory isn’t enough. The default Unloader for Unity objects, for example, will call UnityEngine.Object.Destroy on a loaded asset if needed. The manager chooses an unloader by CanUnload similarly to loaders. When ContentManager.Unload(descriptor) is called, it ensures the asset is loaded and not already unloading, then queues an unload task via the found unloader. After completion, the cache is updated to Unloaded state and an AssetUnloaded event triggers. Modders rarely need custom unloaders unless dealing with non-standard resources (e.g., if a mod opens file handles or network connections that need closing – then a custom IUnloader could be warranted).

Save Workflow: In addition to loading, the ContentManager handles saving assets (e.g. saving game states or writing mod-modified files). The Save pipeline is essentially the reverse of load, using Save Pre-Processors (tagged [SavePreProcessor]) to transform in-memory objects into a storable form. When ContentManager.Save(object, fileDescriptor) is called, it resolves the FileDescriptor (target path) and checks if the file already exists (honoring the overwrite flag). It then clears any previous save processors applied and begins the processing chain. The chain works similarly to load: the manager finds a SavePreProcessor whose CanProcess is true for the object and descriptor, runs it to get a result, and repeats if the result is not yet a final file write. Some built-in Save processors:

  • Object Serialization: For most data, the first step is converting a game object into a serializable form. The VersionedAssetSerializer and related processors wrap the object along with a version tag. For example, ObjectToVersionedAssetConverter might package an object into a VersionedAsset (with current version) so that it can later be recognized in JSON. Then a serializer step produces fsData (essentially JSON tree) from the object using FullSerializer.

  • JSON String Writer: The JsonStringPrinter takes an fsData (FullSerializer data) and converts it to a JSON string. It checks optional parameters for pretty-print vs compressed and returns the JSON text.

  • File Writer: Finally, the FileHandleWriter catches string or byte[] outputs and writes them to disk via the descriptor’s FileHandle. This processor has the highest priority (0) so it runs last on data that is fundamentally file content. It calls WriteString or WriteBytes on the FileHandle, actually creating/updating the file, and returns AssetState.Loaded to signal completion. Returning an AssetState from a Save processor is a special convention meaning “the save is finalized” – the SaveProcessTask will stop chaining further processors when it sees an AssetState result.

  • Special Cases: Other SavePreProcessors handle Unity objects or Scenes. For instance, UnitySceneSaver detects if the object is a Unity Scene and uses UnityEditor APIs to save the scene to the given path. It directly returns AssetState.Loaded on success (meaning the scene file was written), so no further processing is needed. There’s also an ArtitasSceneSaver and perhaps others for saving custom composite assets.

After the processing chain completes, the ContentManager updates its cache if instructed to cache the asset, and may fire events or handle callbacks. Notably, if the final result was AssetState.Loaded, the save is considered successful. If no processor was able to handle the object (chain ended with a non-matching type and returned null task), it logs an error “no Save Pre-Processor for this type”. Modders who introduce new data types that need custom save logic (e.g. a mod-added object in game state that default serialization can’t handle) can implement a SavePreProcessor to manage it. Often, however, as long as your new classes are serializable via FullSerializer (the system will serialize fields by default, or you can write an fsConverter for complex cases), you might not need a custom save step – it would be covered by the generic JSON serialization path.

Extending ContentManager: Loaders, Processors, Resolvers

One of the ContentManager’s strengths is its strategy pattern design – it uses interfaces and attributes to allow adding new behaviors. Without modifying the ContentManager core, modders can extend asset handling by providing new implementations:

  • Asset Loaders (ILoader/IAsyncLoader): If your mod needs to load assets from a new source or format that isn’t covered by existing loaders, you can create a class with [AssetLoader] and implement ILoader. For example, a mod could implement a CsvFileLoader if the game doesn’t support CSV files out of the box. In CanLoad, you check for your file extension or descriptor type; in Load, you read and return raw data (perhaps as a string or parsed structure). Assign a Priority – lower numbers mean higher priority, so set a small number if you want to override a default loader. The ContentManager at init registers all such classes, so your loader will be considered whenever an asset is loaded. Note: Ensure the loader’s assembly (your mod DLL) is loaded before the ContentManager’s Setup() runs, otherwise it won’t see your loader. Because mod assemblies are themselves loaded by ContentManager during content pack loading, your loader might not register on first load of assets. As a workaround, if needed, you could force a second initialization or manually register (not trivial). In practice, many mods may not need custom ILoader since the file reading step is simple; a LoadPostProcessor is often sufficient (reading via FileSystemLoader then parsing in the processor).

  • Load Post-Processors (IProcessor, post-load): These are often the most useful extension point for modders. By creating a class with [LoadPostProcessor] that implements IProcessor, you can inject custom transformation logic after an asset is loaded in raw form. We saw examples like the InkStory compiler and assembly loader above. Another example: if a mod introduces a new image format that Unity doesn’t support natively, you could let the FileSystemLoader load the image bytes, then write a post-processor that recognizes your image type (by descriptor or file extension) and converts the byte[] into a Unity Texture2D. The CanProcess method should check the incoming asset type and descriptor to decide if it can handle it. Use Priority to order your processor relative to others – e.g., to run before the general JSON parser, use a smaller priority number (the JSON parser uses 5.0 by default, so a priority 4.0 would run earlier). The ContentManager ensures no two processors handle the same asset twice by tracking which ones were applied. If your processor needs the asset’s dependencies loaded first (e.g. a processor that requires another asset), you can override GetDependencies to return descriptor(s) that should be loaded beforehand. The loading system will then load those dependencies (honoring either sync or async flow) before calling your Process function. This is powerful: you can declare, for instance, “when processing X, also ensure Y is loaded.” We saw the AssemblyLoadProcessor use this to load a .pdb file as a dependency for a DLL. Modders can similarly chain asset loads if needed.

  • Save Pre-Processors (IProcessor, pre-save): Modders can also define custom save logic via [SavePreProcessor]. For instance, if a mod maintains a special data structure in memory that the default JSON serializer would bloat or fail on, a SavePreProcessor can intercept that object type and convert it to a simpler form (or even directly write it out). The interface and registration work analogously to Load processors. One should return either a transformed object (to pass to the next stage) or AssetState if the save is fully handled. For example, a mod could save a custom binary format: the processor could take the mod’s object, serialize it to a byte array, then return that byte[]; the next built-in processor (FileHandleWriter) would catch the byte[] and write it to disk. Ensure that your processor’s CanProcess is specific enough so it doesn’t steal objects unintentionally. Also, if your processor chain produces an intermediate result that the default pipeline doesn’t understand, you must also supply the subsequent processor. (E.g., if you output a custom MySerializedData object, nothing in base game knows how to write that – you’d need another SavePreProcessor or modify yours to output a standard type like string/byte[]/AssetState.)

  • Resolvers (IResolver): In cases where mod content has unconventional storage, you could extend resolution. A class with [Resolver] can be registered to handle a custom Descriptor subclass. The resolver’s job is to set the Descriptor’s FileHandle (with correct path or data) and return a success/failure Result. The ContentManager’s Setup() collects all IResolver similarly. An example from base game is AssetBundleBuildManifestResolver, which knows how to construct the path to the manifest.json of an asset bundle given the content pack path. Modders might rarely need to do this, but one scenario could be if a mod wants to define virtual or generated content on the fly – you could create a Descriptor that doesn’t correspond to a real file, and a Resolver that intercepts it and supplies data from code or memory. Keep in mind the resolver is called during ContentManager.Resolve(), which happens each time before loading or checking existence.

  • Redirectors: Not an interface per se, but the AssetConfiguration supports redirect rules (through a dictionary of patterns to redirectors). These can remap asset addresses transparently. For example, if a game update moves an asset’s path, a redirector can ensure old saved descriptors still find the new location. Mods could possibly add redirect rules via Harmony or config if they, say, replace an asset with another location. However, redirectors are configured in AssetConfiguration (which mod code might access via AssetReference.ContentManager.GetConfig()). It’s an advanced use-case; most mods won’t modify this at runtime, but it’s good to know the system exists. The default MovedAssetRedirector uses a list of mappings (old path → new path) to adjust descriptors on resolve. If a mod makes sweeping changes to content paths, adding redirect entries can help maintain compatibility with references in existing saves or code.

Mod Assembly Loading Caveat: As noted, mod code is loaded by ContentManager itself (via AssemblyLoadProcessor). This means your custom loaders/processors defined in the mod assembly will only become available after that assembly asset is loaded. Practically, this often isn’t a big problem: your mod’s content pack manifest (a JSON) is loaded first, then the asset bundle manifest, and then any assets. If you have defined your assembly as an asset in the pack (usually placed in a folder where ContentManager picks it up), the assembly will be loaded early on, before most other assets in that pack. The base game might explicitly load mod assemblies after discovering mods – for example, the code in ModSettings enumerates mods, then calls ContentManager.Load on each mod’s ContentPackManifest descriptor. During that process, the mod’s DLL (if listed in the manifest or present in the content pack) gets loaded. Once AssetReference.ContentManager has your assembly, it will register any [AssetLoader], [LoadPostProcessor], etc. contained in it. If some mod asset needed one of your processors very early, there’s a slight chance of a race (if your processor isn’t registered yet). To avoid issues, a mod can structure its content so that the DLL is one of the first things loaded (which is typically the case as described). In summary, the ContentManager will automatically integrate mod extensions as long as the mod’s code is loaded in time. No separate registration calls are needed – just attribute your classes properly.

Debugging & Testing Tips

Working with the Content pipeline can be complex. Here are some tips for diagnosing issues:

  • Enable Logging: The ContentManager uses a logging system (log4net). At runtime with debugging enabled (e.g. launch with -showDebugUI in Xenonauts 2), you can see ContentManager logs. It logs info-level messages for each load/unload, and error messages when something fails. For example, if an asset can’t be resolved or doesn’t exist, you’ll see a log like “LOAD FAIL: … asset doesn’t exist or could not be resolved”. If a post-processor chain fails to produce the target type, you get a “PROCESS FAIL: Failed to find a post processor to process asset from X into type Y” message. The system also collates nested errors: if an asset fails due to a dependency failing, it will log a summary tracing back through the chain. Reading these logs often pinpoints which file or which processor caused a problem. As a modder, you can also add your own logging (e.g., use GameLogger.GetLogger in your code) to output debug info during your processor’s execution.

  • AssetLoaded Event: If you need to inspect or manipulate an asset right after loading (perhaps for debugging or custom initialization), you can subscribe to ContentManager.AssetLoaded event with ContentManager.AssetLoad(callback). This event provides the Descriptor, final AssetState, and object each time an asset finishes loading. Similarly, AssetUnloaded event exists for unloads. This can help verify that your custom content was loaded as expected, or to trigger post-load adjustments.

  • Using AssetReference: The AssetReference<T> wrapper can simplify mod code and ensure assets are loaded. For example, AssetReference.ContentManager gives the active manager instance, and you can create references like AssetReference.Asset<SomeType>("path/to/asset"). AssetReference will handle calling Load under the hood when you access the asset. It’s useful for grabbing mod assets by type. The AssetReference API also has directory search abilities (Directory<T> to list assets of type T in a folder) which can be handy if your mod organizes assets in folders and you want to enumerate them.

  • Step-through in Editor: If you are developing with the game’s Unity project (if available) or have the source running with a debugger, you can set breakpoints in your custom loader/processor code. The ContentManager calls those just like any other code. Keep in mind some loaders run on a separate thread (IAsyncLoader) – Unity’s main thread context is required for certain operations (like actually instantiating UnityEngine objects or AssetBundle loading). The AssetBundleLoader, for instance, asserts that it runs on main thread. If you implement an async loader or processor, be cautious about thread usage. For debugging, you might temporarily force synchronous loading (the ContentManager’s loadingRule can be set to “Strict” which might avoid background threading) or insert logging instead of breakpoints in multithreaded code.

  • Performance Monitoring: The ContentManager includes a Profiler utility (ContentManagerProfiler). When enabled (contentManager.Profiler.enabled), it tracks the time spent in various sections and even integrates with FullSerializer profiling hooks. After loading, it can log a summary of timing. If you suspect your mod’s loader/processor is slow, you can measure it. Also note the ContentManager uses a frame time budget for processing tasks (by default, a number of milliseconds per frame is allotted to content loading to avoid frame hitches). If a lot of heavy tasks are queued (e.g. loading many large assets), and some are synchronous, the manager may defer some until the next frame once the budget is exceeded. In extreme cases (dependencies taking too long), it has a safety timer: if no progress is made for 10 seconds or 3 frames, it throws an exception to break a stuck loading state. This typically wouldn’t trigger unless something deadlocked or kept failing to load dependencies. As a modder, just be aware that extremely heavy processing in a single frame might be split – if you need to load a huge amount of data, consider using async processing so as not to stall the game. For example, you could perform lengthy parsing in a background thread and only finalize on the main thread.

  • Testing Save/Load: If your mod influences the save game (for instance, adding new fields or custom data in the world state), do test the save and reload cycle thoroughly. Ensure that your SavePreProcessor and LoadPostProcessor are inverses: the data you save can be read back correctly. Use a small test scenario, save the game, and open the save file (often JSON) to verify your mod’s data is present and well-formed. Then reload with the mod active to see if everything restores. The versioning system (Uplift) can be leveraged if you plan to change your data format in future mod updates – you could register an IUpliftOperation to upgrade old saved data to new structure (the base game does this for its own data evolution). This is advanced, but worth noting if your mod will evolve over time.

Summary

The ContentManager is essentially a pluggable content pipeline. It loads content packs (including mods) and processes assets through a series of extensible steps: resolving where to get the asset, loading raw data via loaders, and post-processing that data into game-ready objects, plus the analogous steps for saving content. For modders, the key extensibility points are implementing new Loaders or Processors with the proper attribute – the game will incorporate these automatically, allowing you to support new asset types or behaviors. Modders might create custom loaders for new file sources, but more commonly they will write LoadPostProcessors to parse new file formats or inject custom logic, and SavePreProcessors if custom data needs special handling when writing to disk.

The system’s design ensures mods can add content and processing without altering the base game’s code: you drop in a new DLL and content files, and the ContentManager’s discovery mechanism does the rest. With this power comes complexity – it’s important to carefully manage asset dependencies, use correct priorities, and thoroughly test the asset pipeline for your mod. By following the patterns in the game’s code (as shown in the cited examples) and using the debugging techniques above, you can reliably extend the ContentManager to make Xenonauts 2 (or any game using this engine) load and understand your custom content.

Finally, always keep an eye on performance. The ContentManager will cache and skip loading assets it already has, so utilize that (don’t needlessly reload your assets). If your mod dynamically generates or modifies assets at runtime, clean up via unloaders or other means if appropriate. Thanks to the ContentManager’s flexible architecture, even ambitious mods can integrate deeply – from new media formats to entirely new gameplay data – as long as they adhere to the content pipeline.

⚠️ **GitHub.com Fallback** ⚠️