ECR Architecture - worldscapes/engine GitHub Wiki

General

Entity-Component-Rule architecture is pattern that is used for organizing world state and it's updates.

How is ECR related to Entity-Component-System pattern?

The idea is that Systems are just changes and conditions on which those changes happen. Why do we need to couple them then?

We split Systems into smaller parts which describe one change at a time. Those changes are separate world simulation Rules. More to it, we split each Rule condition and effect. This is important because Engine can use conditions for further optimizations.

What this pattern gives us:

  • Improved extensibility
  • Space for optimization on large sets of objects (though little performance overhead because of additional function calls)
  • Improved testability
  • Ability to enable / disable features one by one when needed
  • And most importantly, clearer picture of simulation logic

Also, because of how JS works, we give up memory management benefits of ECS. Primary goal of Engine is to be extensible, it's not intended to be used in performance-heavy projects.

ECR pattern is needed to organize data and logic. It describes world state as structures (plain objects) that we call Components. Each Component belongs to some Entity, which is basically representation of any game world entity. Some data in scene does not belong to Entities, so it's held in Resources. Simulation is done using Rules. Each Rule describes Entities and Resources it needs as input, condition which defines when Rule should be applied and logic that generates changes. All changes are done using Events, which are returned from a Rule.

Entity - numeric ID which represents game world entity and groups components related to it. Id is needed to refer entities.
Component - plain structure that contains data of game world entity. Rule - structure that describes how game world should change. Resource - singleton object that holds some data and can be requested by system if needed.

Generally, user extends game this way:

  • Create / change components to describe game objects in more wide way, add some additional properties to them
  • Create / change rules to define additional game world logic

Entities can be initialized as pre-defined Archetypes or created by combining Components.

ECR Benefits:

  • Separates logic from state
  • Favors composition which is good for extensibility
  • Makes state easily serializable
  • Logic is easily testable because rules are pure functions
  • Rules do not explicitly depend on each other, allowing easy logic extension
  • Simplifies future implementation of multithreading
  • Creates space for optimization bacause of how rules work

ECR Drawbacks:

  • Components need to be designed carefully, as rules depend on their structure and will need to be changed when components change
  • Order of rule execution can change overall behaviour and need to be handled by engine user

Resources

Resource is structure that is not related to any entities in scene. Each resource has it's unique name and contains data. Data can be some general game information, structure with events, etc.

Resources allow simulation to store and use additional data without losing benefit of pure Rules. You can think of them as singleton components that can be requested directly.

Each Rule can request it's own unique local resource that can be used as cache.

Resource types

There are Resources that are updated for different reasons and serve different role. In this case they can be divided into several types to be conviniently handled by Simulation. It's important that from Store perspective there are all the same and can be contained the same way.

  • Local resource - available only for given Rule

  • Shared resource - shared between several Rules to allow them to exchange some information and save-up performance (like sharing collisions)

  • Managed resource - managed by Simulation and updated periodically. Can be used to provide access to some external state (like player input)

Shared and Managed resources need to be cleaned each frame automatically.

Store

Store is API object that decides how state is stored and querried. It's needed to provide additional abstraction layer over state structure, so we can change it while Commands and Queries are unchanged.

It's done as abstract class and can have different implementations to optimize performance for specific game needs.

Store implements queries using Observer pattern, which allows Store to contain and update query result when needed using custom algorithms while also allowing Simulation to read it at any time.

Since store just provides API and commands are not aware of it's structure, Client and Server can have different Store implementations. It gives additional flexibility for performance optimization.

Rules

Rule - structure that describe change and conditions when this change should happen. Rule iterates over components each tick and updates them accordingly.

Rule structure:

  1. Query description - object that describes (by entity structure) which Entites are required by rule
  2. Condition - function that checks (by component state) if Entity should be affected by Rule
  3. Body - funnction that takes selected Entity components and returns their update (Original components are never changed directly)

This way, we could for example enable / disable rules on by one. It would help to keep rules as pure functions, which can be especially great in making tests and logic organization.

Important thing that rules should depend on query protocol and not on exact querying implementation. This will allow engine to use different state implementations, therefore allow fast prototyping with space for optimization in future.

Condition can be checked at the moment of entity change and not every tick. This allows to save up a lot of processing and get great performance boost.

Order of Rules can play large role. To minimize that, user should think of input data itself when creating Rules and not about where the data came from.

Some rules depend on some special conditions, like damage happens only after a collision. To handle such situation Resources can be used to store additional information.

As result Rule should return list of Events that Engine will use to create side-effects (update data).

Thoughts on Rule Sets

If game has too many rules, pipeline can become crowded. To solve this, we can group those rules as RuleSets, which are basically array of rules that are related. It will make it easiler to organize logic

Rules can be grouped into role sets for several reasons:

  1. Improved semantic structure
  2. Easier rule management

Rule Side-effects

For side-effects like changing world state rules use Events. This allows Rules to do changes while staying completely pure.

Events describe side-effect which rule wants to do. For example:

  • Create entity, delete entity
  • Add, update, remove component
  • Add, update, remove resource
  • Save game snapshot, load game snapshot

Queries

Rules need some mechanism to describe set of data needed for processing. Data is searched by engine, but it needs enough information to find and optimize data set accordingly. Rules use queries to provide this information.

Rules can need to query Resources or Components, but their lookup should be implemented separately since those can strongly differ.

Information includes two parts: data structure and data purpose. Data structure is needed to find corresponding entities or resources. Data purpose is needed for organizing data flow and optimization.

While data structure should be known by storage, data purpose should only be known in simulation to avoid storage dependency on data processing.

After Engine knows which Entities and Components are needed for Rule it can cleverly structure data and do things like eager condition filtering.

Querry can influence which commands can be returned by Rule since some operations can require Write request.

Potentially there are several types of Querries (RuleQuerries, DataQuerries, StoreQuerries). All of them are similar by structure, but differ by Purposes. Engine's archtecture should allow to easily reuse some basis while maintaining their difference.

Data structure

By data structure we mean different things in case of resources and entities.

Data structure for Resources

Since resources are plain objects we request them by name.

Data structure for Entities

Entities are composed from Components. To define Entity we define list of Components it should have or not have.

  1. Request entity to HAVE component A (It should exists but won't be passed to Rule body or conditions)
  2. Request entity to NOT HAVE component B

Data purpose

Purposes of requesting data can be:

  1. CHECK (it will be checked by Rule condition, but not needed in rule itself)
  2. READ (it will be needed in rule by the time of change)
  3. WRITE (rule will return command to update this component)

It relates both to Entities and Resources. If purpose is not declared, data structure will be checked, but not passed to rule.

Since Entity purposes require Component to be present, we can translate them all to HAVE before passing to storage.

Problem:
Not only Components or Resources can be requested for WRITE, but also Entities or actually any data that is requested. Current mechanism does not allow to restrict Entity WRITEs.

Condition filtering

After Engine picked Entity lists it checks conditions on every entity to filter them. It can give large performance boost (reduces number of combinations in next step), but is not required for user to define those conditions.

If Rule requested several lists with different entities, each list is filtered separately.

Combination

After condition filtering we combine Entities. They are passed into combination function which restructures them (removes not needed data, makes pairs or just does some preprocessing which involves several Entity types).

Combination function returns array of values which will be passed into Rule body itself.

Combination function needed reduce number of resulting combinations which are processed in Rule and also allow user to do more complex combining logic when needed. For example, it can help when we need to process one entity list, but form pairs inside of it. You can think of it as that we do iteration outside of actual rule to check conditions, and later we just call our rule for different entities.

Since we do it outside of Rule, Engine can get more information before rule processing to do additional optimizations.

Serialization

Some data needs to be transferred from server ECR to client ECR. For this purpose we need to keep it serializable. This includes:

  • Entities
  • Resources
  • Components
  • Commands

Things that are not meant to be serializable:

  • Queries (including Requests and Selectors)
  • Rules (they contain logic)

Optimization

Change detection

Since all rules do changes by Commands and not directly, we have list of change Commands as result of simulation. This allows to avoid explicit change detection run and get little performance boost.

Order of execution

Order of execution is defined strictly by user, but can be optimized using multithreading. It's possible because Engine knows data used by rules.

Because Rule condition is separate and pure, it can be optimized additionaly by checking it eagerly on parameters update. This can include Resource or Component. If we depend on side effects additionaly (generate random number, take data from some other source) it can break the workflow, so rule condition should be marked as impure.

Important question here is: Should rules change state right after execution or only after entire tick?

External parallelisation

If rules process different components, they can be processed by different threads because they do not intersect.

If several rules process same components, second rule can start processing as soon as first rule processed them. Important: This cannot work if rule queries one entity with many (time entity and npc entities). Since time component can be changed as well, it blocks entire parallelisation. To solve that, we need to designate components to be read and to be changed.

Internal parallelisation

If one rule processes a lot of components, it can be parellilised to process them in several threads.

Multithreading

It's possible to use Web Workers for multithreading in Javascript. Because Rules are pure functions, they can be easily delegated to another threads.

Engine can allow customization of load balancing so user can optimize performance, but generally user should not be involved into this.

For load balancing engine should have enough information.

  • Components handled by given rule
  • Execution order of all rules

Memory Optimization deprecated

It's important to note, that JS doesn't allocate memory linearly, so ECS loses it's benefit of linear data processing.

To solve this problem, component memory allocation can be delegated to Engine which can do custom linear data allocation using WebAssembly.

Though, to allow immutability, Rules can create updated component instances as regular objects so they replace current component state (memory region) after Rule execution tick.

It's pretty expensive to pass data from WASM world to JS world, so all calculations on components should be performed on WASM side and data is taken to JS just on final stage before sending to client (on server-side) or presentation (on client side). More to it, only updated state should be passed to JS to get stable performance.

To allow all of that, Rules should be written as pure WASM functions and components should be created on WASM side. It can be done using AssemblyScript functions and classes.

To prove performance gain proof of concept needs to be done.

Result:
AssemblyScript not proved to be much faster. It can be faster then Javascript, but overall on large repeating data calculation JS outperforms AS. But JS itself is much less stable in performance, it can vary from test to test by 50% sometimes. AS allows to achieve much higher stability in performance and do low level optimizations, but it makes user to care about more details. Also, JS performance can differ several times even with small unobvious code changes.

If used with Multithreading, it's maybe possible to raise performance by parallel Rule executions

Interrule communication deprecated

One of main problems is that Rules can influence each others behavior. So, for example, when collision of bullet and character happens, damage rule should apply damage to it. This process should be coordinated and strictly organized to describe those relations explicitly and allow easy game extention.

Possible ways to organize it:

  1. Interrule event mechanism

Pros:

  • It can be described explicitly who generates which events and who handles them
  • New events can be added easily

Cons:

  • Events are not in the world state, so they complicate serialization (We most likely only serialize state in beetween Rule ticks, so those events should not impact)
  • Rules should be pure, so they can only handle events as input and generate them as output, otherwise it can create sideeffects -> break purity profits
  1. Additional fields in components describing their state just in current frame

  2. Additional entity that can be accessed by Rules to read events (see Additional entity access)

  3. (Chosen solution) Each Rule should check it's conditions separately and not depend on other Rules in any way. If many Rules need to do the same check for the same components, they should use shared implementation which is done as separate function. This way we have much more independent rules and it's easier to think about them this way. It can be not so performant, but further optimiaztions as memoizing can be possible.

System Example

// Container function is impure and can 
function bulidDamageSystem(

    // Additional system configuration can be passed to build function
    additionalConfig: { enableLogging?: boolean }

) {

    // Creating local cache
    let cache = {};

    // Create systemImplementationFunction that will be used by WorldLayer
    return (requestedCompoents, relatedEvents) => {
        const result = damageSystemImpl({
            customArguments: { cache }, 
            engineParameters: {
                requestedCompoents
            }, 
            relatedEvents
        });
        
        // Updating cache if changed
        if (updatedAdditionalArguments?.cache) {
            cache = updatedAdditionalArguments.cache;
        }

        return result.updatedComponents;
    }
}

// Implementation function is pure and only depends on it's arguments
function damageSystemImpl({
    
    // System can have custom arguments or settings
    customArguments: {
        cache: any
    }
    
    // Engine-passed parametes
    engineParameters: {

        engineTime: number,

        // Components are provided by World layer depending on what components are listed
        // in system dependency list
        requestedComponents: { 
            health: Component[],
            weapon: Component[] 
        },

        // Events are provided by World layer depending on what event types system subscribed to
        relatedEvents: Event[] 
    }
}) {

    const attackEvents = relatedEvents.filter(event => event.type === ATTACK_EVENT_TYPE);

    // Handle all attack events
    ...

    return {
        customArguments: customArguments,
        changedComponents: {
            health: copiedAndChangedHealthComponents,
            weapon: copiedAndChangedWeaponComponents,
        }
    }
}
⚠️ **GitHub.com Fallback** ⚠️