Project Architecture Overview - trueagi-io/metta-wam GitHub Wiki

To better understand how these components interact within the MeTTa system, here's a high-level flow of operations:

1. Startup and Initialization

   • The system begins by executing init.default.metta, which sets up the initial environment.
   • Necessary modules and core libraries (corelib.metta, metta_corelib.pl) are loaded using metta_loader.pl.

2. User Interaction via REPL or Script Execution

   • If the user engages with MeTTa interactively, metta_repl.pl and repl.default.metta handle inputs and outputs.
   • For script execution, the interpreter processes MeTTa files directly.

3. Parsing Source Code

   • Source code is fed into metta_parser.pl, which parses it into internal representations.
   • metta_convert.pl may further process these representations for compatibility with the evaluator or compiler.

4. Compilation or Interpretation

   • Compilation Path:
     • metta_compiler.pl compiles the code, using templates from metta_comp_templates.pl.
   • Interpretation Path:
     • metta_interp.pl uses metta_eval.pl to evaluate the parsed code directly.

5. Runtime Execution

   • The runtime environment (metta_space.pl, metta_subst.pl, metta_types.pl) manages variable scopes, type checking, and pattern matching during execution.
   • Concurrency support is provided by metta_threads.pl if needed.

6. Utilizing Core Libraries and Extensions

   • Core functions from corelib.metta and metta_corelib.pl are available throughout execution.
   • Logic programming features are accessed via metta_ontology.pfc.pl, metta_pfc_base.pl, and metta_pfc_support.pl.
   • Python interoperability is handled by the metta_python suite of files.

7. Output and Feedback

   • Results and outputs are formatted by metta_printer.pl and displayed to the user.
   • Debugging information (if enabled) is provided by metta_debug.pl.

8. Persistence and State Management

   • metta_persists.pl allows the saving and loading of program states as needed.

9. Server Operations

   • If running in server mode, metta_server.pl manages client connections and request handling.

Below is a textual representation of the MeTTa system architecture, showing the relationships between components:

[ User Interface ]
       |
       V
[ REPL (metta_repl.pl) ] -- [ REPL Script (repl.default.metta) ]
       |
       V
[ Parser (metta_parser.pl) ] -- [ Converter (metta_convert.pl) ]
       |
       V
[ Interpreter (metta_interp.pl) ] -- [ Compiler (metta_compiler.pl) ]
       |                               |
       V                               V
[ Evaluator (metta_eval.pl) ]        [ Compilation Templates (metta_comp_templates.pl) ]
       |
       V
[ Runtime Environment ]
       |
       |-- [ Execution Space (metta_space.pl) ]
       |-- [ Substitution (metta_subst.pl) ]
       |-- [ Type System (metta_types.pl) ]
       |-- [ Concurrency (metta_threads.pl) ]
       |
[ Core Libraries ]
       |
       |-- [ corelib.metta ]
       |-- [ metta_corelib.pl ]
       |
[ Extensions and Integrations ]
       |
       |-- [ Logic Programming ]
       |     |-- [ metta_ontology.pfc.pl ]
       |     |-- [ metta_pfc_base.pl ]
       |     |-- [ metta_pfc_support.pl ]
       |
       |-- [ Python Integration ]
             |-- [ metta_python.pl ]
             |-- [ metta_python_override.py ]
             |-- [ metta_python_patcher.py ]
             |-- [ metta_python_proxy.py ]

In this system, the architecture revolves around a rule-based pattern matching engine that leverages MeTTa's meta-programming capabilities. The system is designed to evaluate expressions dynamically by matching patterns against defined rules. The rules dictate how specific operations, like arithmetic, are handled within the system.

Here’s a breakdown of the architecture and flow of how rule-based arithmetic, such as (add $a $b), is processed and evaluated in the system:

1. Atomspace (Blackboard):

   • Role: Atomspace serves as the knowledge base and storage mechanism for all code modules (rules, operations, and logic). In this context, it holds the rules for pattern matching and transformations, like the rule (= (add $a $b) (+ $a $b)).
   • Storage: Rules and expressions are stored as atoms in Atomspace. These atoms represent the symbolic expressions and patterns used in the system. For example, the rule (= (add $a $b) (+ $a $b)) would be represented as atoms that link the patterns and their transformations.

2. Rule-Based Pattern Matching Engine:

   • Role: This engine is the core processing unit that evaluates MeTTa expressions. It works by matching input expressions (e.g., (add 5 10)) against predefined rules stored in Atomspace.
   • Functionality: When an expression is submitted for evaluation, the engine:
     1. Searches Atomspace for matching patterns.
     2. Transforms the expression according to the matched rule (e.g., (add $a $b) becomes (+ $a $b)).
     3. Evaluates the transformed expression to produce a result.
   • Dynamic Behavior: This component is capable of dynamically creating and modifying rules during runtime, allowing for flexible and adaptive programming.

3. Dynamic Evaluation (MeTTa Interpreter):

   • Role: The interpreter is responsible for executing the transformed expressions after the rule-based engine has processed them. It recursively evaluates expressions that have been transformed (e.g., (+ 5 10)), calculating their values.
   • Interaction with Atomspace: The interpreter may retrieve additional rules or definitions from Atomspace during the evaluation process if needed (e.g., recursive function calls or additional pattern matches).

4. Module Manager:

   • Role: The module manager oversees the organization and management of code modules within Atomspace. These modules may contain arithmetic rules, logical rules, or even higher-level control structures. The manager is responsible for loading, linking, and organizing these modules in Atomspace.
   • Interaction with Subprocesses: In a more complex scenario, the module manager might also delegate certain tasks to subprocesses (e.g., Python or Scheme) when MeTTa modules define external dependencies or computations that need to be performed in other languages.

5. Subprocess Manager (Optional):

   • Role: While not necessary for basic rule-based arithmetic, in more complex systems, the subprocess manager would handle the interaction with external subprocesses (e.g., calling a Python subprocess for specialized computations). In the context of rule-based arithmetic, the subprocess manager would be activated when a MeTTa rule specifies that external code needs to be executed.

6. Data Conversion Layer:

   • Role: If subprocesses are used, the data conversion layer would handle translating the internal MeTTa expressions into forms that other subprocesses (like Python) can understand, and vice versa. For example, a rule might specify that a computation should be performed by an external Python subprocess, in which case the system would convert the MeTTa representation into Python code, execute it, and convert the results back into MeTTa atoms.

Let’s go through the system flow when an expression like (add 5 10) is evaluated:

1. Input Expression:

   • The user inputs the expression (add 5 10) for evaluation. This expression is represented internally as atoms in Atomspace.

2. Pattern Matching:

   • The pattern matching engine searches Atomspace for a rule that matches the structure of the expression. It finds the rule (= (add $a $b) (+ $a $b)).

3. Transformation:

   • The engine transforms the expression (add 5 10) by substituting 5 for $a and 10 for $b, resulting in the new expression (+ 5 10).

4. Evaluation:

   • The transformed expression (+ 5 10) is passed to the dynamic evaluation component (MeTTa interpreter), which evaluates the expression and calculates the result: 15.

5. Return Result:

   • The result 15 is returned as the output of the evaluation. This result may also be stored back into Atomspace for further use by other components or subsequent computations.

+-----------------+        
|    Atomspace    |     <- Stores rules, expressions, and evaluation results
| (Blackboard for |     
| Code and Rules) |     
+-----------------+        
        |                         
        v                         
+-------------------------+     
|  Rule-Based Pattern     |     <- Matches and transforms input expressions
|  Matching Engine        |     
+-------------------------+     
        |                         
        v                         
+-------------------------+     
|   Dynamic Evaluation    |     <- Evaluates transformed expressions (e.g., (+ 5 10))
|  (MeTTa Interpreter)    |     
+-------------------------+     
        |                         
        v                         
+-------------------------+     
|   Execution Results     |     <- Returns the evaluated result (e.g., 15)
|                         |     
+-------------------------+     

1. Atomspace as Central Knowledge Base:

   • Atomspace holds not only code (e.g., arithmetic rules) but also the logic for how expressions are transformed and evaluated. This allows the system to be highly flexible, as the rules and logic can be updated at runtime without changing the underlying system architecture.

2. Rule-Based Pattern Matching Engine:

   • The engine is responsible for transforming expressions based on the rules defined in Atomspace. For arithmetic, it matches expressions like (add $a $b) and replaces them with corresponding mathematical operations.

3. Dynamic Evaluation:

   • Once the expression has been transformed, the MeTTa interpreter evaluates it recursively. The interpreter’s role is to carry out the final computation, such as resolving (+ 5 10) to 15.

4. Module and Subprocess Management:

   • Although not needed for simple arithmetic, in a more complex system, the module manager would oversee the organization of different rule sets and subprocesses. When rules or code modules specify that external systems (like Python) need to be invoked, the subprocess manager would handle the orchestration.

5. Data Flow:

   • The data flow follows a clear path: from the initial expression, through rule transformation, evaluation, and finally to the output result. Atomspace plays a crucial role by allowing different components of the system to access and modify shared information, ensuring consistency across the system.

In this architecture:

• MeTTa enables meta-programming through dynamic pattern matching and rule-based transformations.
• Atomspace serves as the central blackboard where rules and expressions are stored and accessed by various components.
• The Rule-Based Pattern Matching Engine handles the transformation of input expressions based on predefined rules.
• The Dynamic Evaluation Component (MeTTa Interpreter) evaluates the transformed expressions and computes the results.
• The architecture is extensible to handle more complex interactions, such as integrating external subprocesses (like Python or Scheme) for specialized computations, managed by a module manager and subprocess manager.

The MeTTa language system is structured to facilitate ease of use, extensibility, and powerful features for logical reasoning and interoperability. Here's a brief recap:

• Front-End: Parses and prepares code for execution.
• Core Libraries: Provide essential functions and definitions.
• Interpreter/Compiler: Execute or compile code for running programs.
• Runtime Environment: Manages execution contexts and supports runtime features.
• REPL: Offers an interactive environment for code execution.
• Debugging/Testing: Tools for ensuring code correctness.
• Extensions: Enhances MeTTa with logic programming and Python interoperability.
• Server Support: Allows MeTTa to function as a networked service.
• Utilities: Additional support functions and compatibility layers.
• Initialization: Handles startup routines and environment setup.

Note: The descriptions and architecture are based on standard naming conventions and typical functionalities associated with similar files in programming language implementations. For precise implementation details, please refer to the actual source code and official documentation.

The other directories and files in the project provide additional functionality and support. Each top-level directory is expandable, and the link is provided outside the spoiler.

This detailed overview provides an in-depth look into the architecture and workflow of the Metta language processing system. Starting from the REPL interface, which allows users to interact with the interpreter and load files, the system efficiently manages code addition, parsing, forward chaining for logical inference, compilation, and execution. The inclusion of utilities and support modules enhances the system's capabilities, offering integration with external languages like Python and providing tools for debugging and testing.

Understanding each component's role and how they interact enhances appreciation of the system's workflow and capabilities. This comprehensive guide serves as a valuable resource for both users and developers navigating and utilizing the Metta language processing system.

For further clarification, refer to the docs directory or contact the development team.