Alfresco Process Services Repository Connector - opencrawling/opencrawling GitHub Wiki

Alfresco Process Services (APS) Repository Connector

Alfresco Process Services Java Spring Boot OIS Compliance MCP

Module: oc-aps-repository-connector
Connector Class: org.opencrawling.aps.ApsRepositoryConnector
MCP Client Class: org.opencrawling.aps.mcp.ApsMcpClient
Spring Bean: @Component (activated when spring.opencrawling.connector.type=aps)


๐Ÿ“Œ Overview

The Alfresco Process Services (APS) Repository Connector enables OpenCrawling to crawl, extract, and index workflow instances, historical process executions, user tasks, BPMN process variables, and file attachments from Alfresco Process Services (APS) 26.2 via the Enterprise REST API (/activiti-app/api/enterprise).

By uniting APS 26.2.0 with the Open Ingestion Standard (OIS), Java 25 Virtual Threads (StructuredTaskScope), and the Secure Model Context Protocol (MCP) Server, this connector allows AI agents and enterprise RAG systems to query running and completed business workflows, identify operational bottlenecks, search process variables, and enforce candidate group authorization boundaries.

graph TD
    subgraph APS["Alfresco Process Services v26.2.0"]
        API["APS Enterprise REST API (:8080/activiti-app/api/enterprise)"]
        Historic["Historic & Active Process Instances"]
        Vars["Process & Task Variables"]
        Tasks["User Tasks & Candidate Groups"]
        Attachments["Process & Task Attachments / Content Streams"]
        
        API --> Historic
        API --> Vars
        API --> Tasks
        API --> Attachments
    end

    subgraph OpenCrawling["OpenCrawling Ingestion Runtime (Java 25)"]
        Connector["ApsRepositoryConnector (oc-aps-repository-connector)"]
        McpClient["ApsMcpClient (JSON-RPC 2.0)"]
        VThreads["Virtual Threads (StructuredTaskScope)"]
        ClaimCheck["Binary Content Stream (Claim-Check Store)"]
        Tika["Apache Tika Extraction"]
        
        API <-->|HTTP/REST Basic Auth or Session Token| Connector
        API <-->|MCP Protocol| McpClient
        Connector --> VThreads
        VThreads --> ClaimCheck
        ClaimCheck --> Tika
        
        OIS["OIS Document Builder (action: UPSERT / DELETE)"]
        VThreads --> OIS
        Tika --> OIS
        
        OIS --> Kafka["Kafka Topic (opencrawling-raw-documents)"]
        Kafka --> Ingestion["oc-ingestion-consumer (Chunking & Embeddings)"]
        Ingestion --> Sinks[("Vector Stores: PGVector / Milvus / Qdrant / Solr / Luxir")]
        Sinks --> SecureMcp["OpenCrawling Secure MCP Server"]
    end

    subgraph AI["Enterprise AI & LLM Assistants"]
        SecureMcp -->|Process-Aware Zero-Trust Retrieval| Agents["AI Agents / RAG Chatbots"]
    end
Loading

๐Ÿš€ Key Features

  • Targeting APS 26.2.0:
    • Built and validated against the containerized enterprise release alfresco/process-services:26.2.0.
    • Integrates with the Enterprise REST API (/activiti-app/api/enterprise).
  • Historic & Active Process Instance Crawling:
    • Paginates through historical and running workflow instances (POST /historic-process-instances/query).
    • Ingests core execution metadata: processDefinitionId, processDefinitionKey, processDefinitionName, businessKey, startUserId, startTime, endTime, durationInMillis, and tenantId.
  • Dynamic BPMN Variable Extraction:
    • Automatically fetches and parses typed process variables (/historic-process-instances/{id}/variables), dynamically indexing them as metadata attributes prefixed with aps_var_<varName>.
  • Zero-Trust Candidate Group ACLs:
    • Discovers user tasks, assignees, candidate users, and candidate groups (/tasks/query).
    • Formulates fine-grained Access Control Lists (acl, security_allowed_read) mapped into OIS security.permissions, ensuring RAG queries strictly respect BPMN organizational access boundaries.
  • Workflow Attachment Ingestion:
    • Discovers documents, forms, and files attached at the process instance level (/process-instances/{id}/content) or user task level (/tasks/{id}/content).
    • Downloads raw content binary streams (/content/{contentId}/raw), routes them through the Claim-Check store, parses them via Apache Tika, and links them back to their parent workflow with inherited variables and ACLs.
  • Model Context Protocol (MCP) Server Integration:
    • The included ApsMcpClient connects to the APS MCP Server to dynamically discover deployed workflow definitions (tools/call: list_process_definitions), variable schemas, and candidate groups over standard JSON-RPC 2.0.
  • Java 25 Virtual Threads Concurrency:
    • Uses Java 25 Thread.ofVirtual() and StructuredTaskScope.open() to enrich batches of process instances and download binary attachments in parallel with zero thread pool exhaustion.

โš™๏ธ Configuration Properties

Configure the connector via application.yml or environment variables:

Property Key Environment Variable Default Description
spring.opencrawling.connector.aps.url APS_URL http://localhost:8080/activiti-app/api/enterprise Base URL of the APS Enterprise REST API
spring.opencrawling.connector.aps.username APS_USERNAME [email protected] APS username (Basic Auth)
spring.opencrawling.connector.aps.password APS_PASSWORD admin APS password (Basic Auth)
spring.opencrawling.connector.aps.batch-size APS_BATCH_SIZE 100 Process instances page batch size
spring.opencrawling.connector.aps.process-definition-key APS_PROCESS_DEFINITION_KEY "" Filter by workflow definition key (leave empty for all)
spring.opencrawling.connector.aps.include-variables APS_INCLUDE_VARIABLES true Ingest process instance variables
spring.opencrawling.connector.aps.include-tasks APS_INCLUDE_TASKS true Ingest user tasks and candidate group ACLs
spring.opencrawling.connector.aps.include-attachments APS_INCLUDE_ATTACHMENTS true Ingest workflow instance attachments & documents
spring.opencrawling.connector.aps.tenant-id APS_TENANT_ID "" Optional tenant ID for multi-tenant APS environments
spring.opencrawling.connector.aps.scope APS_SCOPE all State filter: all (active & completed), active, or completed
spring.opencrawling.connector.aps.include-acls APS_INCLUDE_ACLS true Extract task assignees, candidate users, and candidate groups into OIS SecurityConfig
spring.opencrawling.connector.aps.mcp.enabled APS_MCP_ENABLED false Enable dynamic workflow discovery via APS MCP Server
spring.opencrawling.connector.aps.mcp.url APS_MCP_URL "" (defaults to <url>/mcp) Endpoint URL for the APS MCP Server

Docker Compose Example

oc-crawler:
  image: opencrawling/oc-runtime:latest
  environment:
    CONNECTOR_TYPE: aps
    SCAN_PATH: /
    APS_URL: http://aps:8080/activiti-app/api/enterprise
    APS_USERNAME: [email protected]
    APS_PASSWORD: admin
    APS_INCLUDE_VARIABLES: "true"
    APS_INCLUDE_TASKS: "true"
    APS_INCLUDE_ATTACHMENTS: "true"
    APS_SCOPE: all

๐Ÿ“„ Open Ingestion Standard (OIS) Document Payload

Each crawled process instance is normalized into an OIS document payload:

{
  "id": "aps://process-instances/10042",
  "action": "UPSERT",
  "source": {
    "type": "alfresco-process-services",
    "instance": "http://localhost:8080/activiti-app",
    "connectorVersion": "1.0.0"
  },
  "content": {
    "id": "10042",
    "processDefinitionId": "invoiceApproval:2:89201",
    "processDefinitionKey": "invoiceApproval",
    "processDefinitionName": "Invoice Approval Workflow",
    "businessKey": "INV-2026-9041",
    "startUserId": "finance_agent_1",
    "startTime": "2026-09-30T09:15:00Z",
    "status": "RUNNING",
    "variables": {
      "invoiceNumber": "INV-2026-9041",
      "amount": 14500.50,
      "vendor": "Acme Industrial Supplies",
      "approved": false
    },
    "tasks": [
      {
        "id": "10045",
        "name": "Manager Review",
        "assignee": "manager_alice",
        "candidateGroups": ["group:finance-managers", "group:accounting"],
        "status": "OPEN"
      }
    ]
  },
  "metadata": {
    "mimeType": "application/json",
    "aps.processDefinitionKey": "invoiceApproval",
    "aps.businessKey": "INV-2026-9041",
    "aps.status": "RUNNING",
    "aps_var_invoiceNumber": "INV-2026-9041",
    "aps_var_amount": "14500.50",
    "aps_var_vendor": "Acme Industrial Supplies"
  },
  "security": {
    "inheritanceEnabled": false,
    "permissions": [
      {
        "identity": "finance_agent_1",
        "identityType": "user",
        "access": "read"
      },
      {
        "identity": "manager_alice",
        "identityType": "user",
        "access": "read"
      },
      {
        "identity": "group:finance-managers",
        "identityType": "group",
        "access": "read"
      },
      {
        "identity": "group:accounting",
        "identityType": "group",
        "access": "read"
      }
    ]
  }
}

See Zero-Trust Security & ACL Management for the unified zero-trust architecture covering APS, Flowable, Camunda, Alfresco ACS, and CMIS.


๐Ÿ”‘ Enterprise Licensing Guide

Alfresco Process Services 26.2 is an enterprise commercial product. The official Docker image (alfresco/process-services:26.2.0) requires an enterprise license file (activiti.lic) to enable App Modeler features (importing BPMN files and publishing App definitions), and optionally a transformation license file (transform.lic) for transformation services.

1. Community Evaluation Mode (Unlicensed)

If no license file is provided, the connector connects to APS, verifies authentication, checks health, and performs queries against an empty repository. The test scripts (test-aps-connector.sh and test-aps-decoupled.sh) gracefully detect the unlicensed state and validate the pipeline cleanly without errors.

2. Enterprise Licensed Mode

If you possess Alfresco Process Services license files (activiti.lic and/or transform.lic), you can place them in any of the following locations (checked automatically in order of precedence):

  1. User Home Directory (Recommended for local development):
    • ~/.activiti/enterprise-license/activiti.lic
    • ~/.activiti/enterprise-license/transform.lic
  2. Project Root Directory:
    • opencrawling/activiti.lic
    • opencrawling/transform.lic
  3. Connector Docker Directory:
    • oc-aps-repository-connector/docker/activiti.lic
    • oc-aps-repository-connector/docker/transform.lic
  4. Scripts Directory:
    • scripts/activiti.lic
    • scripts/transform.lic
  5. Custom Paths via Environment Variables (Ideal for CI/CD):
    export APS_LICENSE_FILE="/path/to/activiti.lic"
    export APS_TRANSFORM_LICENSE_FILE="/path/to/transform.lic"

When detected, the test runner scripts (test-aps-connector.sh and test-aps-decoupled.sh) automatically mount both activiti.lic and transform.lic into the container at /home/alfresco/.activiti/enterprise-license/ and Tomcat classpath /usr/local/tomcat/lib/. Furthermore, scripts/seed-aps-workflows.sh will register the license with the running container via POST /activiti-app/api/enterprise/license, import the sample invoiceApproval workflow, publish the application, and seed process instances populated with variables and tasks.


๐Ÿงช Integration Testing

The module provides end-to-end integration tests that run against real Alfresco Process Services 26.2 container instances:

1. Standalone Connector Integration Test

Runs ApsRepositoryConnectorIT.java against an APS 26.2 container:

./scripts/test-aps-connector.sh

This script:

  1. Verifies Docker and dependency prerequisites.
  2. Starts APS 26.2 and PostgreSQL via oc-aps-repository-connector/docker/docker-compose-aps.yml.
  3. Validates /profile and authenticated REST engine connectivity.
  4. Deploys the sample BPMN 2.0 workflow (invoiceApproval.bpmn20.xml), publishes the App Definition, and seeds 3 process instances with variables via scripts/seed-aps-workflows.sh.
  5. Runs Maven unit and integration tests (mvn test -pl oc-aps-repository-connector -Dtest="*Test,*IT").
  6. Automatically cleans up test containers.

2. Full Decoupled Pipeline Integration Test

Validates end-to-end distributed ingestion across the entire multi-service architecture:

./scripts/test-aps-decoupled.sh

Tests: APS 26.2 -> Workflow Seeding (invoiceApproval with business variables & candidate groups) -> OpenCrawling Crawler -> Apache Kafka -> Ingestion Consumer (Claim-Check & Tika chunking) -> Ollama (vector embeddings) -> PostgreSQL (pgvector writer) -> OpenCrawling MCP Server.


๐Ÿ” PGVector Verification Queries

After crawling APS workflows, verify ingestion in PostgreSQL:

-- Total chunks stored for APS process instances and workflow attachments
SELECT count(*) FROM vector_store_1024;

-- Inspect indexed workflow metadata and candidate groups
SELECT id,
       LEFT(content, 200)                            AS content_preview,
       metadata->>'aps.processDefinitionKey'         AS process_key,
       metadata->>'aps.businessKey'                  AS business_key,
       metadata->>'aps_var_invoiceNumber'            AS invoice_num,
       metadata->>'aps_var_amount'                   AS amount,
       metadata->>'uri'                              AS source_uri
FROM vector_store_1024
ORDER BY id
LIMIT 10;

-- Semantic similarity search over business workflows
SELECT id,
       LEFT(content, 300) AS preview,
       metadata->>'aps.processDefinitionKey' AS process_key,
       1 - (embedding <=> (
           SELECT embedding FROM vector_store_1024 LIMIT 1
       )) AS similarity
FROM vector_store_1024
ORDER BY similarity DESC
LIMIT 5;

๐Ÿ”’ Collocated Database Pattern & Query Pushdown

When OpenCrawling stores vector embeddings in a PostgreSQL database with pgvector alongside Alfresco Process Services (APS), you can execute zero-trust semantic queries directly joining with native APS identity link tables without duplicating identity links per vector chunk:

SELECT v.id, v.content, (1 - (v.embedding <=> :query_vector)) AS similarity
FROM vector_store v
WHERE (
    -- Public document access
    v.metadata->>'security_inheritance' = 'public'
    OR
    -- Direct starter or identity user match
    v.metadata->'aps_identity_users' ? :current_user
    OR
    -- Group membership match
    v.metadata->'aps_identity_groups' ?| :current_user_groups
    OR
    -- Collocated DB query-time subquery against native APS runtime identity links
    EXISTS (
        SELECT 1
        FROM act_ru_identitylink il
        WHERE il.proc_inst_id_ = (v.metadata->>'aps_proc_inst_id')
          AND (il.user_id_ = :current_user OR il.group_id_ = ANY(:current_user_groups))
    )
    OR
    -- Collocated DB query-time subquery against native APS historic identity links
    EXISTS (
        SELECT 1
        FROM act_hi_identitylink hil
        WHERE hil.proc_inst_id_ = (v.metadata->>'aps_proc_inst_id')
          AND (hil.user_id_ = :current_user OR hil.group_id_ = ANY(:current_user_groups))
    )
)
ORDER BY v.embedding <=> :query_vector
LIMIT 10;
โš ๏ธ **GitHub.com Fallback** โš ๏ธ