Module: oc-cmis-repository-connector
Connector Class: org.opencrawling.cmis.CmisRepositoryConnector
Client Class: org.opencrawling.cmis.CmisClient
Document Builder: org.opencrawling.cmis.CmisDocumentBuilder
Security Mapper: org.opencrawling.cmis.CmisSecurityMapper
Spring Bean: @Component (activated when spring.opencrawling.connector.type=cmis or CONNECTOR_TYPE=cmis)

The OASIS CMIS Repository Connector provides OpenCrawling with native, high-performance connectivity to any Enterprise Content Management (ECM) system implementing the OASIS Content Management Interoperability Services (CMIS) 1.0 and 1.1 open standard.

Compatible out of the box with industry-leading ECM systems — including Alfresco Content Services (ACS), OpenText Content Server / Documentum, IBM FileNet P8, Apache Chemistry InMemory / FileShare, Nuxeo, and Microsoft SharePoint (via CMIS bridges) — this connector crawls folder hierarchies, executes CMIS-SQL queries, retrieves version series, extracts fine-grained Access Control Lists (ACLs), captures secondary types / aspects, and streams binary content payloads directly into OpenCrawling's RAG and vector ingestion pipelines.

Built entirely on modern Java 25 (java.net.http.HttpClient), the connector features zero legacy dependencies (avoiding deprecated Apache Chemistry client jars), leverages Java 25 Virtual Threads (StructuredTaskScope) for concurrent traversal, and seamlessly integrates with the Open Ingestion Standard (OIS) and OpenCrawling's Claim Check Pattern.

graph TD
    subgraph ECM["Enterprise ECM Repositories (OASIS CMIS 1.0 / 1.1)"]
        ACS["Alfresco Content Services"]
        DCTM["OpenText Documentum"]
        FN["IBM FileNet P8"]
        NX["Nuxeo / Apache Chemistry"]
        
        ACS & DCTM & FN & NX -->|CMIS 1.1 Browser Binding / JSON| Endpoint["CMIS REST Endpoint (:8080/.../versions/1.1/browser)"]
    end

    subgraph OpenCrawling["OpenCrawling Ingestion Runtime (Java 25)"]
        CmisClient["CmisClient (Native JDK 25 HttpClient)"]
        Connector["CmisRepositoryConnector (oc-cmis-repository-connector)"]
        VThreads["Virtual Threads (StructuredTaskScope)"]
        SecMap["CmisSecurityMapper (ACE -> OIS Permissions)"]
        DocBld["CmisDocumentBuilder (Properties & Aspects)"]
        ClaimCheck["Binary Content Stream (Claim Check Store)"]
        Tika["Apache Tika Text Extraction"]
        
        Endpoint <-->|HTTP GET / POST JSON| CmisClient
        CmisClient --> Connector
        Connector --> VThreads
        VThreads --> SecMap
        VThreads --> DocBld
        VThreads -->|cmisselector=content| ClaimCheck
        ClaimCheck --> Tika
        
        OIS["OIS Document (UPSERT / Tombstone DELETE)"]
        DocBld --> OIS
        SecMap --> OIS
        Tika --> OIS
        
        OIS --> Kafka["Kafka Topic (opencrawling-raw-documents)"]
        Kafka --> Ingestion["oc-ingestion-consumer (Chunking & Embeddings)"]
        Ingestion --> Sinks[("Vector Stores: PGVector / Milvus / Qdrant / Solr / Luxir")]
    end

    subgraph AI["Enterprise AI & RAG Assistants"]
        Sinks --> SecureMcp["OpenCrawling Secure MCP Server"]
        SecureMcp -->|Zero-Trust ACL-Filtered RAG| Agents["AI Agents / Enterprise LLMs"]
    end

• Standardized OASIS CMIS 1.0 & 1.1 Support:
  • Implements the CMIS 1.1 Browser Binding (JSON over HTTP/REST), providing superior serialization speed and payload efficiency compared to legacy AtomPub or SOAP bindings.
  • Connects to any compliant CMIS service endpoint with standard HTTP Basic Authentication or Bearer tokens.
• Zero Legacy Dependencies:
  • Built using pure Java 25 HttpClient and Jackson JSON parsing. Eliminates bulky, end-of-life Apache Chemistry client libraries while delivering faster startup, smaller footprint, and full compatibility with modern JDK runtime modules.
• Flexible Crawl Modes:
  1. Folder Hierarchy (FOLDER): Recursively crawls folders and documents starting from a configurable root path (e.g. /, /Company Home/Sites) or folder ID, with configurable exclusion rules (excluded-folder-paths).
  2. CMIS-SQL Query (QUERY): Ingests documents using standard CMIS-SQL queries (e.g. SELECT * FROM cmis:document WHERE cmis:creationDate >= '2026-01-01T00:00:00.000Z'), allowing targeted, dynamic document selection.
  3. Change Log Ingestion (changeLogEnabled): Consumes the repository change log (getContentChanges) using incremental change tokens for efficient near-real-time synchronization.
• Configurable Versioning Policies:
  • latest_major: Ingests only the latest released major version (cmis:isLatestMajorVersion = true), ideal for official corporate documentation.
  • latest: Ingests the latest version (cmis:isLatestVersion = true), including work-in-progress minor revisions.
  • all: Ingests all versions in the document version series.
• Zero-Trust Access Control Lists (ACLs):
  • Automatically inspects repository Access Control Entries (ACEs) and maps permissions (cmis:read, cmis:write, cmis:all) into Open Ingestion Standard (OIS) SecurityConfig and PermissionRule structures.
  • Distinguishes between user principals, organizational groups (group:*), and public/everyone identities, preserving strict authorization barriers during vector search.
• Secondary Types & Aspects Extraction:
  • Parses both concise (succinctProperties) and verbose CMIS 1.1 property structures.
  • Automatically extracts secondary object type IDs (cmis:secondaryObjectTypeIds) and all attached custom aspect properties (e.g., Alfresco cm:titled, cm:author, custom metadata namespaces).
• Claim Check Pattern for Large Binaries:
  • Streams raw document content streams (cmisselector=content) directly to the configured Claim Check store (local shared volume, Apache Ozone, AWS S3) with configurable size limits (max-content-size-bytes), passing lightweight URI references through Kafka.
• Document Deletion & Tombstones:
  • Generates standard OIS tombstone records (DocumentAction.DELETE) when objects are removed or detected via change logs, triggering automated purging from downstream vector databases.
• High-Throughput Concurrency with Virtual Threads:
  • Leverages Java 25 StructuredTaskScope and virtual threads (Thread.ofVirtual()) to parallelize folder traversal, document metadata enrichment, and binary content retrieval.

Configure the CMIS Repository Connector via application.yml or standard environment variables:

When launching jobs via the OpenCrawling Admin UI, REST API, or CLI, the SCAN_PATH parameter dynamically overrides the crawl target:

Each crawled CMIS document is normalized into an OIS RepositoryDocument with structured metadata attributes:

See Zero-Trust Security & ACL Management for the unified architecture covering CMIS, Alfresco, Flowable, and Camunda.

The connector's CmisSecurityMapper inspects CMIS Access Control Lists (ACLs) and maps them into OIS SecurityConfig:

• Principal Resolution:
  • everyone, group_everyone, anonymous $\rightarrow$ public identity type.
  • Principals starting with group: or containing group $\rightarrow$ group identity type.
  • Named user accounts $\rightarrow$ user identity type.
• Permission Level Mapping:
  • CMIS cmis:all $\rightarrow$ OIS admin permission.
  • CMIS cmis:write $\rightarrow$ OIS write permission.
  • CMIS cmis:read $\rightarrow$ OIS read permission.

During vector retrieval, queries automatically enforce these permissions:

{
  "security": {
    "isExact": true,
    "permissions": [
      {
        "principal": "GROUP_ENGINEERING",
        "identityType": "group",
        "access": "read"
      },
      {
        "principal": "john.doe",
        "identityType": "user",
        "access": "write"
      }
    ]
  }
}

In the production decoupled architecture, oc-crawler streams CMIS documents to Kafka, oc-ingestion-consumer chunks content, oc-embedding-service generates vector embeddings, and oc-writer-consumer writes to PostgreSQL PGVector:

services:
  oc-crawler:
    image: opencrawling/oc-runtime:latest
    container_name: oc-crawler-cmis
    environment:
      CONNECTOR_TYPE: cmis
      SCAN_PATH: /Company Home
      CMIS_ENDPOINT_URL: http://alfresco:8080/alfresco/api/-default-/public/cmis/versions/1.1/browser
      CMIS_REPOSITORY_ID: -default-
      CMIS_USERNAME: admin
      CMIS_PASSWORD: admin
      CMIS_CRAWL_MODE: folder
      CMIS_INCLUDE_SUBFOLDERS: "true"
      CMIS_VERSIONS_MODE: latest_major
      CMIS_INCLUDE_ACLS: "true"
      CMIS_INCLUDE_CONTENT_STREAM: "true"
      SPRING_OPENCRAWLING_CLAIM_CHECK_STORE: local
      SPRING_OPENCRAWLING_CLAIM_CHECK_LOCAL_DIR: /data/claims
      KAFKA_BOOTSTRAP_SERVERS: kafka:9092
    volumes:
      - shared-data:/data/claims
    depends_on:
      - kafka
      - alfresco

For quick local testing or single-container jobs:

services:
  oc-runtime:
    image: opencrawling/oc-runtime:latest
    environment:
      CONNECTOR_TYPE: cmis
      SPRING_OPENCRAWLING_OUTPUT_TYPE: vector
      CMIS_ENDPOINT_URL: http://cmis-server:8080/cmis/browser
      CMIS_USERNAME: admin
      CMIS_PASSWORD: admin
      CMIS_CRAWL_MODE: folder
      CMIS_ROOT_FOLDER_PATH: /
      SPRING_AI_VECTORSTORE_PGVECTOR_URL: jdbc:postgresql://postgres:5432/opencrawling

The OpenCrawling repository includes automated end-to-end integration tests for both direct and decoupled CMIS pipelines:

# Direct integration test with mock CMIS repository
./opencrawling-in-org/scripts/test-cmis-connector.sh

# Decoupled distributed test (Crawler -> Kafka -> Ingestion -> Embeddings -> PGVector)
./opencrawling-in-org/scripts/test-cmis-decoupled.sh

# Complete integration test runner
./run-integration-tests.sh scripts/test-cmis-decoupled.sh

After a crawl job finishes, verify ingested CMIS documents and chunks in PostgreSQL:

-- Check total embedded document chunks
SELECT count(*) FROM vector_store_1024;

-- Preview chunk content and CMIS metadata
SELECT id,
       LEFT(content, 200)               AS preview,
       metadata->>'cmis.name'           AS document_name,
       metadata->>'cmis.objectId'       AS object_id,
       metadata->>'cmis.versionLabel'   AS version,
       metadata->>'cmis.path'           AS cmis_path
FROM vector_store_1024
ORDER BY id
LIMIT 10;

-- Test ACL-aware semantic similarity search
SELECT id,
       LEFT(content, 300) AS preview,
       metadata->>'cmis.name' AS document_name,
       1 - (embedding <=> (SELECT embedding FROM vector_store_1024 LIMIT 1)) AS similarity
FROM vector_store_1024
WHERE metadata->'security'->'permissions' @> '[{"principal":"GROUP_ENGINEERING"}]'
ORDER BY similarity DESC
LIMIT 5;