CMISクエリ - aegif/NemakiWare GitHub Wiki

CMIS-SQL Query Processing Technical Documentation

Overview

This document provides comprehensive technical details about the CMIS-SQL to Solr query translation pipeline in NemakiWare, including logging enhancements for debugging TCK test failures.

Query Processing Pipeline

1. Query Entry Point

  • Location: SolrQueryProcessor.query() method (lines 131-313)
  • Input: CMIS-SQL statement, repository context, user permissions
  • Output: ObjectList with search results

2. TIMESTAMP Preprocessing Phase

// Replace backslashed timestamps for TIMESTAMP only
Pattern time_p = Pattern.compile("(TIMESTAMP\\s?'[\\-\\d]*T\\d{2})\\\\:(\\d{2})\\\\:([\\.\\d]*Z')", Pattern.CASE_INSENSITIVE);
Matcher time_m = time_p.matcher(statement);
statement = time_m.replaceAll("$1:$2:$3");

3. CMIS-SQL Parsing Phase

  • Parser: QueryUtilStrict with CmisTypeManager
  • Output: QueryObject containing parsed query structure
  • WHERE clause extraction: extractWhereTree() method

4. WHERE Clause Translation

  • Handler: SolrPredicateWalker.walkPredicate()
  • Input: CMIS WHERE tree structure
  • Output: Solr query string
  • Example: cmis:objectId = 'abc123' → object_id_s:abc123

5. FROM Clause Translation

  • Type Resolution: Main type from queryObject.getMainFromName()
  • Inheritance Handling: typeManager.getTypeDescendants() for subtype inclusion
  • Filter Query: Builds Solr filter query for object types
  • Example: FROM cmis:document → (object_type_id_s:cmis:document object_type_id_s:custom:document)

6. Solr Query Execution

SolrQuery solrQuery = new SolrQuery();
solrQuery.setQuery(whereQueryString);        // WHERE clause
solrQuery.setFilterQueries(fromQueryString); // FROM clause

7. Permission Filtering

  • Location: permissionService.getFiltered()
  • Process: Filters Solr results based on user permissions
  • Critical: Can reduce result count from Solr matches to 0 if permissions fail

Logging Enhancements

Comprehensive Query Translation Logging

The enhanced logging captures every step of the query translation process:

logger.info("=== CMIS Query Processing Started ===");
logger.info("Repository ID: " + repositoryId);
logger.info("CMIS-SQL Statement: " + statement);
logger.info("includeAllowableActions parameter: " + includeAllowableActions);
logger.info("User: " + callContext.getUsername());

Translation Phase Logging

TIMESTAMP Preprocessing

logger.info("=== TIMESTAMP Preprocessing Phase ===");
// Logs any TIMESTAMP format corrections

CMIS-SQL Parsing

logger.info("=== CMIS-SQL Parsing Phase ===");
// Logs successful parsing or errors

WHERE Clause Translation

logger.info("=== WHERE Clause Translation Phase ===");
logger.info("WHERE clause translation successful: " + whereQueryString);

FROM Clause Translation

logger.info("=== FROM Clause Translation Phase ===");
logger.info("Main FROM type: " + td.getId() + " (query name: " + td.getQueryName() + ")");
logger.info("Type descendants found: " + tables.size());
logger.info("Object type property in Solr: " + objectTypeProperty);
logger.info("Final FROM query string: " + fromQueryString);

Final Solr Query

logger.info("=== Generated Solr Query Details ===");
logger.info("Full Solr query: " + solrQuery.toString());
logger.info("WHERE query string: " + whereQueryString);
logger.info("FROM query string: " + fromQueryString);

Common Translation Issues

1. Field Mapping Problems

  • Issue: CMIS property names not correctly mapped to Solr field names
  • Example: cmis:objectId should map to object_id_s
  • Debug: Check SolrUtil.getPropertyNameInSolr() method

2. Type Hierarchy Issues

  • Issue: Subtype queries not including parent types
  • Debug: Verify typeManager.getTypeDescendants() results

3. Permission Filtering Over-restriction

  • Issue: Valid Solr results filtered out by permission checks
  • Debug: Compare Solr result count vs. final result count

TCK Test Debugging

Current Issues (33% Pass Rate)

  1. Root Folder Queries: SELECT * FROM cmis:folder WHERE cmis:objectId = 'rootId' returns 0 results
  2. Document Queries: SELECT * FROM cmis:document WHERE cmis:objectId = 'docId' returns 0 results
  3. LIKE Queries: Pattern matching failures
  4. IN_FOLDER/IN_TREE: Hierarchical queries failing

Debugging Strategy

  1. Enable Comprehensive Logging: Deploy enhanced SolrQueryProcessor
  2. Check Solr Index: Verify documents exist with *:* query
  3. Trace Translation: Follow CMIS-SQL → Solr query conversion
  4. Permission Analysis: Compare pre/post permission filtering counts

AllowableActions Consistency

Problem

  • UI Layer: Explicitly sets includeAllowableActions=true
  • API Layer: Defaults to false, causing permission check failures

Solution

Boolean queryIncludeAllowableActions = includeAllowableActions == null ? true : includeAllowableActions;
logger.info("Effective includeAllowableActions: " + queryIncludeAllowableActions);

Configuration Files

Solr Schema

  • Location: docker/solr/solr/nemaki/conf/schema.xml
  • Key Fields: object_id_s, object_type_id_s, name_s

CMIS Properties

  • Location: core/src/main/webapp/WEB-INF/classes/nemakiware.properties
  • Solr Connection: solr.host, solr.port, solr.context

TCK Configuration

  • Parameters: cmis-tck-parameters.properties
  • Filters: cmis-tck-filters.properties

Performance Considerations

Query Optimization

  1. Filter Queries: Use Solr filter queries for type restrictions (cached)
  2. Field Selection: Limit returned fields when possible
  3. Permission Caching: Cache permission calculations

Logging Impact

  • Development: Enable comprehensive logging for debugging
  • Production: Use INFO level for key translation steps only
  • Performance: Logging adds ~5-10ms per query

Future Improvements

Enhanced Translation

  1. Better Error Messages: More specific translation failure reasons
  2. Query Validation: Pre-validate CMIS-SQL before translation
  3. Caching: Cache translated queries for repeated patterns

Monitoring

  1. Metrics: Track translation success rates
  2. Alerting: Monitor permission filtering ratios
  3. Performance: Query execution time tracking

References

  • CMIS Specification: OASIS Content Management Interoperability Services (CMIS) Version 1.1
  • Apache Chemistry: OpenCMIS implementation details
  • Solr Documentation: Apache Solr query syntax and filtering
  • NemakiWare Architecture: Core CMIS implementation patterns