repository_tracing_aspect - italoag/wallet GitHub Wiki
The Repository Tracing Aspect is a Spring AOP-based instrumentation component that automatically creates distributed tracing spans for all JPA repository operations in the Wallet Hub application. This module provides comprehensive observability into database interactions, enabling performance monitoring, query optimization, and troubleshooting of data access patterns.
- Automatic Repository Instrumentation: Traces all public methods in repository interfaces without code changes
- SQL Sanitization: Removes sensitive data from SQL statements while preserving query structure
- Slow Query Detection: Identifies and tags queries exceeding performance thresholds
- Transaction Tracing: Captures Spring transaction boundaries and attributes
- Feature Flag Control: Runtime enable/disable via configuration properties
- OpenTelemetry Compliance: Follows semantic conventions for database operations
âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â Application Layer â
âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââĪ
â Domain Layer â
â âââââââââââââââââââââââââââââââââââââââââââââââââââââââ â
â â Domain Repositories â â
â â âĒ WalletRepository âĒ UserRepository â â
â â âĒ AddressRepository âĒ TransactionRepository â â
â âââââââââââââââââââââââââââââââââââââââââââââââââââââââ â
âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââĪ
â Infrastructure Layer â
â âââââââââââââââââââââââââââââââââââââââââââââââââââââââ â
â â Infrastructure Tracing â â
â â âââââââââââââââââââââââââââââââââââââââââââââââ â â
â â â Repository Tracing Aspect âââââžââââžââ Intercepts
â â âââââââââââââââââââââââââââââââââââââââââââââââ â â
â â âââââââââââââââââââââââââââââââââââââââââââââââ â â
â â â Infrastructure Data â â â
â â â âĒ JpaWalletRepository âââââžââââžââ Implements
â â â âĒ JpaUserRepository â â â
â â â âĒ JpaAddressRepository â â â
â â â âĒ JpaTransactionRepository â â â
â â âââââââââââââââââââââââââââââââââââââââââââââââ â â
â âââââââââââââââââââââââââââââââââââââââââââââââââââââââ â
âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
graph TB
subgraph "Infrastructure Tracing"
RTA[RepositoryTracingAspect]
SSA[SensitiveDataSanitizer]
SQD[SlowQueryDetector]
SAB[SpanAttributeBuilder]
TFF[TracingFeatureFlags]
OR[ObservationRegistry]
end
subgraph "Infrastructure Data"
JWR[JpaWalletRepository]
JUR[JpaUserRepository]
JAR[JpaAddressRepository]
JTR[JpaTransactionRepository]
SDR[SpringDataRepositories]
end
subgraph "Domain Layer"
WR[WalletRepository Interface]
UR[UserRepository Interface]
AR[AddressRepository Interface]
TR[TransactionRepository Interface]
end
subgraph "External Systems"
OTEL[OpenTelemetry Collector]
TRACING_BACKEND[Tracing Backend<br/>Jaeger/Tempo]
end
RTA -->|Intercepts| JWR
RTA -->|Intercepts| JUR
RTA -->|Intercepts| JAR
RTA -->|Intercepts| JTR
RTA -->|Uses| SSA
RTA -->|Uses| SQD
RTA -->|Uses| SAB
RTA -->|Checks| TFF
RTA -->|Creates Spans| OR
JWR -->|Implements| WR
JUR -->|Implements| UR
JAR -->|Implements| AR
JTR -->|Implements| TR
OR -->|Exports| OTEL
OTEL -->|Sends to| TRACING_BACKEND
The main aspect class that intercepts repository method executions and wraps them in observation spans.
Key Responsibilities:
- Intercepts all public methods in repository interfaces matching
dev.bloco.wallet.hub.infra.provider.data.repository.*Repository.*(..) - Creates spans with standardized naming:
repository.{ClassName}.{methodName} - Adds database-specific attributes following OpenTelemetry conventions
- Handles both successful and failed operations
- Integrates with Spring's
@Transactionalannotation for transaction tracing
Configuration:
@Aspect
@Component
@RequiredArgsConstructor
@ConditionalOnProperty(
value = "tracing.features.database",
havingValue = "true",
matchIfMissing = true
)
public class RepositoryTracingAspect {
// Aspect implementation
}| Component | Purpose | Relationship |
|---|---|---|
SpanAttributeBuilder |
Provides OpenTelemetry-compliant attribute constants | Used for standardized span attributes |
SensitiveDataSanitizer |
Sanitizes SQL statements and error messages | Removes PII from trace data |
SlowQueryDetector |
Identifies slow database queries | Tags slow queries for alerting |
TracingFeatureFlags |
Runtime control of tracing features | Enables/disables tracing per environment |
ObservationRegistry |
Micrometer observation infrastructure | Creates and manages spans |
Span Naming Convention:
repository.{RepositoryName}.{methodName}
Example: repository.WalletRepository.findById
Standard Attributes Added:
| Attribute | Value | Description |
|---|---|---|
db.system |
postgresql, h2, etc. |
Database system type |
db.operation |
SELECT, INSERT, UPDATE, DELETE
|
Operation type derived from method name |
db.statement |
Sanitized SQL pattern | Query structure with placeholders |
db.sql.table |
Table name | Primary table accessed |
repository.class |
Repository interface name | e.g., WalletRepository
|
repository.method |
Method name | e.g., findById
|
status |
success or error
|
Operation outcome |
Operation Type Detection:
private String deriveOperationType(String methodName) {
if (methodName.startsWith("find") || methodName.startsWith("get") ||
methodName.startsWith("read") || methodName.startsWith("query") ||
methodName.startsWith("exists") || methodName.startsWith("count")) {
return "SELECT";
} else if (methodName.startsWith("save") || methodName.startsWith("persist") ||
methodName.startsWith("insert") || methodName.startsWith("create")) {
return "INSERT";
} else if (methodName.startsWith("update") || methodName.startsWith("modify")) {
return "UPDATE";
} else if (methodName.startsWith("delete") || methodName.startsWith("remove")) {
return "DELETE";
} else {
return "UNKNOWN";
}
}Span Naming Convention:
transaction.{ClassName}.{methodName}
Transaction Attributes:
| Attribute | Description | Example |
|---|---|---|
tx.isolation_level |
Transaction isolation level | READ_COMMITTED |
tx.propagation |
Transaction propagation behavior | REQUIRED |
tx.read_only |
Read-only transaction flag |
true or false
|
tx.timeout_seconds |
Transaction timeout | 30 |
tx.duration_ms |
Transaction duration | 45 |
tx.status |
Transaction outcome |
COMMITTED or ROLLED_BACK
|
When repository operations fail, the aspect:
- Marks the span with
error=true - Adds error type and sanitized message
- Preserves original exception type (RuntimeException/Error rethrown as-is)
- Converts checked exceptions to RuntimeException for AOP compatibility
private void addErrorAttributes(Observation observation, Throwable throwable) {
observation.lowCardinalityKeyValue(SpanAttributeBuilder.ERROR, "true");
observation.lowCardinalityKeyValue(SpanAttributeBuilder.ERROR_TYPE,
throwable.getClass().getSimpleName());
String message = throwable.getMessage();
if (message != null) {
String sanitizedMessage = sanitizer.sanitizeSql(message);
observation.highCardinalityKeyValue(SpanAttributeBuilder.ERROR_MESSAGE,
truncate(sanitizedMessage, 512));
}
}sequenceDiagram
participant C as Client Code
participant A as RepositoryTracingAspect
participant R as Repository Implementation
participant DB as Database
participant O as ObservationRegistry
C->>A: Call repository method
Note over A: Aspect intercepts call
A->>A: Check feature flag (tracing.features.database)
alt Feature disabled
A->>R: Proceed directly
R->>DB: Execute query
DB-->>R: Return result
R-->>C: Return result
else Feature enabled
A->>O: Create observation span
O-->>A: Return observation
A->>A: Add database attributes
A->>A: Start timer
A->>R: Proceed with observation
R->>DB: Execute query
DB-->>R: Return result
A->>A: Calculate duration
A->>SQD: Check if slow query
SQD-->>A: Detection result
alt Query successful
A->>O: Mark span as success
A->>A: Add performance tags
else Query failed
A->>A: Add error attributes
A->>O: Mark span as error
end
R-->>C: Return result/exception
A->>O: Stop observation
O->>OTEL: Export span data
end
graph LR
A[Original SQL] --> B[String Literal Replacement]
B --> C[Number Literal Replacement]
C --> D[IN Clause Sanitization]
D --> E[UUID Masking]
E --> F[Sanitized SQL]
subgraph "Sanitization Steps"
B["'value' â ?"]
C["= 123 â = ?"]
D["IN (1,2,3) â IN (?)"]
E["UUID â [UUID]"]
end
style F fill:#d4edda
Example:
-- Original SQL:
SELECT * FROM users WHERE email = '[email protected]'
AND phone = '+1234567890' AND id IN (1, 2, 3)
-- Sanitized SQL:
SELECT * FROM users WHERE email = ?
AND phone = ? AND id IN (?)The aspect is controlled by the tracing.features.database property:
# application.yml
tracing:
features:
database: true # Enable/disable repository tracingConfigure slow query detection threshold:
tracing:
sampling:
slow-query-threshold-ms: 50 # Default: 50msThe aspect intercepts methods in:
dev.bloco.wallet.hub.infra.provider.data.repository.*Repository.*(..)
Supported Repositories:
-
JpaWalletRepository- Wallet entity operations -
JpaUserRepository- User entity operations -
JpaAddressRepository- Address entity operations -
JpaTransactionRepository- Transaction entity operations -
SpringData*Repository- Spring Data JPA repositories -
OutboxRepository- Outbox pattern implementation -
StateMachineRepository- Saga state machine persistence
| Operation | Typical Overhead | Notes |
|---|---|---|
| Feature flag check | < 1Ξs | Simple boolean field access |
| Span creation | 1-2ms | Includes context propagation |
| SQL sanitization | 0.5-2ms | Depends on SQL complexity |
| Slow query detection | < 0.1ms | Simple threshold comparison |
| Span export | 2-5ms | Async, non-blocking |
- Selective Tracing: Use feature flags to disable tracing in high-throughput scenarios
- Sampling: Configure sampling rates to reduce volume (see sampling_system.md)
- Batch Operations: Consider manual instrumentation for batch operations
- Caching: Cache sanitized SQL patterns for repeated queries
The aspect integrates with domain repositories through Spring Data JPA:
// Domain repository interface
public interface WalletRepository extends DomainRepository<Wallet> {
Optional<Wallet> findById(WalletId id);
List<Wallet> findByUserId(UserId userId);
}
// Infrastructure implementation (traced)
@Repository
public class JpaWalletRepository implements WalletRepository {
// All public methods automatically traced
public Optional<Wallet> findById(WalletId id) {
// Implementation traced by aspect
}
}- UseCaseTracingAspect: Complements repository tracing with business logic tracing
- R2dbcObservationHandler: Provides reactive database tracing
- WebFluxTracingFilter: Completes end-to-end request tracing
- KafkaObservationHandlers: Connects messaging traces with database operations
@Repository
public interface UserRepository extends JpaRepository<UserEntity, UUID> {
// Automatically traced:
// - Span name: "repository.UserRepository.findByEmail"
// - Attributes: db.system, db.operation, db.statement
Optional<UserEntity> findByEmail(String email);
// Transaction tracing for custom methods
@Transactional
@Modifying
@Query("UPDATE UserEntity u SET u.status = :status WHERE u.id = :id")
int updateStatus(@Param("id") UUID id, @Param("status") UserStatus status);
}@Repository
public class CustomWalletRepositoryImpl implements CustomWalletRepository {
private final EntityManager entityManager;
// All public methods traced
public List<Wallet> findActiveWallets() {
// Complex query traced with attributes
return entityManager.createQuery(
"SELECT w FROM WalletEntity w WHERE w.status = 'ACTIVE'",
Wallet.class
).getResultList();
}
}@Service
public class WalletService {
@Transactional
public Wallet createWallet(CreateWalletCommand command) {
// Creates transaction span: "transaction.WalletService.createWallet"
// Includes: isolation, propagation, timeout attributes
Wallet wallet = walletFactory.create(command);
return walletRepository.save(wallet); // Creates repository span
}
}-
Query Performance:
db.operation+ duration percentiles -
Error Rates:
status=errorby repository class -
Slow Queries:
slow_query=truecount over time -
Transaction Outcomes:
tx.statusdistribution
# Slow query rate
rate(traces_total{slow_query="true"}[5m])
# Error rate by repository
sum by (repository.class) (
rate(traces_total{status="error"}[5m])
) /
sum by (repository.class) (
rate(traces_total[5m])
)
# P95 query duration by operation
histogram_quantile(0.95,
sum by (le, db.operation) (
rate(traces_duration_seconds_bucket[5m])
)
)
# Prometheus alerting rules
groups:
- name: database-alerts
rules:
- alert: HighSlowQueryRate
expr: rate(traces_total{slow_query="true"}[5m]) > 10
for: 5m
labels:
severity: warning
annotations:
summary: "High rate of slow database queries"
- alert: RepositoryErrorRateHigh
expr: |
sum by (repository.class) (
rate(traces_total{status="error"}[5m])
) /
sum by (repository.class) (
rate(traces_total[5m])
) > 0.05
for: 2m
labels:
severity: critical-
Tracing Not Working
- Check
tracing.features.databaseproperty - Verify aspect is registered (check Spring logs)
- Ensure repository methods are public
- Check
-
High Overhead
- Consider increasing sampling rate
- Review SQL sanitization patterns
- Check for N+1 query patterns in traces
-
Missing Attributes
- Verify SpanAttributeBuilder configuration
- Check database connection metadata
- Review method naming conventions
Enable debug logging for troubleshooting:
logging:
level:
dev.bloco.wallet.hub.infra.adapter.tracing.aspect.RepositoryTracingAspect: DEBUG- infrastructure_tracing.md - Overview of tracing infrastructure
- sampling_system.md - Sampling configuration and strategies
- sensitive_data_sanitizer.md - Data sanitization implementation
- use_case_tracing_aspect.md - Business logic tracing aspect
- span_attribute_builder.md - OpenTelemetry attribute standards
- tracing_feature_flags.md - Runtime tracing control
| Version | Date | Changes |
|---|---|---|
| 1.0.0 | 2024-01-15 | Initial release with repository and transaction tracing |
| 1.1.0 | 2024-02-01 | Added slow query detection and SQL sanitization |
| 1.2.0 | 2024-02-15 | Enhanced error handling and feature flag support |
When modifying the RepositoryTracingAspect:
-
Add New Attributes: Use
SpanAttributeBuilderconstants for consistency - Test Performance: Measure overhead for new instrumentation
- Update Documentation: Keep this document synchronized with code changes
- Consider Backwards Compatibility: Maintain existing span naming conventions
This module is part of the Wallet Hub application and follows the same licensing terms as the main project.