Testing Guide - sgajbi/portfolio-analytics-system GitHub Wiki

Overview

The Portfolio Analytics System is a mission-critical financial platform — correctness of calculations, stability of services, and resilience to edge cases are essential.

This guide explains how to:

  • Run unit tests for each microservice.
  • Run integration tests across the pipeline.
  • Incorporate tests into CI/CD.

Test Philosophy

  • Unit tests → Validate service logic in isolation (no DB, no Kafka).
  • Integration tests → Validate service interaction with DB, Kafka, and downstream services.
  • End-to-end tests → Validate the entire pipeline from ingestion to API.

1️⃣ Unit Tests

Each microservice has a /tests/unit/ folder.

Running Unit Tests

Example for Cost Calculator Service:

cd services/calculators/cost_calculator_service
pytest tests/unit --disable-warnings -q

Best Practices

  • Mock Kafka producers/consumers.
  • Mock DB sessions (SQLAlchemy sessionmaker).
  • Test calculation logic with parameterized test cases.

2️⃣ Integration Tests

Integration tests live in /tests/integration/.

They test:

  • Kafka topic publishing and consuming.
  • DB persistence (PostgreSQL test container).
  • Idempotency via processed_events.

Running Integration Tests

Example for Position Calculator Service:

docker-compose -f docker-compose.test.yml up -d
pytest tests/integration --disable-warnings -q
docker-compose -f docker-compose.test.yml down

Key Validations

  • Events flow correctly through Kafka.
  • Correlation ID propagates across services.
  • Duplicate Kafka messages do not break idempotency.
  • DB tables updated correctly.

3️⃣ End-to-End Tests

End-to-end tests ensure full pipeline correctness:

  • Transaction → Ingestion → Persistence → Calculators → API query

Running E2E Tests

docker-compose -f docker-compose.e2e.yml up -d
pytest tests/e2e --disable-warnings -q
docker-compose -f docker-compose.e2e.yml down

Checks

  • API responses include correct positions, valuations, cashflows, performance.
  • X-Correlation-ID present in API response matches logs in Splunk.
  • Database reflects correct final state.

4️⃣ CI/CD Integration

In production pipeline:

  1. Pre-merge:

    • Run unit tests for all services.
    • Run integration tests for modified services.

  2. Nightly build:

    • Run full end-to-end suite.

  3. Pre-deploy:

    • Run smoke tests against staging environment.

Troubleshooting

  • Failing unit test → Check mocks or updated logic.
  • Failing integration test → Verify Kafka topics, DB schema alignment.
  • Failing E2E test → Check correlation ID flow, event ordering.