Testing Plan and Continuous Integration

Operon uses automated backend tests and GitHub Actions to check functionality, code quality, and test coverage.

Current Testing

Area	Checks
Unit tests	Text decoding, normalization, and extraction from TXT, Markdown, and PDF files, including invalid inputs.
Database and integration tests	Document persistence, relationships, extraction results, repeated requests, and failure handling.
API tests	Client creation and retrieval, document uploads, extraction, response codes, and client-scoped document retrieval.
Test isolation	A separate PostgreSQL test database, synthetic records, and temporary file storage keep tests separate from development data. Selected failures are simulated using dependency overrides and monkeypatching.

Tests use pytest and pytest-cov. Coverage can be generated with:

pytest --cov --cov-branch --cov-report=xml

Database-dependent tests require PostgreSQL and a TEST_DATABASE_URL pointing to a database whose name ends in _test.

Continuous Integration

GitHub Actions workflows are configured for pull requests and pushes to main:

Check	Purpose
Backend tests	Runs pytest using Python 3.13 and a PostgreSQL 17 service.
Coverage reporting	Generates coverage.xml, saves it as a workflow artifact, and uploads it to Codecov.
Code analysis	Runs SonarQube Cloud analysis using the generated coverage report.
Linting and formatting	Runs ruff check and ruff format --check.

Codecov and SonarQube Cloud reporting depend on repository secrets. No minimum coverage threshold is configured in the reviewed test command.

Planned Testing

Future work should add frontend and end-to-end tests, authentication and authorization tests, and AI evaluations for accuracy, missing information, unsupported claims, and source references. Automated frontend builds, container builds, and deployment checks are not included in the current CI workflows.

Coverage Graphs