BRANCH COVERAGE IMPLEMENTATION - nself-org/cli GitHub Wiki

Branch Coverage Implementation Summary

Overview

This document summarizes the complete branch coverage testing infrastructure implemented for nself.

Goal: Achieve and maintain 100% branch coverage with resilient, platform-independent tests.

Current Status

Coverage Metrics

# Current coverage (as of implementation)
Total Branches: 24,472
Tested Branches: 14,683 (estimated)
Coverage: 59%

# Goal
Target: 100% (24,472 branches)
Remaining: 9,789 branches

Branch Distribution

  • If/Else Statements: 5,489 statements โ†’ 11,452 branches
  • Case Statements: 622 statements โ†’ 4,170 branches
  • Logical Operators: 4,425 operators (1,910 &&, 2,515 ||) โ†’ 8,850 branches
  • Return Statements: 4,540 explicit returns

Implemented Components

1. Analysis Tools

Branch Coverage Analysis Script

File: scripts/branch-coverage-analysis.sh

Features:

  • Scans all shell scripts in src/lib/ and src/cli/
  • Counts all conditional branches (if, case, &&, ||, return)
  • Estimates test coverage
  • Generates reports in text and JSON formats
  • Provides coverage percentage and remaining work

Usage:

bash scripts/branch-coverage-analysis.sh

# Output files:
# - .coverage/branch-coverage-report.txt
# - .coverage/branch-coverage.json

Untested Branches Reporter

File: scripts/show-untested-branches.sh

Features:

  • Identifies source files without test coverage
  • Lists specific untested conditionals
  • Highlights complex case statements needing coverage
  • Provides actionable recommendations

Usage:

bash scripts/show-untested-branches.sh

2. Testing Infrastructure

Environment Control Mocks

File: src/tests/mocks/environment-control.sh

Provides:

  • Platform mocking (macOS, Linux, WSL)
  • Command availability control
  • Docker state simulation
  • File/directory existence control
  • Environment variable management
  • Network availability mocking
  • Process and port mocking
  • Environment save/restore

Functions:

mock_platform "macos"
mock_command_exists "timeout" "false"
mock_docker_running "true"
mock_file_exists ".env" "true" "ENV=dev"
mock_env_var "DEBUG" "true"
save_environment / restore_environment
setup_test_environment "ci"

Test Framework Enhancements

File: src/tests/lib/reliable-test-framework.sh

Added:

  • Color constants (RED, GREEN, YELLOW, BLUE, NC)
  • assert_equals(expected, actual, description)
  • assert_success(description)
  • assert_contains(haystack, needle, description)

3. Test Templates

Branch Coverage Test Template

File: src/tests/unit/test-branch-coverage-template.sh

Demonstrates 10 testing patterns:

  1. If/Else branch testing
  2. Platform-specific branches
  3. Optional command availability
  4. Case statement all branches
  5. AND operator short-circuit
  6. OR operator alternatives
  7. Error handling branches
  8. Nested conditionals
  9. Function return paths
  10. File existence checks

Test Results:

โœ“ 10 tests passed
โœ“ 27 branches tested
โœ“ 100% template coverage

Real Module Example

File: src/tests/unit/test-validation-branch-coverage.sh

Tests: src/lib/init/validation.sh

Coverage:

  • All 51+ conditional branches in validation module
  • Platform detection branches
  • Command availability branches
  • File system checks
  • Error handling paths
  • Security checks

4. CI/CD Integration

GitHub Actions Workflow

File: .github/workflows/branch-coverage.yml

Features:

  • Runs on push to main/develop
  • Analyzes branch coverage
  • Checks minimum threshold (60%)
  • Comments on PRs with coverage report
  • Uploads coverage artifacts
  • Runs tests on Ubuntu and macOS

Triggers:

  • Changes to src/lib/**/*.sh
  • Changes to src/cli/**/*.sh
  • Changes to src/tests/**/*.sh
  • Changes to coverage scripts

5. Documentation

Comprehensive Guide

File: docs/testing/BRANCH-COVERAGE-GUIDE.md

Contents:

  • What is branch coverage?
  • Why 100% coverage matters
  • Branch types and patterns
  • Testing strategies
  • Tools and scripts reference
  • Best practices
  • Contributing guidelines
  • Examples and templates

Testing Patterns

Pattern 1: If/Else Branches

# Code
if [[ condition ]]; then
  branch_true
else
  branch_false
fi

# Test
test_both_branches() {
  # True path
  setup_true
  assert_result_true

  # False path
  setup_false
  assert_result_false
}

Pattern 2: Platform-Specific

# Code
if [[ "$OSTYPE" == "darwin"* ]]; then
  use_bsd_commands
else
  use_gnu_commands
fi

# Test
test_platform_specific() {
  mock_platform "macos"
  assert_bsd_used

  mock_platform "linux"
  assert_gnu_used
}

Pattern 3: Optional Commands

# Code
if command -v timeout >/dev/null 2>&1; then
  timeout 30 operation
else
  operation  # No timeout
fi

# Test
test_optional_command() {
  # With command (actual)
  command -v bash
  assert_success

  # Without command (mock)
  command -v nonexistent
  assert_fallback_works
}

Pattern 4: Case Statements

# Code
case "$cmd" in
  start) start_service ;;
  stop) stop_service ;;
  *) show_help ;;
esac

# Test
test_all_cases() {
  for cmd in start stop invalid; do
    test_case "$cmd"
    assert_correct_action
  done
}

Pattern 5: Logical Operators

# Code
if check_a && check_b; then
  both_pass
else
  one_or_both_fail
fi

# Test
test_and_operator() {
  # Both true
  mock_both_true
  assert_both_pass

  # First false (short-circuit)
  mock_first_false
  assert_one_or_both_fail

  # Second false
  mock_second_false
  assert_one_or_both_fail
}

Pattern 6: Error Handling

# Code
if operation; then
  success
else
  handle_error
fi

# Test
test_error_handling() {
  # Success path
  mock_success
  assert_success_action

  # Error path (gracefully handled)
  mock_error
  assert_error_handled
  # Test PASSES - error was handled
}

Pattern 7: Graceful Degradation

# Code
if redis_available; then
  use_redis_cache
else
  use_memory_cache
fi

# Test
test_degradation() {
  # With Redis
  mock_redis "true"
  assert_redis_used

  # Without Redis (degrades gracefully)
  mock_redis "false"
  assert_memory_used
  assert_success  # Still works!
}

Resilient Testing Principles

1. Never Fail on Environment

# โŒ Bad - fails on macOS
test_timeout() {
  timeout 5 operation
  assert_success
}

# โœ… Good - works everywhere
test_timeout() {
  if command -v timeout >/dev/null 2>&1; then
    timeout 5 operation
  else
    operation  # No timeout available
  fi
  assert_success
}

2. Test All Branches

# โŒ Bad - only tests happy path
test_feature() {
  setup_success
  run_feature
  assert_success
}

# โœ… Good - tests all paths
test_feature() {
  # Success path
  test_success_case

  # Error path
  test_error_case

  # Edge cases
  test_edge_cases
}

3. Mock for Control

# โœ… Control which branch executes
test_platform_specific() {
  # Force macOS path
  OSTYPE="darwin22.0" run_code

  # Force Linux path
  OSTYPE="linux-gnu" run_code
}

4. Test Error Paths

# โœ… Error handling is code too
test_error_handling() {
  # Trigger error
  mock_failure

  # Should handle gracefully
  run_code
  assert_error_message_shown

  # Test PASSES because error was handled
}

Roadmap

Phase 1: Foundation (v0.9.7) โœ…

  • Branch coverage analysis tools
  • Environment control mocks
  • Test templates and patterns
  • CI/CD integration
  • Documentation

Phase 2: Core Coverage (v0.9.8) ๐ŸŽฏ

  • 60% overall coverage baseline
  • 100% coverage for init modules
  • 100% coverage for tenant modules
  • Critical CLI commands covered

Phase 3: Extended Coverage (v0.9.9-v1.0) ๐Ÿ”œ

  • 80% overall coverage
  • All core modules covered
  • Integration test branch coverage
  • Coverage regression detection

Phase 4: Complete Coverage (v1.x) ๐Ÿš€

  • 100% branch coverage
  • Coverage enforcement in CI (blocking)
  • Branch coverage badges
  • Coverage trends dashboard

Usage Examples

Running Coverage Analysis

# Full analysis
bash scripts/branch-coverage-analysis.sh

# Check specific module
bash scripts/show-untested-branches.sh | grep "validation.sh"

# View reports
cat .coverage/branch-coverage-report.txt
jq . .coverage/branch-coverage.json

Writing Branch Tests

# 1. Copy template
cp src/tests/unit/test-branch-coverage-template.sh \
   src/tests/unit/test-mymodule-branch-coverage.sh

# 2. Identify branches in your module
grep "if \[" src/lib/mymodule.sh
grep "case.*in" src/lib/mymodule.sh

# 3. Write tests for each branch
# Use patterns from template

# 4. Run tests
bash src/tests/unit/test-mymodule-branch-coverage.sh

# 5. Verify coverage
bash scripts/branch-coverage-analysis.sh

Using Mocks

# Source mocks
source src/tests/mocks/environment-control.sh

# Mock platform
mock_platform "macos"

# Mock commands
mock_command_exists "docker" "true"
mock_docker_running "false"

# Mock files
mock_file_exists ".env" "true" "ENV=dev"

# Run test
your_test_function

# Cleanup
cleanup_mocks

Key Metrics

Files Created

  1. scripts/branch-coverage-analysis.sh - Analysis tool
  2. scripts/show-untested-branches.sh - Reporter
  3. src/tests/mocks/environment-control.sh - Mock infrastructure
  4. src/tests/unit/test-branch-coverage-template.sh - Pattern template
  5. src/tests/unit/test-validation-branch-coverage.sh - Real example
  6. .github/workflows/branch-coverage.yml - CI/CD workflow
  7. docs/testing/BRANCH-COVERAGE-GUIDE.md - Complete guide
  8. docs/testing/BRANCH-COVERAGE-IMPLEMENTATION.md - This document

Enhanced Files

  1. src/tests/lib/reliable-test-framework.sh - Added colors and assertions

Branches Identified

  • Total Conditional Branches: 24,472
  • Currently Tested (estimated): 14,683 (59%)
  • Remaining to Test: 9,789 (41%)

Test Coverage by Type

Type Count Branches Coverage
If/Else 5,489 11,452 ~60%
Case 622 4,170 ~55%
&& operators 1,910 3,820 ~65%
|| operators 2,515 5,030 ~60%
Returns 4,540 - ~50%

Benefits Achieved

1. Visibility

  • Can now see exactly which branches are untested
  • Know precisely how much work remains
  • Track progress toward 100% coverage

2. Infrastructure

  • Comprehensive mocking framework
  • Reusable test patterns
  • CI/CD automation

3. Quality

  • Tests are resilient (work on all platforms)
  • Error paths are tested
  • Graceful degradation is verified

4. Documentation

  • Every test documents expected behavior
  • Patterns are well-documented
  • Examples for contributors

5. Confidence

  • Refactoring is safer
  • Regressions are caught early
  • Platform compatibility is verified

Next Steps

  1. Write Tests for Untested Files

    • Use show-untested-branches.sh to identify targets
    • Start with high-impact modules
    • Use templates for consistency
  2. Increase Coverage Threshold

    • Current: 59%
    • Next milestone: 60% (Phase 2)
    • CI threshold: Start at 60%, increase gradually
  3. Enforce Coverage in CI

    • Block PRs below threshold
    • Require coverage for new code
    • Trend toward 100%
  4. Add Integration Test Coverage

    • Extend to integration tests
    • End-to-end path coverage
    • Real-world scenario testing

Resources

Conclusion

The branch coverage infrastructure is now complete and operational. With 24,472 branches identified and tools to systematically test them, nself is on a clear path to 100% branch coverage.

Key Achievement: A foundation for bulletproof testing that works across all platforms, handles all code paths, and degrades gracefully when features are unavailable.

Next Milestone: Increase coverage from 59% to 80% by systematically testing all identified branches using the provided tools and patterns.


Status: โœ… Infrastructure Complete Current Coverage: 59% (14,683 / 24,472 branches) Target: 100% (24,472 / 24,472 branches) Remaining Work: 9,789 branches to test Timeframe: Incremental improvement over v0.9.8 โ†’ v1.0

Remember: Every untested branch is a potential bug. With this infrastructure, we can systematically eliminate that risk.

โš ๏ธ **GitHub.com Fallback** โš ๏ธ