Test Reliability Cheat Sheet

Quick reference for writing bulletproof tests

Basic Template

#!/usr/bin/env bash
source src/tests/lib/reliable-test-framework.sh
source src/tests/mocks/docker-mock.sh  # Optional

test_my_feature() {
  # Your test code here
  echo "✓ Test passed"
}

run_test_with_timeout test_my_feature 30

Common Patterns

Pattern: Test with Auto-Cleanup

test_dir="/tmp/test-$$"
mkdir -p "$test_dir"

# ... test code ...

rm -rf "$test_dir"  # Manual cleanup

Or use:

test_dir=$(create_isolated_test_dir)
# ... test code ...
# Cleanup automatic!

Pattern: Docker Test (No Docker Needed)

source src/tests/mocks/docker-mock.sh

docker run --name test nginx  # Instant!
docker ps | grep -q "test"    # Works!
docker stop test
docker rm test

Pattern: HTTP Request Test (No Network)

source src/tests/mocks/network-mock.sh

register_mock_response "https://api.example.com/status" 200 '{"ok":true}'
response=$(curl -s https://api.example.com/status)  # Instant!
echo "$response" | grep -q "ok"

Pattern: Timeout Test (Instant)

source src/tests/mocks/time-mock.sh

enable_time_mock
set_time_multiplier 100.0  # 100x speed

sleep 60  # Completes in 0.6 seconds

disable_time_mock

Pattern: Retry Flaky Operation

ATTEMPT=0
flaky_func() {
  ATTEMPT=$((ATTEMPT + 1))
  [[ $ATTEMPT -ge 3 ]] && return 0 || return 1
}

retry_on_failure flaky_func 5  # Retries up to 5 times

Pattern: Wait for Async

# Start async operation
do_something_async &

# Wait for file
wait_for_file "/tmp/ready.txt" 10

# Wait for port
wait_for_port 8080 30

Pattern: Test Isolation

# Unique resources per test
project=$(get_unique_project_name "myapp")
db=$(get_unique_db_name "testdb")
port=$(get_random_port)

Quick Reference

Task	Command
Source framework	`source src/tests/lib/reliable-test-framework.sh`
Run with timeout	`run_test_with_timeout my_test 30`
Guaranteed cleanup	`with_cleanup test_func cleanup_func`
Docker mock	`source src/tests/mocks/docker-mock.sh`
Network mock	`source src/tests/mocks/network-mock.sh`
Time mock	`source src/tests/mocks/time-mock.sh`
Filesystem mock	`source src/tests/mocks/filesystem-mock.sh`
Retry flaky	`retry_on_failure test 3`
Wait for file	`wait_for_file "/path" 10`
Wait for port	`wait_for_port 8080 30`
Random port	`port=$(get_random_port)`
Unique name	`name=$(get_unique_project_name "app")`
Platform detect	`platform=$(detect_platform)`

Analysis Tools

# Find flaky tests
bash scripts/find-flaky-tests.sh --iterations 10

# Analyze performance
bash scripts/test-performance-analysis.sh

# Install pre-commit hook
bash scripts/install-pre-commit-hook.sh

# Run examples
bash src/tests/examples/reliable-test-example.sh

Pre-Commit Hook

# Install
bash scripts/install-pre-commit-hook.sh

# Skip (temporary)
git commit --no-verify

Common Mistakes to Avoid

❌ Don't use real services

docker run ...  # Slow, requires Docker

✅ Use mocks

source src/tests/mocks/docker-mock.sh
docker run ...  # Instant, no Docker

❌ Don't use sleep without polling

sleep 10  # Always waits 10s

✅ Use wait functions

wait_for_file "/tmp/ready" 10  # Returns as soon as ready

❌ Don't share resources

TEST_PORT=8080  # Collision!

✅ Use unique resources

TEST_PORT=$(get_random_port)  # Unique!

❌ Don't forget cleanup

setup_resources
run_test  # If fails, cleanup might not run!
cleanup_resources

✅ Guarantee cleanup

with_cleanup run_test cleanup_resources

Best Practices

✅ Always use timeout protection
✅ Always guarantee cleanup
✅ Use mocks, not real services
✅ Isolate test resources
✅ Poll instead of sleep
✅ Make tests deterministic
✅ Run fast (<5s per test)
✅ Zero tolerance for flakiness

Full Documentation

Quick Start: docs/development/TEST-RELIABILITY-Quick-Start.md
Best Practices: docs/development/TEST-RELIABILITY-GUIDE.md
Implementation: docs/development/TEST-RELIABILITY-IMPLEMENTATION.md
Complete Summary: TEST-RELIABILITY-COMPLETE.md

TEST RELIABILITY CHEATSHEET - nself-org/cli GitHub Wiki

Test Reliability Cheat Sheet

Basic Template

Common Patterns

Pattern: Test with Auto-Cleanup

Pattern: Docker Test (No Docker Needed)

Pattern: HTTP Request Test (No Network)

Pattern: Timeout Test (Instant)

Pattern: Retry Flaky Operation

Pattern: Wait for Async

Pattern: Test Isolation

Quick Reference

Analysis Tools

Pre-Commit Hook

Common Mistakes to Avoid

Best Practices

Full Documentation

⚠️ GitHub.com Fallback ⚠️

TEST RELIABILITY CHEATSHEET - nself-org/cli GitHub Wiki

Test Reliability Cheat Sheet

Basic Template

Common Patterns

Pattern: Test with Auto-Cleanup

Pattern: Docker Test (No Docker Needed)

Pattern: HTTP Request Test (No Network)

Pattern: Timeout Test (Instant)

Pattern: Retry Flaky Operation

Pattern: Wait for Async

Pattern: Test Isolation

Quick Reference

Analysis Tools

Pre-Commit Hook

Common Mistakes to Avoid

Best Practices

Full Documentation

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️