Testing Testing Strategy - aku11i/phantom GitHub Wiki

Testing Strategy

This document outlines Phantom's comprehensive testing approach and philosophy.

Testing Philosophy

Phantom follows these core testing principles:

  1. Test Behavior, Not Implementation - Focus on what the code does, not how
  2. Fast Feedback - Tests should run quickly to encourage frequent execution
  3. Isolation - Tests should not depend on external state or other tests
  4. Clarity - Test failures should clearly indicate what's broken
  5. Confidence - Tests should give confidence in code correctness

Testing Stack

Test Runner

Source: package.json#L15

Phantom uses Node.js's built-in test runner:

node --test

Benefits:

  • Zero dependencies
  • Fast execution
  • Native TypeScript support
  • Built-in mocking capabilities
  • Parallel test execution

Testing Tools

  • Node.js Test Runner - Test execution and assertions
  • Mock Module API - Native module mocking
  • TypeScript - Type-safe test code
  • No External Frameworks - Simplicity and speed

Test Organization

File Structure

Tests are co-located with source files:

src/
├── core/
│   ├── worktree/
│   │   ├── create.ts
│   │   ├── create.test.ts      # Unit tests
│   │   ├── delete.ts
│   │   └── delete.test.ts
│   ├── git/
│   │   ├── executor.ts
│   │   └── executor.test.ts
│   └── process/
│       ├── exec.ts
│       └── exec.test.ts

Benefits:

  • Easy to find tests
  • Encourages testing
  • Simplifies imports
  • Clear relationships

Test File Naming

  • Pattern: <module>.test.ts
  • Location: Same directory as source
  • Execution: Automatically discovered by test runner

Test Categories

1. Unit Tests

Purpose: Test individual functions and modules in isolation

Characteristics:

  • Mock all external dependencies
  • Test pure functions directly
  • Fast execution (< 10ms per test)
  • High code coverage

Example Structure:

import { describe, it, beforeEach, mock } from "node:test";
import assert from "node:assert";

describe("createWorktree", () => {
  beforeEach(() => {
    // Setup mocks
  });

  it("should create worktree with valid name", async () => {
    // Test implementation
  });

  it("should fail with invalid name", async () => {
    // Test error cases
  });
});

2. Integration Tests

Purpose: Test interaction between modules

Characteristics:

  • Limited mocking (only external systems)
  • Test module boundaries
  • Verify data flow
  • Medium execution time

Current Status: Primarily unit tests with some integration aspects

3. End-to-End Tests

Purpose: Test complete user workflows

Characteristics:

  • No mocking
  • Real Git operations
  • Full command execution
  • Slower execution

Current Status: Not implemented (future consideration)

Mocking Strategy

Module Mocking

Source: Various test files like create.test.ts

Phantom uses Node.js's native module mocking:

import { mock } from "node:test";

// Mock file system
mock.module("node:fs/promises", {
  namedExports: {
    mkdir: mock.fn(() => Promise.resolve()),
    access: mock.fn(() => Promise.reject(new Error("Not found")))
  }
});

// Mock child process
mock.module("node:child_process", {
  namedExports: {
    spawn: mock.fn(() => mockProcess)
  }
});

Mock Patterns

1. Git Command Mocking

const mockGitExecutor = {
  execute: mock.fn((args: string[]) => {
    if (args[0] === "worktree" && args[1] === "list") {
      return Result.ok("worktree /path/to/main\nHEAD abc123");
    }
    return Result.error(new Error("Command failed"));
  })
};

2. File System Mocking

const mockFs = {
  access: mock.fn(async (path: string) => {
    if (path === "/tmp/phantom/repo") {
      throw new Error("ENOENT");
    }
  }),
  mkdir: mock.fn(async () => {})
};

3. Process Mocking

const mockProcess = {
  stdout: { on: mock.fn() },
  stderr: { on: mock.fn() },
  on: mock.fn((event, callback) => {
    if (event === "exit") {
      callback(0);
    }
  })
};

Test Patterns

1. Result Type Testing

Test both success and failure paths:

it("should return ok result on success", async () => {
  const result = await createWorktree("feature");
  assert.strictEqual(result.ok, true);
  if (result.ok) {
    assert.strictEqual(result.value.name, "feature");
  }
});

it("should return error result on failure", async () => {
  const result = await createWorktree("../invalid");
  assert.strictEqual(result.ok, false);
  if (!result.ok) {
    assert.match(result.error.message, /invalid/i);
  }
});

2. Error Scenario Testing

Test various error conditions:

describe("error handling", () => {
  it("should handle git command failure", async () => {
    mockGit.execute.mockReturnValue(
      Result.error(new Error("Git failed"))
    );
    
    const result = await operation();
    assert.strictEqual(result.ok, false);
  });

  it("should handle file system errors", async () => {
    mockFs.access.mockRejectedValue(new Error("Permission denied"));
    
    const result = await operation();
    assert.strictEqual(result.ok, false);
  });
});

3. Mock Verification

Verify correct external calls:

it("should call git with correct arguments", async () => {
  await createWorktree("feature", "main");
  
  assert.strictEqual(mockGit.execute.mock.calls.length, 1);
  assert.deepStrictEqual(
    mockGit.execute.mock.calls[0].arguments[0],
    ["worktree", "add", "--detach", "/tmp/phantom/repo/feature", "main"]
  );
});

Test Data Management

Test Fixtures

Common test data patterns:

const fixtures = {
  validWorktree: {
    name: "feature-test",
    branch: "main",
    path: "/tmp/phantom/repo/feature-test"
  },
  
  gitWorktreeList: `worktree /path/to/main
HEAD abc123
branch refs/heads/main

worktree /tmp/phantom/repo/feature
HEAD def456
branch refs/heads/feature`,

  invalidNames: [
    "",
    ".",
    "..",
    "../escape",
    "with/slash",
    "with space"
  ]
};

Test Helpers

Reusable test utilities:

function createMockGitExecutor(responses: Record<string, Result<string>>) {
  return {
    execute: mock.fn((args: string[]) => {
      const key = args.join(" ");
      return responses[key] || Result.error(new Error("Not mocked"));
    })
  };
}

function assertResultOk<T>(result: Result<T>): asserts result is { ok: true; value: T } {
  assert.strictEqual(result.ok, true, "Expected ok result");
}

Coverage Strategy

Coverage Goals

  • Target: 80%+ code coverage
  • Focus: Critical paths and error handling
  • Exclude: Simple getters/setters, type definitions

Coverage Measurement

# Run tests with coverage
node --test --experimental-test-coverage

# Generate coverage report
node --test --experimental-test-coverage --test-reporter=lcov

Coverage Priorities

  1. High Priority:

    • Core business logic
    • Error handling paths
    • Data transformations
    • Validation logic

  2. Medium Priority:

    • CLI handlers
    • Output formatting
    • Helper utilities

  3. Low Priority:

    • Simple wrappers
    • Type definitions
    • Constants

Test Execution

Running Tests

# Run all tests
pnpm test

# Run specific test file
node --test src/core/worktree/create.test.ts

# Run with watch mode
node --test --watch

# Run with coverage
node --test --experimental-test-coverage

CI/CD Integration

Source: GitHub Actions workflow

Tests run on:

  • Every push
  • Every pull request
  • Multiple Node.js versions (v22, v24)
  • Multiple operating systems

Test Quality Guidelines

Good Test Characteristics

  1. Independent - No dependency on other tests
  2. Repeatable - Same result every time
  3. Self-Validating - Clear pass/fail
  4. Timely - Written with the code
  5. Focused - Test one thing

Test Naming Conventions

// Good: Descriptive behavior
it("should create worktree with custom branch name")
it("should return error when name contains invalid characters")
it("should cleanup resources on failure")

// Bad: Implementation focused
it("should call git command")
it("should set property")
it("should work")

Assertion Best Practices

// Good: Specific assertions
assert.strictEqual(result.ok, true);
assert.strictEqual(worktree.name, "feature");
assert.match(error.message, /already exists/);

// Bad: Generic assertions
assert(result.ok);
assert(worktree);
assert(error);

Debugging Tests

Test Isolation

// Use beforeEach for clean state
beforeEach(() => {
  mock.module("node:fs/promises", {
    namedExports: createMockFs()
  });
});

// Clean up after tests
afterEach(() => {
  mock.reset();
});

Debug Output

// Temporary debug logging
it("should handle complex scenario", async () => {
  const result = await operation();
  
  // Debug output
  console.log("Result:", JSON.stringify(result, null, 2));
  console.log("Mock calls:", mockGit.execute.mock.calls);
  
  assert.strictEqual(result.ok, true);
});

Future Testing Enhancements

1. Property-Based Testing

Generate random valid/invalid inputs:

it("should handle any valid worktree name", () => {
  for (const name of generateValidNames()) {
    const result = validateWorktreeName(name);
    assert.strictEqual(result.ok, true);
  }
});

2. Mutation Testing

Verify test quality by mutating code:

  • Change operators
  • Modify constants
  • Remove statements

3. Performance Testing

Add benchmarks for critical paths:

it("should complete within performance budget", async () => {
  const start = performance.now();
  await operation();
  const duration = performance.now() - start;
  
  assert(duration < 100, `Operation took ${duration}ms`);
});

Summary

Phantom's testing strategy ensures:

  1. Reliability - Comprehensive test coverage
  2. Maintainability - Clear, focused tests
  3. Speed - Fast test execution
  4. Confidence - Thorough error testing
  5. Simplicity - Native tools, no complexity

The focus on isolated unit tests with native Node.js tooling creates a robust, fast, and maintainable test suite.

⚠️ **GitHub.com Fallback** ⚠️