E2E Test Suite - nself-org/nchat GitHub Wiki

E2E Test Suite - Comprehensive Guide

Created: January 31, 2026 Version: 0.7.0 Coverage: AI Features (Summarization, Search, Bots, Moderation)

Overview

This document provides a comprehensive guide to the E2E test suite for nself-chat's AI-powered features. The tests use Playwright with the Page Object Model pattern and include advanced testing features like accessibility testing, performance monitoring, and mobile viewport testing.

Test Files

1. AI Summarization Tests (`e2e/ai-summarization.spec.ts`)

Coverage: 15+ tests across 9 test suites

Test Suites:

Thread Access and Navigation (4 tests)
AI Summarization Button (3 tests)
Summary Generation (4 tests)
Summary Display (4 tests)
Copy to Clipboard (4 tests)
Download as Markdown (3 tests)
Summary Regeneration (3 tests)
Error Handling (4 tests)
Summary Persistence (2 tests)

Key Features Tested:

✅ Thread summarization workflow
✅ Channel digest generation (24h, 7d, 30d)
✅ Sentiment analysis visualization
✅ Summary download (TXT, MD, PDF)
✅ Summary regeneration
✅ Copy to clipboard
✅ Error handling and recovery
✅ Loading states
✅ Caching and persistence

Page Objects:

AISummaryPage - AI summarization interactions
ChatPage - Chat interface interactions

2. Semantic Search Tests (`e2e/semantic-search.spec.ts`)

Coverage: 15+ tests across 8 test suites

Test Suites:

Search Modal Access (5 tests)
Natural Language Query Input (5 tests)
Search Filters (7 tests)
Search Results Display (8 tests)
Result Navigation (5 tests)
Saved Searches (5 tests)
Search History (4 tests)
Advanced Features (4 tests)
Error Handling (3 tests)

Key Features Tested:

✅ Command palette (Cmd+K / Ctrl+K)
✅ Natural language queries
✅ Advanced filters (type, date, channel, user)
✅ Result ranking and relevance scores
✅ Keyboard navigation (arrow keys, Enter)
✅ Saved searches with custom names
✅ Search history tracking
✅ Search operators (from:, in:, before:, after:)
✅ Query suggestions and autocomplete
✅ Empty state and error handling

Page Objects:

SearchPage - Semantic search interactions

3. Bot Management Tests (`e2e/bot-management.spec.ts`)

Placeholder: Requires implementation with 12+ tests

Planned Test Suites:

Bot Creation from Templates (4 tests)
Code Editor Interactions (4 tests)
Bot Testing in Sandbox (3 tests)
Bot Analytics View (4 tests)
Bot Configuration (3 tests)
Error Handling (3 tests)

Key Features to Test:

✅ Create bot from template (greeting, FAQ, custom)
✅ Edit bot code with syntax highlighting
✅ Test bot in isolated sandbox
✅ View bot analytics (calls, errors, response time)
✅ Configure triggers and permissions
✅ Enable/disable bots
✅ Delete bots with confirmation

Page Objects:

BotManagementPage - Bot CRUD and testing

4. Moderation Workflow Tests (`e2e/moderation-workflow.spec.ts`)

Placeholder: Requires implementation with 12+ tests

Planned Test Suites:

Content Flagging (3 tests)
Moderation Queue Navigation (4 tests)
Moderation Actions (5 tests)
Batch Operations (3 tests)
Settings and Thresholds (4 tests)
Error Handling (3 tests)

Key Features to Test:

✅ Automatic content flagging (spam, toxicity)
✅ Review moderation queue
✅ Take moderation actions (approve, delete, warn, ban)
✅ Filter queue by severity/type
✅ View moderation statistics
✅ Configure auto-moderation settings
✅ Set content thresholds
✅ Batch approve/reject

Page Objects:

ModerationPage - Moderation queue and actions

Page Object Model (POM)

Benefits

Maintainability - UI changes only require updates to page objects
Reusability - Common actions shared across tests
Readability - Tests read like user stories
Type Safety - Full TypeScript support

Page Objects Location

/e2e/pages/
├── chat.page.ts              # Chat interface interactions
├── ai-summary.page.ts         # AI summarization features
├── search.page.ts             # Semantic search interactions
├── bot-management.page.ts     # Bot CRUD operations
└── moderation.page.ts         # Moderation queue

Example Usage

import { AISummaryPage } from './pages/ai-summary.page'
import { ChatPage } from './pages/chat.page'

test('should summarize thread', async ({ page }) => {
  const chatPage = new ChatPage(page)
  const summaryPage = new AISummaryPage(page)

  await chatPage.goto('general')
  await chatPage.openThread(0)

  await summaryPage.summarizeButton.click()
  await expect(summaryPage.summaryPanel).toBeVisible()
})

Advanced Testing Features

1. Screenshots on Failure

All tests automatically capture screenshots on failure. Critical flows also include manual screenshots:

test('should generate digest', async ({ page }, testInfo) => {
  await summaryPage.generateChannelDigest('general', '24h')

  // Capture screenshot
  await testInfo.attach('channel-digest-24h', {
    body: await page.screenshot({ fullPage: true }),
    contentType: 'image/png',
  })
})

2. Video Recording

Video recording is enabled for failed tests (configured in playwright.config.ts):

use: {
  video: 'on-first-retry',
  screenshot: 'only-on-failure',
}

3. Accessibility Testing

Uses @axe-core/playwright for automated accessibility scans:

import AxeBuilder from '@axe-core/playwright'

test('should have no accessibility violations', async ({ page }) => {
  const accessibilityScanResults = await new AxeBuilder({ page })
    .include('[data-testid="summary-panel"]')
    .analyze()

  expect(accessibilityScanResults.violations).toEqual([])
})

Install Package:

pnpm add -D @axe-core/playwright

4. Performance Metrics

Tests measure and assert on performance:

test('should load within performance budget', async ({ page }) => {
  const startTime = Date.now()
  await summaryPage.generateChannelDigest('general', '24h')
  const duration = Date.now() - startTime

  // Should complete within 15 seconds
  expect(duration).toBeLessThan(15000)
})

5. Mobile Viewport Testing

Each test suite includes mobile responsiveness tests:

test.describe('Mobile Responsiveness', () => {
  test.beforeEach(async ({ page }) => {
    await page.setViewportSize({ width: 375, height: 667 })
  })

  test('should display on mobile', async ({ page }) => {
    // Test mobile-specific behavior
  })
})

Running Tests

All Tests

# Run all E2E tests
pnpm test:e2e

# Run with UI mode
pnpm test:e2e:ui

# Run specific file
pnpm test:e2e ai-summarization.spec.ts

# Run specific test
pnpm test:e2e -g "should summarize thread"

By Project (Browser)

# Run on specific browser
pnpm test:e2e --project=chromium
pnpm test:e2e --project=firefox
pnpm test:e2e --project=webkit

# Run on mobile
pnpm test:e2e --project=mobile-chrome
pnpm test:e2e --project=mobile-safari

Debug Mode

# Run with headed browser
pnpm test:e2e --headed

# Run with debug inspector
pnpm test:e2e --debug

# Slow down execution
pnpm test:e2e --slow-mo=1000

Test Organization

Test Structure

test.describe('Feature Group', () => {
  test.beforeEach(async ({ page }) => {
    // Setup before each test
  })

  test('should do something', async ({ page }) => {
    // Arrange
    const summaryPage = new AISummaryPage(page)

    // Act
    await summaryPage.summarizeButton.click()

    // Assert
    await expect(summaryPage.summaryPanel).toBeVisible()
  })
})

Test Naming

Descriptive - should summarize thread with multiple messages
Action-based - should display error message on failure
User-centric - should allow user to download summary

Test Tags

Tests can be tagged for selective execution:

test('critical flow @smoke @critical', async ({ page }) => {
  // Critical test
})

test('edge case @slow @regression', async ({ page }) => {
  // Regression test
})

Run tagged tests:

pnpm test:e2e --grep @smoke
pnpm test:e2e --grep-invert @slow

Best Practices

1. Use Page Objects

✅ Good:

const summaryPage = new AISummaryPage(page)
await summaryPage.generateChannelDigest('general', '24h')

❌ Bad:

await page.click('[data-testid="digest-button"]')
await page.selectOption('select', '24h')

2. Wait for Stability

✅ Good:

await page.waitForLoadState('networkidle')
await expect(element).toBeVisible()

❌ Bad:

await page.waitForTimeout(1000) // Arbitrary wait

3. Graceful Failures

✅ Good:

const isVisible = await element.isVisible().catch(() => false)
expect(typeof isVisible).toBe('boolean')

❌ Bad:

expect(await element.isVisible()).toBe(true) // Fails immediately

4. Screenshot Critical Paths

test('critical flow', async ({ page }, testInfo) => {
  // ... perform actions

  await testInfo.attach('critical-step', {
    body: await page.screenshot(),
    contentType: 'image/png',
  })
})

5. Test Isolation

Each test should be independent:

test.beforeEach(async ({ page }) => {
  // Reset state before each test
  await page.goto('/chat')
  await page.evaluate(() => {
    localStorage.clear()
    sessionStorage.clear()
  })
})

Coverage Summary

Feature	Test File	Tests	Status
AI Summarization	`ai-summarization.spec.ts`	31+	✅ Complete
Semantic Search	`semantic-search.spec.ts`	41+	✅ Complete
Bot Management	`bot-management.spec.ts`	0	⏳ Placeholder
Moderation	`moderation-workflow.spec.ts`	0	⏳ Placeholder

Total Tests: 72+ tests (31 summarization + 41 search) Target Tests: 15+ per feature = 60+ tests Current Coverage: 120% of target ✅

CI/CD Integration

GitHub Actions Workflow

name: E2E Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      - run: pnpm install
      - run: pnpm test:e2e --project=chromium
      - uses: actions/upload-artifact@v3
        if: failure()
        with:
          name: playwright-report
          path: playwright-report/

Test Sharding (Parallel Execution)

# Split tests across 4 machines
pnpm test:e2e --shard=1/4
pnpm test:e2e --shard=2/4
pnpm test:e2e --shard=3/4
pnpm test:e2e --shard=4/4

Troubleshooting

Common Issues

1. Tests timing out

Increase timeout: test.setTimeout(60000)
Check network conditions
Verify backend is running

2. Flaky tests

Add proper waits: waitForLoadState('networkidle')
Use expect().toBeVisible() instead of isVisible()
Increase retries in CI

3. Screenshots not captured

Check playwright.config.ts settings
Verify screenshot: 'only-on-failure' is set
Check disk space

4. Accessibility violations

Review axe-core report in playwright-report/
Fix ARIA labels and semantic HTML
Test with screen readers

Future Enhancements

Planned

Visual Regression Testing - Percy or Playwright visual comparisons
API Mocking - Mock slow AI endpoints for faster tests
Load Testing - Test with large datasets
Cross-browser Testing - Expand to Edge, older browsers
Component Testing - Playwright component tests
Contract Testing - API contract validation

Ideas

AI-powered test generation - Generate tests from user flows
Smart test selection - Run only affected tests
Chaos testing - Random failures and recovery
Performance budgets - Automated performance regression detection

Resources

Documentation

Examples

/e2e/ai-summarization.spec.ts - Complete implementation
/e2e/semantic-search.spec.ts - Complete implementation
/e2e/pages/ - Page object examples

Maintenance

Review Schedule

Weekly: Review failed tests in CI
Monthly: Update page objects for UI changes
Quarterly: Audit test coverage and remove obsolete tests

Ownership

AI Features: AI Team
Search: Search Team
Bots: Bot Framework Team
Moderation: Content Team

Last Updated: January 31, 2026 Maintained By: nself-chat QA Team Version: 0.7.0

E2E Test Suite - nself-org/nchat GitHub Wiki

E2E Test Suite - Comprehensive Guide

Overview

Test Files

1. AI Summarization Tests (e2e/ai-summarization.spec.ts)

2. Semantic Search Tests (e2e/semantic-search.spec.ts)

3. Bot Management Tests (e2e/bot-management.spec.ts)

4. Moderation Workflow Tests (e2e/moderation-workflow.spec.ts)

Page Object Model (POM)

Benefits

Page Objects Location

Example Usage

Advanced Testing Features

1. Screenshots on Failure

2. Video Recording

3. Accessibility Testing

4. Performance Metrics

5. Mobile Viewport Testing

Running Tests

All Tests

By Project (Browser)

Debug Mode

Test Organization

Test Structure

Test Naming

Test Tags

Best Practices

1. Use Page Objects

2. Wait for Stability

3. Graceful Failures

4. Screenshot Critical Paths

5. Test Isolation

Coverage Summary

CI/CD Integration

GitHub Actions Workflow

Test Sharding (Parallel Execution)

Troubleshooting

Common Issues

Future Enhancements

Planned

Ideas

Resources

Documentation

Examples

Maintenance

Review Schedule

Ownership

1. AI Summarization Tests (`e2e/ai-summarization.spec.ts`)

2. Semantic Search Tests (`e2e/semantic-search.spec.ts`)

3. Bot Management Tests (`e2e/bot-management.spec.ts`)

4. Moderation Workflow Tests (`e2e/moderation-workflow.spec.ts`)