Voice Video Calls Implementation - nself-org/nchat GitHub Wiki

Voice/Video Calls - Complete Implementation Guide

Overview

This document provides a comprehensive guide to the complete, production-ready voice and video call implementation in nself-chat, including all features from the requirements.

Implementation Status

✅ COMPLETED Components

  1. WebRTC Infrastructure (/Users/admin/Sites/nself-chat/src/lib/webrtc/)

    • ✅ peer-connection.ts - RTCPeerConnection manager
    • ✅ media-manager.ts - Media stream management
    • ✅ signaling.ts - Socket.io signaling
    • ✅ servers.ts - STUN/TURN server configuration
    • ✅ call-manager.ts - NEW Complete call lifecycle orchestration

  2. Call State Management (/Users/admin/Sites/nself-chat/src/stores/)

    • ✅ call-store.ts - Zustand store for call state

  3. Call State Machine (/Users/admin/Sites/nself-chat/src/lib/calls/)

    • ✅ call-state-machine.ts - State transitions
    • ✅ call-invitation.ts - Invitation management with ringtone
    • ✅ audio-processor.ts - Noise suppression, echo cancellation, VAD
    • ✅ background-blur.ts - Background blur using MediaPipe

  4. UI Components (/Users/admin/Sites/nself-chat/src/components/calls/)

    • ✅ VideoControls.tsx - Control bar
    • ✅ VideoCallModal.tsx - Video call interface
    • ✅ audio-call.tsx - Audio-only UI
    • ✅ IncomingCallModal.tsx - NEW Incoming call UI with ringing

  5. Hooks (/Users/admin/Sites/nself-chat/src/hooks/)

    • ✅ use-call.ts - Main call hook

  6. GraphQL Operations (/Users/admin/Sites/nself-chat/src/graphql/)

    • ✅ calls.ts - All mutations, queries, subscriptions

  7. Type Definitions (/Users/admin/Sites/nself-chat/src/types/)

    • ✅ calls.ts - Comprehensive type definitions

Features Checklist

✅ Voice Calls (1:1)

  • Initiate voice call
  • Accept/reject incoming call
  • Call ringing UI with ringtone
  • Call in progress UI
  • Mute/unmute microphone
  • Speaker/earpiece toggle
  • Call volume controls
  • Call timer
  • End call
  • Call history
  • Missed call notifications

✅ Video Calls (1:1)

  • Initiate video call
  • Accept/reject incoming call
  • Call ringing UI
  • Call in progress UI
  • Camera on/off
  • Switch camera (front/back)
  • Video layouts (speaker view, grid view)
  • Call timer
  • End call

✅ Advanced Features

  • Screen sharing
  • Background blur for video
  • Noise suppression
  • Echo cancellation
  • Automatic gain control
  • Voice activity detection (VAD)
  • Network quality indicator
  • Reconnection handling
  • Call waiting
  • Picture-in-picture support

⏳ Partial/Not Implemented

  • Group calls (infrastructure ready, needs UI)
  • Virtual backgrounds (requires additional processing)
  • Call recording
  • Call transfer

Architecture

┌─────────────────────────────────────────────────────────────┐
│                        User Interface                        │
├─────────────────────────────────────────────────────────────┤
│  IncomingCallModal  │  VideoCallModal  │  AudioCall         │
└─────────────────────────────────────────────────────────────┘
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                          Hooks Layer                         │
├─────────────────────────────────────────────────────────────┤
│              use-call.ts (Main Integration)                  │
└─────────────────────────────────────────────────────────────┘
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                     State Management                         │
├─────────────────────────────────────────────────────────────┤
│      call-store.ts (Zustand) + CallStateMachine             │
└─────────────────────────────────────────────────────────────┘
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                    Call Manager (NEW)                        │
├─────────────────────────────────────────────────────────────┤
│         Orchestrates all call lifecycle operations          │
└─────────────────────────────────────────────────────────────┘
                              ▼
┌──────────────┬──────────────┬──────────────┬───────────────┐
│ Peer         │ Media        │ Signaling    │ Invitation    │
│ Connection   │ Manager      │ Manager      │ Manager       │
└──────────────┴──────────────┴──────────────┴───────────────┘
       ▼              ▼              ▼              ▼
┌──────────────┬──────────────┬──────────────┬───────────────┐
│ WebRTC       │ getUserMedia │ Socket.io    │ Ringtone      │
│ API          │ getDisplay   │ Events       │ + Vibration   │
└──────────────┴──────────────┴──────────────┴───────────────┘

Usage Examples

1. Initiate a Voice Call

import { useCall } from '@/hooks/use-call'

function MyComponent() {
  const { initiateVoiceCall } = useCall()

  const handleCall = async () => {
    await initiateVoiceCall(
      'user-id-123',
      'John Doe',
      'channel-id-optional'
    )
  }

  return <button onClick={handleCall}>Call John</button>
}

2. Handle Incoming Calls

import { useCall } from '@/hooks/use-call'
import { IncomingCallModal } from '@/components/calls/IncomingCallModal'

function CallContainer() {
  const { incomingCalls, acceptCall, declineCall } = useCall()

  return (
    <>
      {incomingCalls.map((call) => (
        <IncomingCallModal
          key={call.id}
          callId={call.id}
          callerId={call.callerId}
          callerName={call.callerName}
          callerAvatarUrl={call.callerAvatarUrl}
          callType={call.type}
          onAccept={(withVideo) => acceptCall(call.id)}
          onDecline={() => declineCall(call.id)}
        />
      ))}
    </>
  )
}

3. Video Call with Controls

import { useCall } from '@/hooks/use-call'
import { VideoCallModal } from '@/components/calls/VideoCallModal'

function VideoCall() {
  const {
    isInCall,
    callType,
    isMuted,
    isVideoEnabled,
    isScreenSharing,
    localStream,
    remoteStreams,
    toggleMute,
    toggleVideo,
    toggleScreenShare,
    endCall,
  } = useCall()

  if (!isInCall || callType !== 'video') return null

  return (
    <VideoCallModal
      userId="current-user-id"
      userName="Current User"
      onClose={endCall}
    />
  )
}

Environment Variables

Add these to your .env.local:

# STUN Servers (Free Google STUN - already configured)
# No configuration needed for basic peer-to-peer calls

# TURN Servers (Required for calls behind strict NAT/Firewall)
NEXT_PUBLIC_TURN_SERVER_URL=turn:your-turn-server.com:3478
NEXT_PUBLIC_TURN_USERNAME=your-username
NEXT_PUBLIC_TURN_CREDENTIAL=your-password

# Additional TURN servers (comma-separated)
NEXT_PUBLIC_TURN_ADDITIONAL_URLS=turn:backup1.com:3478,turn:backup2.com:3478

# Force TURN (for testing)
NEXT_PUBLIC_FORCE_TURN=false

# Call Configuration
NEXT_PUBLIC_CALL_RING_TIMEOUT=30000
NEXT_PUBLIC_CALL_RECONNECT_ATTEMPTS=5

TURN Server Setup

For production, you MUST have a TURN server. Here are the options:

Option 1: Managed TURN Services

Option 2: Self-Hosted Coturn

# Install coturn
sudo apt-get install coturn

# Configure /etc/turnserver.conf
listening-port=3478
fingerprint
lt-cred-mech
use-auth-secret
static-auth-secret=your-secret-key
realm=your-domain.com
total-quota=100
stale-nonce=600
cert=/path/to/cert.pem
pkey=/path/to/key.pem

# Start service
sudo systemctl enable coturn
sudo systemctl start coturn

Test TURN Server

# Use the built-in test function
import { testTurnServer } from '@/lib/webrtc/servers'

const result = await testTurnServer({
  urls: 'turn:your-server.com:3478',
  username: 'user',
  credential: 'pass',
})

console.log(result) // { success: true, latencyMs: 123 }

Database Schema

Ensure these tables exist in your PostgreSQL database:

-- Calls table
CREATE TABLE nchat_calls (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  call_id TEXT UNIQUE NOT NULL,
  type TEXT NOT NULL, -- 'voice' | 'video'
  status TEXT NOT NULL, -- 'ringing' | 'active' | 'ended' | 'missed'
  caller_id UUID NOT NULL REFERENCES nchat_users(id),
  channel_id UUID REFERENCES nchat_channels(id),
  started_at TIMESTAMPTZ,
  ended_at TIMESTAMPTZ,
  duration INTEGER, -- seconds
  metadata JSONB DEFAULT '{}',
  created_at TIMESTAMPTZ DEFAULT NOW(),
  updated_at TIMESTAMPTZ DEFAULT NOW()
);

-- Call participants
CREATE TABLE nchat_call_participants (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  call_id TEXT NOT NULL REFERENCES nchat_calls(call_id) ON DELETE CASCADE,
  user_id UUID NOT NULL REFERENCES nchat_users(id),
  joined_at TIMESTAMPTZ,
  left_at TIMESTAMPTZ,
  is_muted BOOLEAN DEFAULT FALSE,
  is_video_enabled BOOLEAN DEFAULT FALSE,
  is_screen_sharing BOOLEAN DEFAULT FALSE,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  CONSTRAINT call_participants_call_user_unique UNIQUE(call_id, user_id)
);

-- Indexes
CREATE INDEX idx_calls_caller ON nchat_calls(caller_id);
CREATE INDEX idx_calls_status ON nchat_calls(status);
CREATE INDEX idx_calls_channel ON nchat_calls(channel_id);
CREATE INDEX idx_call_participants_user ON nchat_call_participants(user_id);
CREATE INDEX idx_call_participants_call ON nchat_call_participants(call_id);

WebRTC Configuration

Optimal Media Constraints

// Voice Call
const audioConstraints = {
  echoCancellation: true,
  noiseSuppression: true,
  autoGainControl: true,
  sampleRate: 48000,
  channelCount: 1,
}

// Video Call - HD Quality
const videoConstraints = {
  width: { ideal: 1280 },
  height: { ideal: 720 },
  frameRate: { ideal: 30 },
  facingMode: 'user',
}

// Video Call - Low Bandwidth
const lowBandwidthVideo = {
  width: { ideal: 640 },
  height: { ideal: 480 },
  frameRate: { ideal: 15 },
}

ICE Configuration

const iceConfig = {
  iceServers: [
    { urls: 'stun:stun.l.google.com:19302' },
    {
      urls: 'turn:your-turn.com:3478',
      username: 'user',
      credential: 'pass',
    },
  ],
  iceTransportPolicy: 'all', // 'all' | 'relay'
  bundlePolicy: 'max-bundle',
  rtcpMuxPolicy: 'require',
}

Troubleshooting

No Audio/Video

  1. Check Permissions

    const permissions = await navigator.mediaDevices.getUserMedia({
  audio: true,
  video: true,
})

  2. Check Device Enumeration

    const devices = await navigator.mediaDevices.enumerateDevices()
console.log(
  'Audio inputs:',
  devices.filter((d) => d.kind === 'audioinput')
)
console.log(
  'Video inputs:',
  devices.filter((d) => d.kind === 'videoinput')
)

Connection Fails

  1. Test STUN/TURN

    import { testAllServers } from '@/lib/webrtc/servers'
const results = await testAllServers()
console.log(results.summary)

  2. Check ICE Candidates

    • Enable verbose logging in peer-connection.ts
    • Check for 'srflx' (STUN) and 'relay' (TURN) candidates

  3. Firewall Issues

    • Ensure ports 3478 (TURN/STUN) are open
    • For UDP, ports 49152-65535 should be open

Echo/Feedback

  • Enable echo cancellation in audio constraints
  • Check that remote audio is not playing through speakers near microphone
  • Use headphones

Poor Quality

  1. Reduce Video Quality

    await mediaManager.applyVideoConstraints({
  width: { ideal: 640 },
  height: { ideal: 480 },
  frameRate: { ideal: 15 },
})

  2. Monitor Stats

    const stats = await peerConnection.getConnectionStats()
console.log('Packet loss:', stats.packetsLost)
console.log('RTT:', stats.roundTripTime)

Performance Optimization

1. Lazy Loading

Only load WebRTC modules when needed:

const loadCallManager = async () => {
  const { createCallManager } = await import('@/lib/webrtc/call-manager')
  return createCallManager(config)
}

2. Adaptive Bitrate

Automatically adjust quality based on network conditions:

import { BandwidthManager } from '@/lib/calls/bandwidth-manager'

const bandwidthManager = new BandwidthManager()
bandwidthManager.on('quality-change', (quality) => {
  // Adjust video constraints
})

3. Simulcast

For group calls, enable simulcast to send multiple quality streams:

import { setupSimulcast } from '@/lib/calls/simulcast'

await setupSimulcast(peerConnection, stream)

Security Considerations

1. TURN Credentials

  • Never expose TURN credentials in client code
  • Use a server endpoint to generate time-limited credentials
  • Implement proper authentication before providing credentials

2. Signaling Security

  • All signaling goes through authenticated Socket.io connections
  • Validate user permissions before allowing calls
  • Implement rate limiting on call initiation

3. Media Security

  • All media is encrypted via WebRTC (SRTP)
  • Use HTTPS and WSS in production
  • Consider end-to-end encryption for sensitive calls

Testing

Unit Tests

# Test WebRTC components
pnpm test src/lib/webrtc

# Test call components
pnpm test src/components/calls

# Test hooks
pnpm test src/hooks/use-call

E2E Tests

// e2e/calls.spec.ts
import { test, expect } from '@playwright/test'

test('initiate voice call', async ({ page, context }) => {
  // Login as user 1
  await page.goto('/chat')
  await page.click('[data-testid="call-button"]')

  // Open second window as user 2
  const page2 = await context.newPage()
  await page2.goto('/chat')

  // Accept call
  await page2.click('[data-testid="accept-call"]')

  // Verify call is active
  await expect(page.locator('[data-testid="call-timer"]')).toBeVisible()
})

Browser Compatibility

Feature Chrome Firefox Safari Edge
Voice Calls ✅ ✅ ✅ ✅
Video Calls ✅ ✅ ✅ ✅
Screen Share ✅ ✅ ✅ ✅
Background Blur ✅ ⚠️ ❌ ✅
Virtual BG ✅ ❌ ❌ ✅

⚠️ = Partial support ❌ = Not supported

Next Steps

  1. Deploy TURN Server - Critical for production
  2. Test at Scale - Load testing with multiple concurrent calls
  3. Implement Call Recording - Server-side recording infrastructure
  4. Add Group Calls UI - Build on existing infrastructure
  5. Mobile Optimization - Test and optimize for mobile browsers
  6. Analytics - Track call quality, duration, success rates

Support

For issues or questions:

  1. Check console logs for detailed error messages
  2. Use the built-in diagnostic tools (testAllServers, getConnectionStats)
  3. Review WebRTC internals at chrome://webrtc-internals/
  4. Check the comprehensive type definitions in /src/types/calls.ts

Implementation Date: February 1, 2026 Status: Production Ready ✅ Test Coverage: Core features tested Performance: Optimized for 1:1 calls

nself-chat v0.3.0 | GitHub | Issues | Discussions | Demo

Edit this page | MIT License | © 2026

⚠️ **GitHub.com Fallback** ⚠️