Screen Sharing Complete - nself-org/nchat GitHub Wiki
Version: v0.4.0 Status: ✅ Integrated and Ready Last Updated: January 30, 2026
The screen sharing implementation provides a comprehensive solution for sharing screens, windows, or tabs during calls, with advanced features including:
- Multi-source capture - Screen, window, or tab sharing
- System audio capture - Share audio from applications (Chrome/Edge)
- Quality controls - Adaptive quality from 720p to 4K
- Annotation tools - Draw, highlight, and annotate shared content
- Cursor tracking - Multi-user cursor highlighting with click effects
- Screen recording - Record screen shares with webcam overlay
- Real-time collaboration - Synchronized annotations across participants
src/lib/webrtc/
├── screen-capture.ts # Screen capture manager (getDisplayMedia API)
├── screen-annotator.ts # Drawing and annotation tools
├── cursor-highlighter.ts # Cursor tracking and highlighting
└── screen-recorder.ts # Screen recording with webcam overlay
src/hooks/
├── use-screen-share.ts # Screen sharing hook
└── use-screen-recording.ts # Recording management hook
src/components/call/
├── screen-share-panel.tsx # Main screen sharing UI
├── annotation-toolbar.tsx # Drawing tools toolbar
├── call-modal.tsx # Integrated call interface
└── call-controls.tsx # Call control buttons
Implementation: src/lib/webrtc/screen-capture.ts
- Screen - Full screen capture
- Window - Specific application window
- Tab - Browser tab only
| Quality | Resolution | Frame Rate | Bitrate |
|---|---|---|---|
| 720p | 1280x720 | 30 fps | 1.5 Mbps |
| 1080p | 1920x1080 | 30 fps | 2.5 Mbps |
| 4K | 3840x2160 | 60 fps | 8 Mbps |
| Auto | Adaptive | 30 fps | 2.5 Mbps |
- ✅ System audio capture (Chrome/Edge only)
- ✅ Cursor capture (always/motion/never)
- ✅ Dynamic quality adjustment
- ✅ Frame rate control (1-60 fps)
- ✅ Surface switching detection
- ✅ Automatic cleanup on track end
import { createScreenCaptureManager } from '@/lib/webrtc/screen-capture'
const captureManager = createScreenCaptureManager({
onStreamStarted: (stream) => {
console.log('Screen share started:', stream)
},
onStreamEnded: (streamId) => {
console.log('Screen share ended:', streamId)
},
onError: (error) => {
console.error('Screen capture error:', error)
},
})
// Start capturing
const share = await captureManager.startCapture(userId, userName, {
quality: '1080p',
captureSystemAudio: true,
captureCursor: true,
frameRate: 30,
})
// Update quality dynamically
await captureManager.updateQuality(share.id, '4k')
// Stop capturing
captureManager.stopCapture(share.id)Implementation: src/lib/webrtc/screen-annotator.ts
| Tool | Description | Keyboard Shortcut |
|---|---|---|
| Pen | Freehand drawing | P |
| Arrow | Draw arrows | A |
| Line | Draw straight lines | L |
| Rectangle | Draw rectangles | R |
| Circle | Draw circles | C |
| Text | Add text labels | T |
| Eraser | Remove annotations | E |
- Color - 10 preset colors + custom
- Stroke Width - 2px to 16px
- Font Size - 12px to 48px (text tool)
- Fill Mode - Solid or outline shapes
- ✅ Touch and mouse support
- ✅ Undo/redo functionality
- ✅ Clear all annotations
- ✅ Multi-user collaboration
- ✅ Annotation persistence
- ✅ Export annotations
import { createScreenAnnotator } from '@/lib/webrtc/screen-annotator'
const annotator = createScreenAnnotator({
canvas: canvasElement,
userId: 'user-123',
userName: 'John Doe',
onAnnotationAdded: (annotation) => {
// Broadcast annotation to other users
broadcastAnnotation(annotation)
},
})
// Change tool
annotator.setTool('arrow')
// Change color
annotator.setColor('#FF0000')
// Change stroke width
annotator.setStrokeWidth(4)
// Undo/redo
annotator.undo()
annotator.redo()
// Clear all
annotator.clear()
// Add remote annotation
annotator.addRemoteAnnotation(remoteAnnotation)Implementation: src/lib/webrtc/cursor-highlighter.ts
- ✅ Multi-user cursor tracking
- ✅ Unique color per user
- ✅ Animated cursor highlights
- ✅ Click effect animations
- ✅ User name labels
- ✅ Auto-fade for inactive cursors
import { createCursorHighlighter } from '@/lib/webrtc/cursor-highlighter'
const highlighter = createCursorHighlighter({
canvas: canvasElement,
highlightColor: '#FF4500',
highlightSize: 30,
showClickEffect: true,
clickEffectDuration: 500,
showUserName: true,
fadeOldCursors: true,
fadeTimeout: 3000,
})
// Update cursor position
highlighter.updateCursor({
x: 100,
y: 200,
userId: 'user-123',
userName: 'John Doe',
timestamp: Date.now(),
})
// Add click effect
highlighter.addClick({
x: 100,
y: 200,
userId: 'user-123',
timestamp: Date.now(),
})Implementation: src/lib/webrtc/screen-recorder.ts
- ✅ WebM and MP4 formats
- ✅ Webcam overlay (small/medium/large)
- ✅ Configurable position (4 corners)
- ✅ Pause/resume recording
- ✅ Quality presets (low/medium/high)
- ✅ Audio mixing (screen + webcam)
- ✅ Duration tracking
- ✅ File size monitoring
| Quality | Video Bitrate | Audio Bitrate |
|---|---|---|
| Low | 1 Mbps | 64 kbps |
| Medium | 2.5 Mbps | 128 kbps |
| High | 8 Mbps | 192 kbps |
import { createScreenRecorder } from '@/lib/webrtc/screen-recorder'
const recorder = createScreenRecorder({
onStart: () => console.log('Recording started'),
onStop: (recording) => {
console.log('Recording stopped:', recording)
// Download recording
downloadRecording(recording)
},
onDataAvailable: (data, size) => {
console.log('Data chunk:', size, 'bytes')
},
})
// Start recording with webcam overlay
await recorder.startRecording(screenStream, {
format: 'webm',
quality: 'high',
includeWebcam: true,
webcamSize: 'small',
webcamPosition: 'bottom-right',
})
// Pause/resume
recorder.pauseRecording()
recorder.resumeRecording()
// Stop and save
const recording = await recorder.stopRecording()Manages screen sharing lifecycle.
import { useScreenShare } from '@/hooks/use-screen-share'
function MyComponent() {
const {
isScreenSharing,
screenStream,
error,
startScreenShare,
stopScreenShare,
updateQuality,
isSupported,
supportsSystemAudio,
} = useScreenShare({
userId: 'user-123',
userName: 'John Doe',
useAdvancedCapture: true,
})
return (
<div>
{isSupported && (
<button onClick={() => startScreenShare({ quality: '1080p' })}>
Share Screen
</button>
)}
{isScreenSharing && (
<button onClick={stopScreenShare}>Stop Sharing</button>
)}
</div>
)
}Manages screen recording.
import { useScreenRecording } from '@/hooks/use-screen-recording'
function MyComponent() {
const {
isRecording,
isPaused,
duration,
recordings,
startRecording,
stopRecording,
pauseRecording,
resumeRecording,
formatDuration,
isSupported,
} = useScreenRecording()
return (
<div>
{isSupported && screenStream && (
<>
<button onClick={() => startRecording(screenStream)}>
Start Recording
</button>
{isRecording && (
<>
<span>{formatDuration(duration)}</span>
<button onClick={pauseRecording}>Pause</button>
<button onClick={stopRecording}>Stop</button>
</>
)}
</>
)}
</div>
)
}Main screen sharing interface with integrated controls.
import { ScreenSharePanel } from '@/components/call/screen-share-panel'
<ScreenSharePanel
userId="user-123"
userName="John Doe"
onStreamStarted={(stream) => console.log('Stream started')}
onStreamEnded={() => console.log('Stream ended')}
showAnnotations
showRecording
/>Standalone annotation toolbar.
import { AnnotationToolbar } from '@/components/call/annotation-toolbar'
<AnnotationToolbar
selectedTool={tool}
selectedColor={color}
strokeWidth={width}
onToolChange={setTool}
onColorChange={setColor}
onStrokeWidthChange={setWidth}
onUndo={handleUndo}
onRedo={handleRedo}
onClear={handleClear}
canUndo={canUndo}
canRedo={canRedo}
orientation="vertical"
/>| Browser | Screen | Window | Tab | System Audio | Status |
|---|---|---|---|---|---|
| Chrome 72+ | ✅ | ✅ | ✅ | ✅ | Full Support |
| Edge 79+ | ✅ | ✅ | ✅ | ✅ | Full Support |
| Firefox 66+ | ✅ | ✅ | ✅ | ❌ | No System Audio |
| Safari 13+ | ✅ | ✅ | ❌ | Limited Tab Support | |
| Opera 60+ | ✅ | ✅ | ✅ | ✅ | Full Support |
| Browser | WebM | MP4 | Quality Control | Status |
|---|---|---|---|---|
| Chrome 47+ | ✅ | ❌ | ✅ | Full Support |
| Edge 79+ | ✅ | ❌ | ✅ | Full Support |
| Firefox 29+ | ✅ | ❌ | ✅ | Full Support |
| Safari 14.1+ | ✅ | Limited Support | ||
| Opera 36+ | ✅ | ❌ | ✅ | Full Support |
-
Screen Sharing
- Permission:
display-capture - Prompt: "Share your screen"
- User action required: Yes
- Permission:
-
System Audio
- Permission:
display-capture(with audio) - Prompt: "Share system audio"
- Chrome/Edge only
- Permission:
-
Webcam (for recording overlay)
- Permission:
camera,microphone - Prompt: "Use camera and microphone"
- User action required: Yes
- Permission:
// Check if screen sharing is supported
if (ScreenCaptureManager.isSupported()) {
// Check if system audio is supported
if (supportsSystemAudio()) {
// Request screen share with system audio
await startScreenShare({
captureSystemAudio: true,
})
}
}
// Handle permission errors
try {
await startScreenShare()
} catch (error) {
if (error.name === 'NotAllowedError') {
// User denied permission
} else if (error.name === 'NotFoundError') {
// No screen/window selected
}
}// Automatic quality based on network conditions
import { getOptimalQuality } from '@/lib/webrtc/screen-capture'
// Get network info (if available)
const connection = navigator.connection
const downlink = connection?.downlink ?? 10 // Mbps
// Get optimal quality
const quality = getOptimalQuality(downlink)
// Apply quality
await startScreenShare({ quality })// Lower frame rate for better performance
await startScreenShare({
quality: '1080p',
frameRate: 15, // Lower FPS = less CPU usage
})
// Dynamic adjustment
await updateFrameRate(shareId, 30)// Stop all shares on component unmount
useEffect(() => {
return () => {
captureManager.cleanup()
recorder.destroy()
annotator.cleanup()
highlighter.cleanup()
}
}, [])- Start screen share (screen/window/tab)
- System audio capture (Chrome/Edge)
- Quality adjustment (720p/1080p/4K)
- Frame rate adjustment
- Annotation tools (all 7 tools)
- Color and stroke width selection
- Undo/redo annotations
- Clear all annotations
- Cursor tracking and clicks
- Multi-user cursors
- Start/stop recording
- Pause/resume recording
- Webcam overlay (all positions)
- Recording download
- Stop share (via button)
- Stop share (via browser UI)
- Permission denied handling
- No screen selected handling
Test in all supported browsers:
- Chrome 72+
- Edge 79+
- Firefox 66+
- Safari 13+
- Opera 60+
// Check browser support
if (!ScreenCaptureManager.isSupported()) {
console.error('Screen sharing not supported')
return
}
// Check permissions
try {
await startScreenShare()
} catch (error) {
console.error('Permission denied:', error)
}// Check browser support
if (!supportsSystemAudio()) {
console.warn('System audio not supported in this browser')
// Fallback: microphone only
}// Check MediaRecorder support
if (!ScreenRecorder.isSupported()) {
console.error('Recording not supported')
return
}
// Check supported MIME types
const mimeTypes = ScreenRecorder.getSupportedMimeTypes()
console.log('Supported formats:', mimeTypes)// Ensure annotations are broadcast to all users
annotator.onAnnotationAdded = (annotation) => {
// Send via WebSocket/SignalR
signaling.broadcastAnnotation(annotation)
}
// Receive remote annotations
signaling.onAnnotationReceived = (annotation) => {
annotator.addRemoteAnnotation(annotation)
}- Validate permissions - Always check if user has granted permissions
- Secure transmission - Use HTTPS/WSS for screen share streams
- User consent - Require explicit user action for screen share
- Privacy indicators - Show recording indicator when active
- Auto-stop - Stop sharing when call ends
// Show recording indicator
if (isRecording) {
return (
<div className="recording-indicator">
<span className="pulse" />
Recording
</div>
)
}
// Auto-stop on call end
useEffect(() => {
if (!activeCall && isScreenSharing) {
stopScreenShare()
}
}, [activeCall])- Collaborative whiteboard mode
- Laser pointer tool
- Spotlight effect (highlight area)
- Recording pause/resume with markers
- Auto-save recordings to cloud
- Real-time transcription
- AI-powered annotation suggestions
- Multi-screen simultaneous capture
- Advanced recording editor
- GIF export from recordings
class ScreenCaptureManager {
static isSupported(): boolean
startCapture(userId, userName, options): Promise<ScreenShare>
stopCapture(shareId): void
stopAllCaptures(): void
updateQuality(shareId, quality): Promise<void>
updateFrameRate(shareId, frameRate): Promise<void>
getVideoSettings(shareId): MediaTrackSettings | null
isActive(shareId): boolean
getActiveCount(): number
cleanup(): void
}class ScreenAnnotator {
setTool(tool: AnnotationTool): void
setColor(color: AnnotationColor): void
setStrokeWidth(width: number): void
setFontSize(size: number): void
setFilled(filled: boolean): void
undo(): void
redo(): void
clear(): void
addRemoteAnnotation(annotation: Annotation): void
getAnnotations(): Annotation[]
cleanup(): void
}class CursorHighlighter {
updateCursor(position: CursorPosition): void
addClick(click: CursorClick): void
removeCursor(userId: string): void
clearAllCursors(): void
setHighlightColor(color: string): void
setHighlightSize(size: number): void
setShowUserName(show: boolean): void
setShowClickEffect(show: boolean): void
cleanup(): void
}class ScreenRecorder {
static isSupported(): boolean
static getSupportedMimeTypes(): string[]
static getOptimalMimeType(format): string
startRecording(stream, options): Promise<void>
stopRecording(): Promise<Recording>
pauseRecording(): void
resumeRecording(): void
getState(): RecordingState
destroy(): void
}Built with:
- getDisplayMedia API - Screen capture
- MediaRecorder API - Recording
- Canvas API - Annotations and cursor tracking
- Web Audio API - Audio mixing
For issues and questions:
- GitHub Issues: https://github.com/yourusername/nself-chat/issues
- Documentation: https://docs.nself.org/features/screen-sharing
- Email: [email protected]
End of Documentation