FILE HANDLING GUIDE - nself-org/nchat GitHub Wiki
Version: 0.9.1 Last Updated: February 3, 2026 Status: Production Ready (95%)
- Overview
- Architecture
- Features
- Security
- API Reference
- Usage Examples
- Configuration
- Storage Quotas
- Processing Pipeline
- Testing
- Known Limitations
The Ι³Chat file handling system provides production-ready file upload, download, storage, and processing capabilities with enterprise-grade security and access control.
- Upload: Direct uploads, presigned URLs, multipart for large files
- Download: Secure signed URLs with expiration
- Processing: Thumbnails, optimization, metadata extraction
- Security: Virus scanning, EXIF stripping, access control
- Storage: S3/MinIO compatible, CDN-ready
- Quotas: Role-based storage limits
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Client Layer β
β Components: FileUploader, FilePreview, ImageGallery β
ββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββ
β
ββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββ
β API Routes β
β /api/files/upload, /api/files/[id]/download β
β /api/files/[id]/thumbnails, /api/files/complete β
ββββββββββββββββββββββ¬βββββββββββββββββββββββββββββββββββββββββ
β
ββββββββββββββββββββββΌβββββββββββββββββββββββββββββββββββββββββ
β Service Layer β
β UploadService, DownloadService, ProcessingService β
β ValidationService, FileAccessService β
βββββββ¬βββββββββββββββ¬βββββββββββββββ¬βββββββββββββββββ¬βββββββββ
β β β β
βββββββΌββββββ ββββββΌββββββ ββββββΌβββββββ βββββββΌβββββββββ
β S3/MinIO β β Hasura β β File Proc β β Database β
β Storage β β GraphQL β β Plugin β β (Postgres) β
βββββββββββββ ββββββββββββ βββββββββββββ ββββββββββββββββ
| Service | Purpose | File |
|---|---|---|
| UploadService | Upload files, presigned URLs, multipart | upload.service.ts |
| DownloadService | Signed URLs, streaming, thumbnails | download.service.ts |
| ProcessingService | Plugin integration, job tracking | processing.service.ts |
| ValidationService | File type, size, virus scan, EXIF | validation.service.ts |
| FileAccessService | RBAC, channel access, quotas | access.service.ts |
-
Direct File Upload
- Single and multipart uploads
- Progress tracking
- Content hash for deduplication
- Automatic MIME type detection
-
Presigned Upload URLs
- Client-side direct-to-S3 uploads
- Configurable expiration
- Reduced server load
-
Secure Downloads
- Signed URLs with expiration
- Inline or attachment disposition
- Access control enforcement
-
File Validation
- Size limits by user tier
- MIME type restrictions
- Dangerous file detection
- Extension validation
-
Access Control
- Role-based permissions
- Channel membership checks
- File ownership verification
- Guest upload restrictions
-
Storage Management
- S3/MinIO compatibility
- Multipart for large files (>5MB)
- CDN-ready with cache headers
- Path-based organization
-
File Processing Integration
- Thumbnail generation
- Metadata extraction
- Job status tracking
- Webhook callbacks
-
Database Integration
- GraphQL queries and mutations
- Real-time subscriptions
- Soft delete support
- Storage usage tracking
-
Virus Scanning
- Interface ready
- Needs ClamAV integration
- Placeholder returns clean
-
EXIF Stripping
- Interface ready
- Needs image library integration
- Placeholder returns original
-
Storage Quotas
- Limits defined
- Enforcement not active
- Usage tracking needed
// Check file access
const accessService = getFileAccessService()
const result = await accessService.canAccessFile(userId, fileId, userRole)
if (!result.allowed) {
throw new Error(result.reason)
}Access Rules:
- Owner/Admin: Access to all files
- Moderator: Access to channel files
- Member: Access to own files + channel files if member
- Guest: Read-only access to public channel files
// Validate before upload
const validation = validateFile(file, {
maxSize: 25 * 1024 * 1024, // 25MB
allowedTypes: ['image/*', 'video/*'],
blockedExtensions: ['exe', 'bat', 'cmd'],
userTier: 'member',
})
if (!validation.valid) {
throw new Error(validation.error)
}Security Measures:
- β File size limits (5MB - 500MB based on tier)
- β MIME type validation
- β Extension blacklist
- β Dangerous file detection
- β Filename sanitization
β οΈ Virus scanning (placeholder)β οΈ EXIF stripping (placeholder)- β SSRF protection in unfurling
All download URLs are signed and expire after 1 hour (configurable):
const { url, expiresAt } = await downloadService.getDownloadUrl({
fileId,
expiresIn: 3600, // 1 hour
disposition: 'inline',
})Upload a file directly or get a presigned URL.
Request (Direct Upload):
const formData = new FormData()
formData.append('file', file)
formData.append('channelId', channelId)
const response = await fetch('/api/files/upload', {
method: 'POST',
body: formData,
})Request (Presigned URL):
const response = await fetch('/api/files/upload', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
fileName: 'photo.jpg',
mimeType: 'image/jpeg',
size: 1024000,
channelId: 'uuid',
}),
})Response:
{
"fileId": "uuid",
"fileName": "photo.jpg",
"mimeType": "image/jpeg",
"size": 1024000,
"storagePath": "channel-id/user-id/file-id.jpg",
"url": "https://storage.example.com/...",
"uploadUrl": "https://storage.example.com/presigned?...",
"expiresAt": "2026-02-03T19:00:00Z"
}Get a secure download URL for a file.
Request:
const response = await fetch(`/api/files/${fileId}/download?expiresIn=3600&disposition=inline`)Query Parameters:
-
expiresIn- URL expiration in seconds (default: 3600, max: 86400) -
disposition-inlineorattachment(default: inline) -
filename- Override download filename
Response:
{
"url": "https://storage.example.com/signed?...",
"expiresAt": "2026-02-03T19:00:00Z",
"contentType": "image/jpeg",
"filename": "photo.jpg",
"size": 1024000
}Get thumbnails for an image or video.
Response:
[
{
"id": "uuid",
"fileId": "uuid",
"path": "thumbnails/file-id/100.jpeg",
"url": "https://storage.example.com/...",
"width": 100,
"height": 100,
"size": 5000,
"format": "jpeg"
},
{
"id": "uuid",
"fileId": "uuid",
"path": "thumbnails/file-id/400.jpeg",
"url": "https://storage.example.com/...",
"width": 400,
"height": 400,
"size": 20000,
"format": "jpeg"
}
]Finalize file upload after client-side upload to presigned URL.
Request:
const response = await fetch('/api/files/complete', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
fileId: 'uuid',
storagePath: 'uploads/file-id.jpg',
channelId: 'uuid',
messageId: 'uuid',
}),
})Webhook endpoint for file-processing plugin callbacks.
import { getUploadService } from '@/services/files'
const uploadService = getUploadService()
const result = await uploadService.uploadFile(
{
file,
channelId,
messageId,
},
(progress) => {
console.log(`${progress.progress}% - ${progress.status}`)
console.log(`Speed: ${formatBytes(progress.speed)}/s`)
console.log(`ETA: ${progress.timeRemaining}s`)
}
)
console.log('Uploaded:', result.file.url)const files = [file1, file2, file3, file4, file5]
const requests = files.map((file) => ({
file,
channelId,
}))
const results = await uploadService.uploadFiles(requests, (fileId, progress) => {
console.log(`File ${fileId}: ${progress.progress}%`)
})
console.log(`Uploaded ${results.length} files`)// Step 1: Get presigned URL
const { uploadUrl, fileId, storagePath } = await fetch('/api/files/upload', {
method: 'POST',
body: JSON.stringify({
fileName: file.name,
mimeType: file.type,
size: file.size,
channelId,
}),
}).then((r) => r.json())
// Step 2: Upload directly to S3
await fetch(uploadUrl, {
method: 'PUT',
body: file,
headers: {
'Content-Type': file.type,
},
})
// Step 3: Finalize
await fetch('/api/files/complete', {
method: 'POST',
body: JSON.stringify({
fileId,
storagePath,
channelId,
messageId,
}),
})import { getDownloadService } from '@/services/files'
const downloadService = getDownloadService()
// Get signed URL
const { url } = await downloadService.getSignedDownloadUrl(storagePath, userId, {
userRole: 'member',
expiresIn: 3600,
disposition: 'attachment',
filename: 'download.pdf',
})
// Access check is automatic
if (url) {
window.location.href = url
}const { data } = await client.query({
query: GET_USER_STORAGE_USAGE,
variables: { userId },
})
const usage = data.nchat_attachments_aggregate.aggregate.sum.file_size
const count = data.nchat_attachments_aggregate.aggregate.count
console.log(`Using ${formatBytes(usage)} across ${count} files`)import { validateFile, SIZE_LIMITS } from '@/services/files/validation.service'
const validation = validateFile(file, {
userTier: 'member',
maxSize: SIZE_LIMITS.member, // 25MB
allowedTypes: ['image/*', 'video/*'],
})
if (!validation.valid) {
alert(validation.error)
return
}
if (validation.warnings) {
console.warn('Warnings:', validation.warnings)
}
// Proceed with upload# Storage Configuration
STORAGE_PROVIDER=minio # minio | s3 | gcs | r2 | b2 | azure
STORAGE_ENDPOINT=http://localhost:9000
STORAGE_BUCKET=nchat-files
STORAGE_ACCESS_KEY=minioadmin
STORAGE_SECRET_KEY=minioadmin
STORAGE_REGION=us-east-1
NEXT_PUBLIC_STORAGE_URL=http://storage.localhost
# File Type Restrictions
FILE_MAX_SIZE=104857600 # 100MB default
FILE_ALLOWED_TYPES= # Comma-separated, empty = all allowed
FILE_BLOCKED_TYPES= # Comma-separated
FILE_ALLOWED_EXTENSIONS= # Comma-separated
FILE_BLOCKED_EXTENSIONS=exe,bat,cmd,com,msi
# Processing Configuration
FILE_PROCESSING_URL=http://localhost:3104
FILE_PROCESSING_WEBHOOK_URL=http://localhost:3000/api/files/webhook
FILE_PROCESSING_WEBHOOK_SECRET=your-secret-key
FILE_ENABLE_VIRUS_SCAN=false
FILE_ENABLE_OPTIMIZATION=true
FILE_STRIP_EXIF=true
FILE_GENERATE_THUMBNAILS=true
FILE_THUMBNAIL_SIZES=100,400,1200
# Queue Configuration
FILE_QUEUE_CONCURRENCY=3
FILE_PROCESSING_TIMEOUT=30000 # millisecondsimport { getFileTypeConfig } from '@/services/files/config'
const config = getFileTypeConfig()
config.maxSize = 50 * 1024 * 1024 // 50MB
config.allowedMimeTypes = ['image/*', 'video/*']
config.enableVirusScan = true
config.stripExif = true
config.generateThumbnails = true
config.thumbnailSizes = [100, 400, 800]| Tier | Max File Size | Total Storage | Status |
|---|---|---|---|
| Guest | 5 MB | - | β Enforced |
| Member | 25 MB | 5 GB | |
| Premium | 100 MB | 50 GB | |
| Admin | 500 MB | 500 GB | |
| Owner | 500 MB | Unlimited | β Enforced |
const accessService = getFileAccessService()
const limits = await accessService.getFileSizeLimits(userId, userRole)
console.log('Max file size:', formatBytes(limits.maxFileSize))
console.log('Total storage:', formatBytes(limits.maxTotalStorage))To fully enforce storage quotas:
- Track Usage: Create background job to calculate per-user storage
- Check Before Upload: Verify user hasn't exceeded quota
- Cleanup Job: Remove files from deleted messages after 30 days
- Admin Dashboard: Show storage usage analytics
// Example implementation needed
async function checkStorageQuota(userId: string, fileSize: number): Promise<boolean> {
const usage = await getUserStorageUsage(userId)
const limits = await getFileSizeLimits(userId)
return usage + fileSize <= limits.maxTotalStorage
}ββββββββββββ
β Upload β
ββββββ¬ββββββ
β
βΌ
ββββββββββββββββββββ
β Validation β β Size, type, extension checks
β - Size check β
β - Type check β
β - Virus scan (*) β
ββββββ¬ββββββββββββββ
β
βΌ
ββββββββββββββββββββ
β Store in S3 β β Upload to storage
β - Multipart β
β - Content hash β
ββββββ¬ββββββββββββββ
β
βΌ
ββββββββββββββββββββ
β Create DB Record β β Insert into database
ββββββ¬ββββββββββββββ
β
βΌ
ββββββββββββββββββββ
β Processing Job β β Send to file-processing plugin
β - Thumbnails β
β - Metadata β
β - Optimize β
β - EXIF strip (*) β
ββββββ¬ββββββββββββββ
β
βΌ
ββββββββββββββββββββ
β Update Record β β Update with results
β - Status β
β - Thumbnails β
β - Metadata β
ββββββββββββββββββββ
(*) = Placeholder, needs implementation
| Operation | Description | Status |
|---|---|---|
| thumbnail | Generate thumbnails (100px, 400px, 1200px) | β Plugin ready |
| optimize | Image optimization (compression, WebP) | β Plugin ready |
| metadata | Extract EXIF, dimensions, duration | β Plugin ready |
| scan | Virus scanning with ClamAV |
The file-processing plugin sends webhooks on completion:
// POST /api/files/webhook
{
"event": "job.completed",
"jobId": "uuid",
"fileId": "uuid",
"status": "completed",
"thumbnails": [
{
"id": "uuid",
"path": "thumbnails/file-id/100.jpeg",
"width": 100,
"height": 100,
"size": 5000,
"format": "jpeg"
}
],
"metadata": {
"width": 1920,
"height": 1080,
"aspectRatio": 1.78,
"exifStripped": true
},
"scan": {
"isClean": true,
"threatsFound": 0
},
"durationMs": 1234
}# Run file service tests
pnpm test src/services/files/__tests__
# Run with coverage
pnpm test:coverage src/services/files-
src/services/files/__tests__/types.test.ts- Type utilities -
src/hooks/__tests__/use-file-upload.test.ts- Upload hook -
src/hooks/__tests__/use-attachments.test.ts- Attachments hook
-
Upload Tests:
# Small file (<5MB) curl -X POST http://localhost:3000/api/files/upload \ -F "[email protected]" \ -F "channelId=uuid" # Large file (>5MB, multipart) curl -X POST http://localhost:3000/api/files/upload \ -F "[email protected]" \ -F "channelId=uuid"
-
Download Tests:
curl http://localhost:3000/api/files/{id}/download -
Access Control Tests:
- Upload as guest (should fail)
- Upload to private channel (only members)
- Download file from public channel (should work)
- Download file from private channel (requires membership)
-
Virus Scanning: Placeholder implementation
- Needs ClamAV or cloud scanner integration
- Currently returns clean for all files
-
EXIF Stripping: Not implemented
- Needs image processing library (sharp, piexifjs)
- Privacy risk for user-uploaded photos
-
Storage Quotas: File size enforced, but not total storage
- Per-file limits work
- Total storage tracking not implemented
- No cleanup of deleted files
-
File Processing Plugin: Code ready, but plugin needs deployment
- Thumbnail generation
- Video transcoding
- Image optimization
- Metadata extraction
- Deduplication: Use content hash to avoid duplicate storage
- CDN Integration: CloudFront, Cloudflare for faster delivery
- Compression: Client-side compression before upload
- Resumable Uploads: For large files and unreliable connections
- WebTorrent: P2P file sharing for large files
- Media Transcoding: Server-side video transcoding
- Smart Thumbnails: AI-powered thumbnail selection
- Image Recognition: Auto-tagging, NSFW detection
- Deploy file-processing plugin (port 3104)
- Integrate virus scanning (ClamAV or cloud service)
- Implement EXIF stripping (sharp or piexifjs)
- Enable storage quota enforcement
- Set up CDN for file delivery
- Configure backup for storage bucket
- Set up monitoring for storage usage
- Test with production-size files
- Verify access control rules
- Document incident response procedures
// Metrics to track
- Upload success rate
- Upload latency (p50, p95, p99)
- Download latency
- Processing job success rate
- Storage usage by user/channel
- Virus scan detections
- File type distribution
- Average file size-
Implementation Plans:
/docs/*-IMPLEMENTATION-PLAN.md -
Media Guide:
/docs/MEDIA-QUICK-REFERENCE.md -
API Documentation:
/src/app/api/files/*/route.ts -
GraphQL Schema:
/src/graphql/files.ts
-
Services:
/src/services/files/ -
API Routes:
/src/app/api/files/ -
Components:
/src/components/files/ -
Hooks:
/src/hooks/use-file*.ts -
Types:
/src/services/files/types.ts
For issues or questions:
- Check this documentation
- Review implementation plans in
/docs - Check
.claude/PROGRESS.mdfor recent changes - File an issue with reproduction steps
Last Updated: February 3, 2026 Maintained By: Ι³Chat Development Team License: See project root