file upload security - nself-org/cli GitHub Wiki
Comprehensive security guide for implementing file uploads in nself applications.
- Security Overview
- Input Validation
- File Type Restrictions
- Virus Scanning
- Storage Security
- Access Control
- Rate Limiting
- Content Security
- Production Checklist
File uploads are a common attack vector. Follow these practices to secure your application:
- Malicious File Upload - Uploading executable files or malware
-
Path Traversal - Using
../to escape storage directory - File Type Spoofing - Fake MIME types or file extensions
- Storage Exhaustion - Uploading massive files to fill disk
- XML Bomb - Specially crafted files that expand when processed
- SSRF (Server-Side Request Forgery) - Uploading files that trigger internal requests
nself implements multiple security layers:
Layer 1: Client-side validation (UX, not security)
โ
Layer 2: File type & size validation (frontend + backend)
โ
Layer 3: Virus scanning (ClamAV)
โ
Layer 4: Storage isolation (MinIO)
โ
Layer 5: Access control (Hasura RLS)
โ
Layer 6: Content Security Policy (Nginx headers)
Always enforce on both client and server:
// Frontend (Next.js)
const MAX_FILE_SIZE = 10 * 1024 * 1024; // 10MB
function validateFileSize(file: File): boolean {
if (file.size > MAX_FILE_SIZE) {
throw new Error(`File too large. Max size: ${MAX_FILE_SIZE / 1024 / 1024}MB`);
}
return true;
}# Backend (.env.prod)
UPLOAD_MAX_FILE_SIZE=10485760 # 10MB-- Database constraint
ALTER TABLE public.files
ADD CONSTRAINT files_size_check
CHECK (size <= 10485760);Strip dangerous characters:
function sanitizeFileName(fileName: string): string {
return fileName
.replace(/[^a-zA-Z0-9._-]/g, '_') // Remove special chars
.replace(/\.{2,}/g, '.') // No multiple dots
.replace(/^\./, '') // No leading dot
.substring(0, 255); // Max 255 chars
}Never trust user-provided file names:
// Generate secure file names
const secureFileName = `${uuid()}_${sanitizeFileName(file.name)}`;Never use user input in file paths:
// โ DANGEROUS
const path = `uploads/${userInput}`;
// โ
SAFE
const path = `uploads/${userId}/${uuid()}_${sanitizedName}`;Server-side validation:
# upload-pipeline.sh validates paths
if [[ "${dest_path}" =~ \.\. ]]; then
output_error "Invalid path: contains .."
return 1
fiOnly allow specific file types:
const ALLOWED_TYPES = {
images: ['image/jpeg', 'image/png', 'image/gif', 'image/webp'],
documents: ['application/pdf', 'application/msword'],
videos: ['video/mp4', 'video/webm'],
};
function validateFileType(file: File, category: keyof typeof ALLOWED_TYPES): boolean {
const allowed = ALLOWED_TYPES[category];
if (!allowed.includes(file.type)) {
throw new Error(`Invalid file type. Allowed: ${allowed.join(', ')}`);
}
return true;
}Never trust client-provided MIME type:
# Server validates actual file content
mime_type="$(file --mime-type -b "${file_path}")"
# Check against allowlist
case "${mime_type}" in
image/jpeg|image/png|image/gif)
# Valid image
;;
*)
output_error "Invalid file type: ${mime_type}"
return 1
;;
esacVerify file signature (first bytes):
async function validateImageSignature(file: File): Promise<boolean> {
const buffer = await file.slice(0, 4).arrayBuffer();
const bytes = new Uint8Array(buffer);
// JPEG: FF D8 FF
if (bytes[0] === 0xFF && bytes[1] === 0xD8 && bytes[2] === 0xFF) {
return true;
}
// PNG: 89 50 4E 47
if (bytes[0] === 0x89 && bytes[1] === 0x50 && bytes[2] === 0x4E && bytes[3] === 0x47) {
return true;
}
throw new Error('Invalid image signature');
}Never allow these extensions:
const DANGEROUS_EXTENSIONS = [
'.exe', '.dll', '.bat', '.cmd', '.sh',
'.php', '.asp', '.jsp', '.js', '.html',
'.svg', // Can contain scripts
'.jar', '.app', '.dmg',
];
function checkDangerousExtension(fileName: string): boolean {
const ext = fileName.toLowerCase().substring(fileName.lastIndexOf('.'));
if (DANGEROUS_EXTENSIONS.includes(ext)) {
throw new Error(`Dangerous file type: ${ext}`);
}
return true;
}# .env.prod
UPLOAD_ENABLE_VIRUS_SCAN=true
# Install ClamAV
sudo apt-get install clamav clamav-daemon
# Update virus definitions daily
sudo freshclam# upload-pipeline.sh automatically scans when enabled
if [[ "${ENABLE_VIRUS_SCAN}" == "true" ]]; then
if ! scan_file_for_viruses "${file_path}"; then
output_error "Virus detected! Upload aborted."
return 1
fi
fi# Move infected files to quarantine
QUARANTINE_DIR="/var/quarantine"
if clamscan "${file_path}" | grep -q "FOUND"; then
mkdir -p "${QUARANTINE_DIR}"
mv "${file_path}" "${QUARANTINE_DIR}/$(date +%s)_$(basename "${file_path}")"
log_security_event "virus_detected" "${file_path}"
fi# cron job: /etc/cron.daily/freshclam
#!/bin/bash
freshclam --quiet
systemctl reload clamav-daemonNever serve uploads from main domain:
# โ DANGEROUS
https://yourdomain.com/uploads/user-file.jpg
# โ
SAFE
https://cdn.yourdomain.com/uploads/user-file.jpgPrevents:
- Cookie theft via XSS in uploaded files
- Same-origin policy bypass
- CSRF attacks
Force download for dangerous types:
# nginx config
location /uploads/ {
# Force download for non-images
if ($request_uri ~ \.(pdf|doc|zip)$) {
add_header Content-Disposition "attachment; filename=$1";
}
# Images can be inline
if ($request_uri ~ \.(jpg|jpeg|png|gif|webp)$) {
add_header Content-Disposition "inline";
}
}# nginx config for storage domain
location /uploads/ {
# Disable PHP, Python, etc.
location ~ \.(php|py|rb|pl|sh)$ {
deny all;
}
# No code execution
add_header X-Content-Type-Options "nosniff" always;
add_header Content-Security-Policy "default-src 'none'; img-src 'self'; style-src 'none'; script-src 'none';" always;
}# MinIO configuration
# Bucket should NOT be public by default
# Use signed URLs for temporary access
mc policy set download nself/uploads-private # No public access
# Generate time-limited URLs
mc share download --expire 1h nself/uploads-private/file.pdfEnforce ownership at database level:
-- Users can only see their own files
CREATE POLICY files_select_own
ON public.files
FOR SELECT
USING (
auth.uid() = user_id
OR is_public = true
);
-- Users can only upload to their own account
CREATE POLICY files_insert_own
ON public.files
FOR INSERT
WITH CHECK (auth.uid() = user_id);Double-layer protection:
# metadata/tables/public_files.yaml
select_permissions:
- role: user
permission:
filter:
_or:
- user_id: { _eq: X-Hasura-User-Id }
- is_public: { _eq: true }
columns:
- id
- name
- url
# Don't expose sensitive columns to othersimport { signUrl } from '@/lib/storage';
// Generate time-limited URL
const signedUrl = await signUrl(file.path, {
expiresIn: 3600, // 1 hour
userId: currentUser.id,
});
// URL expires after 1 hour
return { url: signedUrl };# nginx config - restrict upload endpoint
location /api/upload {
# Only allow from trusted IPs
allow 10.0.0.0/8; # Internal network
allow 192.168.1.0/24; # Office network
deny all;
proxy_pass http://api:3000;
}Prevent abuse:
// Use Redis for rate limiting
import { rateLimit } from '@/lib/redis';
async function uploadHandler(req, res) {
const userId = req.user.id;
// Limit: 10 uploads per hour per user
const allowed = await rateLimit(`upload:${userId}`, {
max: 10,
window: 3600,
});
if (!allowed) {
return res.status(429).json({
error: 'Too many uploads. Try again later.',
});
}
// Process upload...
}# nginx config
limit_req_zone $binary_remote_addr zone=upload:10m rate=1r/s;
location /api/upload {
limit_req zone=upload burst=5 nodelay;
client_max_body_size 10M;
proxy_pass http://api:3000;
}Per-user storage limits:
-- Add quota column
ALTER TABLE auth.users
ADD COLUMN storage_quota_bytes bigint DEFAULT 1073741824; -- 1GB
-- Trigger to check quota before upload
CREATE OR REPLACE FUNCTION check_storage_quota()
RETURNS TRIGGER AS $$
DECLARE
current_usage bigint;
user_quota bigint;
BEGIN
-- Calculate current usage
SELECT COALESCE(SUM(size), 0)
INTO current_usage
FROM public.files
WHERE user_id = NEW.user_id;
-- Get user quota
SELECT storage_quota_bytes
INTO user_quota
FROM auth.users
WHERE id = NEW.user_id;
-- Check if over quota
IF (current_usage + NEW.size) > user_quota THEN
RAISE EXCEPTION 'Storage quota exceeded. Current: %, Quota: %',
current_usage, user_quota;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER enforce_storage_quota
BEFORE INSERT ON public.files
FOR EACH ROW
EXECUTE FUNCTION check_storage_quota();# nginx config for storage domain
add_header Content-Security-Policy "
default-src 'none';
img-src 'self';
media-src 'self';
style-src 'none';
script-src 'none';
object-src 'none';
frame-ancestors 'none';
" always;Strip EXIF data (may contain GPS, camera info):
# Use ImageMagick to strip metadata
convert uploaded.jpg -strip sanitized.jpg// Frontend: Remove EXIF before upload
import piexif from 'piexifjs';
function stripExif(file: File): Promise<File> {
return new Promise((resolve) => {
const reader = new FileReader();
reader.onload = (e) => {
const data = e.target?.result as string;
const stripped = piexif.remove(data);
const blob = dataURLtoBlob(stripped);
resolve(new File([blob], file.name, { type: file.type }));
};
reader.readAsDataURL(file);
});
}Disable JavaScript in PDFs:
# Use pdftk to sanitize PDFs
pdftk uploaded.pdf output sanitized.pdf flatten
# Or use Ghostscript
gs -dSAFER -dNOPAUSE -dBATCH \
-sDEVICE=pdfwrite \
-sOutputFile=sanitized.pdf \
uploaded.pdfNever allow SVG uploads (contains scripts):
// If you MUST allow SVGs, sanitize them
import { sanitize } from 'dompurify';
function sanitizeSVG(svgContent: string): string {
return sanitize(svgContent, {
USE_PROFILES: { svg: true },
ALLOWED_TAGS: ['svg', 'path', 'circle', 'rect', 'line', 'polyline', 'polygon'],
ALLOWED_ATTR: ['width', 'height', 'viewBox', 'd', 'fill', 'stroke'],
});
}Before deploying to production:
- File type allowlist implemented (frontend + backend)
- File size limits enforced (client + server + database)
- Virus scanning enabled (ClamAV with daily updates)
- MIME type validation on server (not client value)
- File name sanitization (remove special characters)
- Path traversal prevention (no
../in paths) - Row Level Security enabled on files table
- Hasura permissions configured correctly
- Separate storage domain (not main app domain)
- Content-Disposition headers set
- Content Security Policy headers set
- Script execution disabled in storage directory
- Rate limiting on upload endpoints
- Storage quotas per user
- Signed URLs for private files
- EXIF data stripping for images
- PDF sanitization if allowing PDFs
- IP restrictions on admin uploads
- CDN with WAF (CloudFlare, AWS WAF)
- Audit logging for all uploads
- Backup strategy for uploaded files
- Alert on large uploads (> 100MB)
- Alert on virus detection
- Alert on quota exceeded
- Alert on unusual upload patterns
- Dashboard for storage usage
- Daily reports of file types uploaded
- Upload malicious file (should be blocked)
- Upload with
../in path (should be sanitized) - Upload fake MIME type (should detect real type)
- Upload > size limit (should be rejected)
- Upload to other user's folder (should fail)
- Access other user's file (should fail)
- Upload with XSS in filename (should be sanitized)
// Complete secure upload implementation
async function secureFileUpload(file: File, userId: string) {
// 1. Client-side validation
validateFileSize(file);
validateFileType(file, 'images');
await validateImageSignature(file);
checkDangerousExtension(file.name);
// 2. Sanitize file name
const safeName = sanitizeFileName(file.name);
// 3. Strip EXIF data
const strippedFile = await stripExif(file);
// 4. Check rate limit
const allowed = await rateLimit(`upload:${userId}`, {
max: 10,
window: 3600,
});
if (!allowed) {
throw new Error('Rate limit exceeded');
}
// 5. Check storage quota
const quota = await checkStorageQuota(userId, file.size);
if (!quota.allowed) {
throw new Error('Storage quota exceeded');
}
// 6. Upload with virus scan
const result = await uploadFile(strippedFile, {
path: `users/${userId}/${uuid()}_${safeName}`,
virusScan: true,
compression: true,
});
// 7. Save metadata with RLS
const fileRecord = await db.files.insert({
user_id: userId,
name: safeName,
size: file.size,
mime_type: result.mimeType,
path: result.path,
url: result.url,
is_public: false,
});
// 8. Audit log
await logAuditEvent({
action: 'file_uploaded',
user_id: userId,
file_id: fileRecord.id,
file_size: file.size,
ip: request.ip,
});
return fileRecord;
}- OWASP File Upload Cheat Sheet
- CWE-434: Unrestricted Upload of File
- Content Security Policy Reference
- ClamAV Documentation
For security concerns:
- Security Policy
- Email: [email protected]