COMPLETE ARCHITECTURE - nself-org/cli GitHub Wiki
Version 0.9.9 | System Design, Patterns, and Technical Decisions
- Architecture Overview
- C4 Model Diagrams
- Component Architecture
- Data Flow
- Security Architecture
- Scalability Architecture
- Design Decisions
- Technology Stack
- Integration Patterns
nself is a full-stack backend platform built on modern cloud-native principles with a focus on:
- Self-hosting - Complete infrastructure control
- Multi-tenancy - Isolated data per tenant with shared infrastructure
- Scalability - Horizontal and vertical scaling capabilities
- Security - Defense-in-depth with RLS, JWT, SSL, and audit logging
- Developer Experience - GraphQL-first API with CLI automation
- Separation of Concerns - Each service has a single responsibility
- API-First - GraphQL as primary interface, REST for legacy compatibility
- Database-Centric - PostgreSQL as source of truth, RLS for isolation
- Stateless Services - Services can be restarted/scaled without data loss
-
Configuration as Code - Everything defined in
.envand generated files - Zero-Trust Security - Every request authenticated and authorized
- Observability Built-In - Metrics, logs, and traces from day one
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β External Actors β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
ββββββββββββββββ ββββββββββββββββ ββββββββββββββββ
β End Users β β Developers β β Admins β
β β β β β β
β Web/Mobile β β CLI/API β β Dashboard β
ββββββββ¬ββββββββ ββββββββ¬ββββββββ ββββββββ¬ββββββββ
β β β
β βΌ β
β βββββββββββββββββββ β
β β GraphQL API β β
βββββββββββββΊβ (Hasura) ββββββββββββββ
ββββββββββ¬βββββββββ
β
ββββββββββββββββββββββββββΌβββββββββββββββββββββββββ
β β β
βΌ βΌ βΌ
βββββββββββββββββ βββββββββββββββββββ ββββββββββββββββββ
β nself β β PostgreSQL β β Object Storageβ
β Platform βββββββΊβ Database β β (MinIO) β
β β β β β β
β β’ Auth β β β’ Data β β β’ Files β
β β’ Functions β β β’ RLS β β β’ Media β
β β’ Monitoring β β β’ Multi-tenant β β β’ Backups β
βββββββββββββββββ βββββββββββββββββββ ββββββββββββββββββ
β
β
βΌ
βββββββββββββββββ
β External β
β Services β
β β
β β’ Email β
β β’ Payment β
β β’ Analytics β
βββββββββββββββββ
External Systems:
- End Users - Access via web/mobile applications
- Developers - Interact via CLI, API, and Hasura Console
- Administrators - Manage via Admin Dashboard and monitoring tools
- External Services - Email providers, payment gateways, analytics
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β nself Platform β
β Docker Compose Deployment β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Internet
β
βΌ
βββββββββββββββββββ
β Nginx β Port 80/443
β Reverse Proxy β β’ SSL Termination
β β β’ Load Balancing
β β β’ Rate Limiting
ββββββββββ¬βββββββββ
β
β Routes to Services
β
ββββββΌβββββββββββββββββ¬ββββββββββββββββ¬βββββββββββββββ
β β β β β
βΌ βΌ βΌ βΌ βΌ
βββββββββββ ββββββββββββββ ββββββββββββ βββββββββββ ββββββββββββ
β Hasura β β Auth β β Functionsβ β Admin β β Custom β
β GraphQL β β Service β β Runtime β β UI β β Services β
β β β β β β β β β (CS_1-10)β
β :8080 β β :4000 β β :3001 β β :3000 β β :800x β
ββββββ¬βββββ βββββββ¬βββββββ ββββββ¬ββββββ ββββββ¬βββββ ββββββ¬ββββββ
β β β β β
βββββββββββββββΌβββββββββββββββΌββββββββββββββ β
β β β
βΌ βΌ βΌ
ββββββββββββββββββββββββββββ ββββββββββββββ
β PostgreSQL β β Redis β
β Port 5432 β β :6379 β
β β β β
β β’ Application Data β β β’ Sessions β
β β’ Auth Tables β β β’ Cache β
β β’ Multi-tenant Schemas β β β’ Queues β
β β’ Row Level Security β ββββββββββββββ
ββββββββββββββββββββββββββββ
ββββββββββββββββ ββββββββββββββββββββββββ
β MinIO β β Monitoring Stack β
β S3 Storage β β β
β :9000 β β β’ Prometheus :9090 β
β β β β’ Grafana :3000 β
β β’ User Files β β β’ Loki :3100 β
β β’ Uploads β β β’ Tempo :3200 β
β β’ Backups β β β’ Alertmanager :9093 β
ββββββββββββββββ ββββββββββββββββββββββββ
Container Responsibilities:
-
Nginx - Entry point for all traffic
- SSL/TLS termination
- Routing to backend services
- Static file serving
- Rate limiting and DDoS protection
-
Hasura GraphQL Engine - API layer
- GraphQL API generation from database schema
- Real-time subscriptions
- Remote schemas and actions
- Authorization via permissions and RLS
-
Auth Service (nHost Auth) - Authentication
- User registration and login
- JWT token generation
- OAuth provider integration
- MFA support
- Session management
-
Functions Runtime - Serverless functions
- Node.js/Deno runtime
- Event-driven execution
- Database triggers
- Scheduled jobs
-
Admin UI - Management interface
- Visual service management
- Database browser
- User management
- Monitoring dashboards
-
Custom Services (CS_1 - CS_10) - User-defined
- Generated from templates
- Any language/framework
- Custom business logic
- Microservices architecture
-
PostgreSQL - Primary database
- Application data storage
- Auth system tables
- Multi-tenant data isolation (RLS)
- Full-text search
- JSON/JSONB support
-
Redis - In-memory data store
- Session storage
- Caching layer
- Rate limiting counters
- Job queues (with BullMQ)
-
MinIO - S3-compatible object storage
- File uploads
- Media storage
- Backup storage
- CDN source
-
Monitoring Stack - Observability
- Prometheus (metrics)
- Grafana (visualization)
- Loki (logs)
- Tempo (traces)
- Alertmanager (alerts)
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β Hasura GraphQL Engine Container β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
ββββββββββββββββββ
β GraphQL β Port 8080
β HTTP Server β /v1/graphql
ββββββββββ¬ββββββββ /v1/metadata
β /healthz
β
ββββββββββΌβββββββββββββββββ¬ββββββββββββββββ
β β β β
βΌ βΌ βΌ βΌ
βββββββββββββββ ββββββββββββββββ βββββββββββββ ββββββββββββββ
β Query β β Subscription β β Metadata β β Actions β
β Engine β β Engine β β Manager β β Handler β
β β β β β β β β
β β’ Parse β β β’ WebSocket β β β’ Schema β β β’ REST β
β β’ Validate β β β’ Live β β β’ Perms β β β’ Custom β
β β’ Execute β β β’ Streaming β β β’ Remote β β β’ Webhook β
ββββββββ¬βββββββ ββββββββ¬ββββββββ βββββββ¬ββββββ βββββββ¬βββββββ
β β β β
β β β β
ββββββββββββββββββΌβββββββββββββββββ β
β β
βΌ βΌ
βββββββββββββββββββββ βββββββββββββββββββ
β Authorization β β Remote Schema β
β Layer β β Proxy β
β β β β
β β’ Session Vars β β β’ Stitching β
β β’ RLS Context β β β’ Federation β
β β’ Permissions β βββββββββββββββββββ
ββββββββββββ¬βββββββββ
β
βΌ
βββββββββββββββββββββ
β SQL Compiler β
β β
β β’ Query Builder β
β β’ Join Optimizer β
β β’ RLS Injection β
ββββββββββββ¬βββββββββ
β
βΌ
βββββββββββββββββββββ
β PostgreSQL β
β Connection Pool β
β β
β β’ Pool Manager β
β β’ Health Check β
β β’ Reconnect β
βββββββββββββββββββββ
β
βΌ
PostgreSQL
Database
Component Interactions:
-
Query Engine
- Receives GraphQL queries
- Parses and validates against schema
- Compiles to SQL
- Returns JSON response
-
Subscription Engine
- Maintains WebSocket connections
- Polls database for changes (live queries)
- Pushes updates to clients
- Multiplexing for efficiency
-
Authorization Layer
- Extracts session variables from JWT
- Sets PostgreSQL session context
- Applies role-based permissions
- Injects RLS policies
-
Metadata Manager
- Stores schema configuration
- Manages permissions
- Handles remote schemas
- Triggers and event handlers
-
Actions Handler
- Proxies to custom REST endpoints
- Transforms requests/responses
- Error handling and retries
-- Multi-Tenant Row Level Security Implementation
-- 1. Enable RLS on table
CREATE TABLE posts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL,
user_id UUID NOT NULL,
title TEXT NOT NULL,
content TEXT,
created_at TIMESTAMPTZ DEFAULT NOW()
);
ALTER TABLE posts ENABLE ROW LEVEL SECURITY;
-- 2. Create policies
-- Policy: Users can only see posts in their tenant
CREATE POLICY tenant_isolation_select ON posts
FOR SELECT
USING (tenant_id = current_setting('app.tenant_id', true)::uuid);
-- Policy: Users can only insert posts in their tenant
CREATE POLICY tenant_isolation_insert ON posts
FOR INSERT
WITH CHECK (
tenant_id = current_setting('app.tenant_id', true)::uuid
AND user_id = current_setting('hasura.user.id', true)::uuid
);
-- Policy: Users can only update their own posts
CREATE POLICY user_update_own ON posts
FOR UPDATE
USING (
tenant_id = current_setting('app.tenant_id', true)::uuid
AND user_id = current_setting('hasura.user.id', true)::uuid
);
-- Policy: Admin can see all posts in tenant
CREATE POLICY admin_select_all ON posts
FOR SELECT
USING (
tenant_id = current_setting('app.tenant_id', true)::uuid
AND current_setting('hasura.user.role', true) = 'admin'
);
-- 3. Session variables set by Hasura from JWT
-- JWT payload:
{
"sub": "user-uuid",
"https://hasura.io/jwt/claims": {
"x-hasura-user-id": "user-uuid",
"x-hasura-allowed-roles": ["user", "admin"],
"x-hasura-default-role": "user",
"x-hasura-tenant-id": "tenant-uuid"
}
}
-- Hasura sets PostgreSQL session:
SET LOCAL app.tenant_id = 'tenant-uuid';
SET LOCAL hasura.user.id = 'user-uuid';
SET LOCAL hasura.user.role = 'user';
-- 4. Query execution with RLS
-- User queries:
SELECT * FROM posts;
-- PostgreSQL rewrites to:
SELECT * FROM posts
WHERE tenant_id = 'tenant-uuid' -- RLS policy applied
AND user_id = 'user-uuid'; -- If user, not admin1. Client Request
β
2. Nginx (SSL termination, routing)
β
3. Hasura GraphQL Engine
β’ Parse GraphQL query
β’ Validate against schema
β’ Extract JWT from Authorization header
β
4. JWT Validation
β’ Verify signature (HMAC-SHA256)
β’ Check expiration
β’ Extract claims (user ID, role, tenant ID)
β
5. Set Session Variables
β’ app.tenant_id
β’ hasura.user.id
β’ hasura.user.role
β
6. Authorization Check
β’ Check table permissions for role
β’ Apply column-level permissions
β
7. SQL Compilation
β’ GraphQL β SQL
β’ Inject RLS policy filters
β’ Optimize joins
β
8. Database Query
β’ PostgreSQL executes query
β’ RLS policies filter rows
β’ Return result set
β
9. Response Transformation
β’ SQL result β JSON
β’ Apply field transformations
β’ Nested object resolution
β
10. Client Response
β’ JSON over HTTPS
Performance Optimizations:
- Query Caching - Hasura caches compiled SQL
- Connection Pooling - Reuse database connections
- Prepared Statements - Faster query execution
- Multiplexing - Batch subscriptions
- Compression - gzip response payloads
1. User Login Request
POST /v1/auth/login
{ "email": "[email protected]", "password": "..." }
β
2. Auth Service
β’ Hash password (bcrypt)
β’ Query: SELECT * FROM auth.users WHERE email = ?
β’ Compare password hash
β
3. Password Match
β
4. Generate JWT
β’ Header: { "alg": "HS256", "typ": "JWT" }
β’ Payload: {
"sub": "user-uuid",
"iat": 1706745600,
"exp": 1706749200,
"https://hasura.io/jwt/claims": {
"x-hasura-user-id": "user-uuid",
"x-hasura-allowed-roles": ["user"],
"x-hasura-default-role": "user",
"x-hasura-tenant-id": "tenant-uuid"
}
}
β’ Sign with secret key
β
5. Generate Refresh Token
β’ Random 64-byte token
β’ Store in database with expiry (30 days)
β
6. Response
{
"accessToken": "eyJhbGc...", // 15 min expiry
"refreshToken": "abc123...", // 30 day expiry
"user": { "id": "...", "email": "..." }
}
β
7. Client Stores Tokens
β’ Access token in memory
β’ Refresh token in httpOnly cookie
β
8. Subsequent Requests
Authorization: Bearer eyJhbGc...
β
9. Token Refresh (when expired)
POST /v1/auth/refresh
{ "refreshToken": "abc123..." }
β
10. New Access Token Issued
Security Features:
- Short-lived access tokens (15 minutes)
- Long-lived refresh tokens (30 days)
- Token rotation on refresh
- Secure storage (httpOnly cookies for refresh token)
- Token revocation support
- Rate limiting on auth endpoints
1. Client Request
POST https://api.yourdomain.com/v1/storage/upload
Content-Type: multipart/form-data
Authorization: Bearer <jwt>
β
2. Nginx Route
β Forward to Hasura Actions endpoint
β
3. Hasura Actions Handler
β’ Validate JWT
β’ Extract user/tenant from claims
β’ Proxy to custom upload function
β
4. Upload Function (Node.js)
β’ Validate file type and size
β’ Generate unique filename
β’ Extract tenant_id from session
β
5. MinIO Upload
const s3 = new AWS.S3({
endpoint: 'http://minio:9000',
accessKeyId: process.env.MINIO_ACCESS_KEY,
secretAccessKey: process.env.MINIO_SECRET_KEY,
});
await s3.putObject({
Bucket: `tenant-${tenant_id}`,
Key: filename,
Body: fileBuffer,
ACL: 'private',
});
β
6. Database Record
INSERT INTO files (id, tenant_id, user_id, filename, url, size)
VALUES (uuid, tenant_id, user_id, filename, url, file_size);
β
7. Response
{
"fileId": "file-uuid",
"url": "https://cdn.yourdomain.com/tenant-uuid/filename.jpg",
"size": 1024000
}
Security:
- Tenant isolation - Separate S3 buckets per tenant
- Access control - Pre-signed URLs for private files
- Virus scanning - ClamAV integration (optional)
- File type validation - Whitelist allowed MIME types
- Size limits - Per-user and per-tenant quotas
1. Client Subscription
subscription {
posts(where: { user_id: { _eq: $userId } }) {
id
title
content
}
}
β
2. WebSocket Handshake
ws://api.yourdomain.com/v1/graphql
Connection: Upgrade
β
3. Hasura Subscription Manager
β’ Parse subscription
β’ Validate permissions
β’ Set session variables
β
4. Initial Data Fetch
β’ Execute query once
β’ Return current data to client
β
5. Polling Setup (Live Query)
β’ Hasura polls database every 1 second (configurable)
β’ Compares result hash with previous
β
6. Data Change Detected
β’ New post inserted
β’ Hash changed
β
7. Push Update to Client
{
"type": "data",
"id": "subscription-id",
"payload": {
"data": {
"posts": [/* updated data */]
}
}
}
β
8. Client Updates UI
β’ React/Vue/Angular component re-renders
β’ New post appears instantly
Optimization:
- Multiplexing - Batch identical subscriptions
- Refetch interval - Configurable (default: 1s)
- Cursor-based - Only fetch changes since last poll
- Connection management - Automatic reconnection
Layer 1: Network Security
βββ Firewall (ufw/iptables)
βββ DDoS Protection (Cloudflare/AWS Shield)
βββ Rate Limiting (Nginx)
Layer 2: Transport Security
βββ TLS 1.3 (SSL certificates)
βββ HSTS (Strict-Transport-Security)
βββ Certificate Pinning (mobile apps)
Layer 3: Application Security
βββ JWT Authentication
βββ CORS Configuration
βββ Security Headers (CSP, X-Frame-Options, etc.)
βββ Input Validation
Layer 4: API Security
βββ GraphQL Query Depth Limiting
βββ Query Cost Analysis
βββ Rate Limiting per User
βββ API Key Management
Layer 5: Database Security
βββ Row Level Security (RLS)
βββ Role-Based Access Control
βββ SQL Injection Prevention (Parameterized Queries)
βββ Encrypted Connections (SSL)
Layer 6: Data Security
βββ Encryption at Rest
βββ Encryption in Transit
βββ PII Anonymization
βββ Secure Backups
Layer 7: Audit & Monitoring
βββ Audit Logging
βββ Anomaly Detection
βββ Intrusion Detection
βββ Security Alerts
Multi-Tenant Isolation:
-- Every query is rewritten by PostgreSQL
-- User query:
SELECT * FROM posts WHERE title LIKE '%search%';
-- PostgreSQL rewrites to:
SELECT * FROM posts
WHERE title LIKE '%search%'
AND tenant_id = current_setting('app.tenant_id')::uuid -- RLS policy
AND (
user_id = current_setting('hasura.user.id')::uuid -- User posts
OR current_setting('hasura.user.role') = 'admin' -- Or admin
);Benefits:
- Zero-trust - Database enforces isolation, not application
- SQL injection proof - Policies can't be bypassed
- Centralized - Security rules in one place
- Performance - Indexes work with RLS
0-10K Users: Single Server
βββββββββββββββββββββββββββββββ
β Single Server (16GB RAM) β
β β
β β’ All services in Docker β
β β’ PostgreSQL β
β β’ Redis β
β β’ Hasura β
β β’ Auth β
βββββββββββββββββββββββββββββββ
10K-100K Users: Separated Database
βββββββββββββββββββ ββββββββββββββββββββ
β App Server β β Database Server β
β β β β
β β’ Hasura βββββββββΊβ β’ PostgreSQL β
β β’ Auth β β β’ Redis β
β β’ Functions β β β
β β’ Nginx β ββββββββββββββββββββ
βββββββββββββββββββ
100K+ Users: Horizontal Scaling
ββββββββββββββββ
β Load Balancerβ
ββββββββ¬ββββββββ
β
βββββββββββΌββββββββββ
βΌ βΌ βΌ
ββββββββββ ββββββββββ ββββββββββ
β App 1 β β App 2 β β App 3 β
ββββββ¬ββββ ββββββ¬ββββ ββββββ¬ββββ
β β β
ββββββββββββΌβββββββββββ
βΌ
ββββββββββββββββββ
β PostgreSQL β
β Primary β
ββββββ¬ββββββββββββ
β
ββββββ΄βββββ
βΌ βΌ
βββββββββββ βββββββββββ
βReplica 1β βReplica 2β
βββββββββββ βββββββββββ
Decision: Use Hasura GraphQL as primary API layer
Rationale:
- Automatic API generation - No manual endpoint coding
- Real-time built-in - WebSocket subscriptions
- Type safety - Schema-driven development
- Performance - Fetch exactly what you need
- RLS integration - Direct PostgreSQL security
Trade-offs:
- β Faster development
- β Better developer experience
- β Built-in subscriptions
- β Learning curve for GraphQL
- β Caching more complex than REST
Decision: PostgreSQL as primary database
Rationale:
- ACID compliance - Strong consistency guarantees
- Row Level Security - Built-in multi-tenancy
- Rich data types - JSON, arrays, full-text search
- Mature ecosystem - 30+ years of development
- Excellent performance - Scales to millions of rows
Trade-offs:
- β Data integrity
- β Complex queries
- β ACID guarantees
- β Harder horizontal scaling than NoSQL
- β Schema migrations required
Decision: Docker Compose for <100K users, Kubernetes for larger
Rationale:
- Simplicity - Single YAML file vs many manifests
- Local development - Same as production
- Resource efficiency - No k8s overhead
- Easier debugging - Logs and exec simpler
- Cost - No k8s control plane costs
Trade-offs:
- β Simpler operations
- β Lower resource usage
- β Faster iteration
- β Less auto-scaling
- β Manual failover
When to switch to Kubernetes:
-
100K concurrent users
- Multi-region deployment
- Advanced auto-scaling needed
- Service mesh requirements
- Container Orchestration: Docker Compose (development/small), Kubernetes (large scale)
- Reverse Proxy: Nginx
- SSL: Let's Encrypt / Commercial certs
- Load Balancer: HAProxy / Cloud LB
- GraphQL API: Hasura GraphQL Engine v2.35+
- Database: PostgreSQL 15+
- Auth: nHost Auth (fork of Hasura Auth)
- Cache/Queue: Redis 7+
- Object Storage: MinIO (S3-compatible)
- Functions: Node.js 20 / Deno 1.40
- Metrics: Prometheus
- Visualization: Grafana
- Logs: Loki + Promtail
- Traces: Tempo
- Alerts: Alertmanager
- Authentication: JWT (HS256/RS256)
- Authorization: RLS + Hasura permissions
- Encryption: TLS 1.3, AES-256
- Secrets: Encrypted environment variables
- CLI: Bash 3.2+ (POSIX-compliant)
- CI/CD: GitHub Actions
- Testing: Bats (Bash), Jest (JS), pytest (Python)
- Documentation: Markdown, Mermaid diagrams
Pattern 1: Database-Mediated
Service A β PostgreSQL β Service B
(via triggers, LISTEN/NOTIFY)
Pattern 2: Event-Driven
Service A β Redis Pub/Sub β Service B
Pattern 3: Direct HTTP
Service A β HTTP β Service B
(via Hasura Actions or custom endpoints)
Pattern 4: Message Queue
Service A β BullMQ (Redis) β Service B
(for async jobs)
- Multi-Tenancy Architecture
- Billing Architecture
- Build Architecture
- Command Reorganization
- API Documentation
Maintainers:
- Architecture Review: Monthly
- Diagram Updates: On major changes
- Performance Benchmarks: Quarterly