PROJECT_STRUCTURE - nself-org/cli GitHub Wiki

nself Project Directory Structure

This guide explains the complete directory structure of an nself project, using the demo setup as an example. Understanding this structure will help you navigate and customize your Backend-as-a-Service deployment.

Complete Project Directory Tree

project-root/                          # Your project directory (e.g., demo-app)
โ”‚
โ”œโ”€โ”€ _backup/                           # Auto-backups from nself operations
โ”‚   โ””โ”€โ”€ [timestamps]/                  # Timestamped backup folders with docker-compose versions
โ”‚
โ”œโ”€โ”€ auth/                              # nHost Auth service configuration
โ”‚   โ””โ”€โ”€ config/                        # Auth service config files (JWT, providers, etc.)
โ”‚
โ”œโ”€โ”€ functions/                         # Serverless functions directory
โ”‚   โ”œโ”€โ”€ dist/                          # Compiled functions output
โ”‚   โ””โ”€โ”€ src/                           # Function source code
โ”‚
โ”œโ”€โ”€ hasura/                           # Hasura GraphQL engine configuration
โ”‚   โ”œโ”€โ”€ metadata/                      # Hasura metadata (tables, relationships, permissions)
โ”‚   โ””โ”€โ”€ migrations/                    # Database migrations managed by Hasura
โ”‚
โ”œโ”€โ”€ logs/                              # Application and service logs
โ”‚
โ”œโ”€โ”€ monitoring/                        # Observability stack configuration
โ”‚   โ”œโ”€โ”€ grafana/                       # Grafana dashboards and provisioning
โ”‚   โ”‚   โ””โ”€โ”€ provisioning/              # Auto-provisioned dashboards and datasources
โ”‚   โ”œโ”€โ”€ loki/                          # Loki log aggregation config
โ”‚   โ”œโ”€โ”€ prometheus/                    # Prometheus metrics collection config
โ”‚   โ”œโ”€โ”€ promtail/                      # Promtail log shipper config
โ”‚   โ””โ”€โ”€ tempo/                         # Tempo distributed tracing config
โ”‚
โ”œโ”€โ”€ nginx/                             # Nginx reverse proxy configuration
โ”‚   โ”œโ”€โ”€ conf.d/                        # Individual service proxy configs
โ”‚   โ”‚   โ”œโ”€โ”€ auth.conf                  # Routes auth.domain โ†’ Auth service
โ”‚   โ”‚   โ”œโ”€โ”€ default.conf               # Main domain and catch-all routing
โ”‚   โ”‚   โ”œโ”€โ”€ hasura.conf                # Routes api.domain โ†’ Hasura GraphQL
โ”‚   โ”‚   โ”œโ”€โ”€ mailpit.conf               # Routes mail.domain โ†’ MailPit UI
โ”‚   โ”‚   โ””โ”€โ”€ storage.conf               # Routes storage.domain โ†’ MinIO S3
โ”‚   โ”œโ”€โ”€ ssl/                           # SSL certificate symlinks
โ”‚   โ””โ”€โ”€ nginx.conf                     # Main Nginx configuration
โ”‚
โ”œโ”€โ”€ postgres/                          # PostgreSQL database configuration
โ”‚   โ””โ”€โ”€ init/                          # Database initialization scripts
โ”‚       โ”œโ”€โ”€ 00-init.sql                # Initial database and user setup
โ”‚       โ”œโ”€โ”€ 01-extensions.sql          # Enable PostgreSQL extensions
โ”‚       โ”œโ”€โ”€ 02-schemas.sql             # Create database schemas
โ”‚       โ”œโ”€โ”€ 10-hasura.sql              # Hasura-specific setup
โ”‚       โ””โ”€โ”€ 20-auth.sql                # Auth service database setup
โ”‚
โ”œโ”€โ”€ services/                          # Custom backend services (from templates)
โ”‚   โ”œโ”€โ”€ bullmq_worker/                 # BullMQ job queue worker service
โ”‚   โ”œโ”€โ”€ express_api/                   # Express.js REST API service
โ”‚   โ”œโ”€โ”€ go_grpc/                       # Go gRPC service
โ”‚   โ””โ”€โ”€ python_api/                    # Python FastAPI ML/data service
โ”‚
โ”œโ”€โ”€ ssl/                               # SSL certificates storage
โ”‚   โ””โ”€โ”€ certificates/
โ”‚       โ”œโ”€โ”€ localhost/                 # Local dev certificates
โ”‚       โ””โ”€โ”€ [domain]/                  # Production domain certificates
โ”‚
โ”œโ”€โ”€ storage/                           # MinIO/S3 file storage
โ”‚   โ”œโ”€โ”€ temp/                          # Temporary file uploads
โ”‚   โ””โ”€โ”€ uploads/                       # Persistent file storage
โ”‚
โ”œโ”€โ”€ .volumes/                          # Docker volume mount points (hidden)
โ”‚   โ”œโ”€โ”€ postgres/                      # PostgreSQL data persistence
โ”‚   โ”œโ”€โ”€ redis/                         # Redis data persistence
โ”‚   โ””โ”€โ”€ minio/                         # MinIO object storage data
โ”‚
โ”œโ”€โ”€ docker-compose.yml                 # Main Docker Compose orchestration file
โ”œโ”€โ”€ .env                               # Active environment configuration
โ”œโ”€โ”€ .env.dev                           # Development environment settings
โ”œโ”€โ”€ .env.staging                       # Staging environment settings (optional)
โ”œโ”€โ”€ .env.prod                          # Production environment settings (optional)
โ”œโ”€โ”€ .env.example                       # Example environment template
โ””โ”€โ”€ .gitignore                         # Git ignore rules

Directory Descriptions

Core Infrastructure

_backup/ - Automatic backups created by nself during operations. Each timestamped folder contains previous versions of configuration files, useful for rollback scenarios.

.volumes/ - Docker volume mount points for persistent data storage. This hidden directory contains subdirectories for each stateful service (PostgreSQL, Redis, MinIO) ensuring data persists across container restarts.

Service Configuration

auth/ - Configuration for the authentication service, including JWT settings, OAuth providers, email templates, and security rules.

hasura/ - GraphQL engine configuration containing:

  • metadata/ - Table definitions, relationships, permissions, and remote schemas
  • migrations/ - Versioned database schema changes

functions/ - Serverless functions that extend your backend capabilities:

  • src/ - TypeScript/JavaScript source code
  • dist/ - Compiled production-ready functions

Custom Services

services/ - Your custom backend services generated from templates:

  • Each subdirectory represents a microservice
  • Contains Dockerfile, source code, and configuration
  • Services can be in any language (Node.js, Python, Go, Rust, etc.)
  • Automatically integrated with the rest of the stack

Routing & Networking

nginx/ - Reverse proxy configuration that routes all traffic:

  • conf.d/ - Individual service routing rules
  • ssl/ - Symbolic links to active SSL certificates
  • nginx.conf - Main configuration with security headers and optimization

ssl/certificates/ - SSL/TLS certificates for HTTPS:

  • localhost/ - Self-signed certificates for local development
  • Domain-specific folders for staging/production certificates

Data Layer

postgres/init/ - Database initialization scripts executed in order:

  • 00-init.sql - Database and user creation
  • 01-extensions.sql - Enable extensions (uuid, pgcrypto, postgis, etc.)
  • 02-schemas.sql - Create logical schemas for service isolation
  • 10-hasura.sql - Hasura-specific tables and functions
  • 20-auth.sql - Authentication service schema

storage/ - File storage managed by MinIO (S3-compatible):

  • uploads/ - User-uploaded files
  • temp/ - Temporary files cleared periodically

Observability

monitoring/ - Complete observability stack configuration:

  • grafana/ - Dashboards and data source provisioning
  • prometheus/ - Metrics collection rules and targets
  • loki/ - Log aggregation configuration
  • tempo/ - Distributed tracing setup
  • promtail/ - Log shipping from containers to Loki

logs/ - Application logs from all services, useful for debugging and audit trails.

Environment Files

nself supports multiple environment configurations:

.env                  # Active configuration (symlink or copy)
.env.dev              # Development settings (local development)
.env.staging          # Staging settings (pre-production testing)
.env.prod             # Production settings (live environment)
.env.example          # Template with all available options

Environment Selection

  • Development: Uses .env.dev with debug enabled, relaxed security
  • Staging: Uses .env.staging with production-like settings for testing
  • Production: Uses .env.prod with security hardening, optimizations

The active environment is determined by:

  1. Direct .env file (highest priority)
  2. .env.local (local overrides)
  3. .env.[environment] based on ENV variable
  4. .env.example as fallback template

File Patterns

Template Files (*.template)

Files ending in .template contain placeholders that are replaced during build:

  • {{SERVICE_NAME}} - Name of the service
  • {{SERVICE_PORT}} - Port number for the service
  • {{PROJECT_NAME}} - Your project name
  • {{BASE_DOMAIN}} - Base domain for routing

Configuration Files

  • *.conf - Nginx configuration files
  • *.yml/*.yaml - Docker Compose and service configurations
  • *.sql - Database initialization and migration scripts
  • *.json - Service metadata and package definitions

Key Concepts

Service Isolation

Each service runs in its own container with isolated:

  • Network namespace (communication via Docker network)
  • File system (only specified volumes are shared)
  • Process space (independent process trees)
  • Resource limits (CPU/memory constraints)

Data Persistence

Stateful data is stored in .volumes/ ensuring:

  • Data survives container restarts
  • Easy backup and restoration
  • Consistent permissions and ownership

Multi-Tenancy Support

The structure supports multiple frontend applications through:

  • Table prefixing in PostgreSQL
  • Separate auth scopes
  • Isolated storage buckets
  • Independent Redis namespaces

Security Layers

  • SSL/TLS encryption for all services
  • Network isolation via Docker networks
  • Secret management via environment variables
  • Rate limiting and security headers in Nginx

Best Practices

  1. Never commit .env files - Use .env.example as template
  2. Keep sensitive data in .volumes/ - Excluded from version control
  3. Use services/ for custom code - Templates ensure consistency
  4. Monitor _backup/ size - Clean old backups periodically
  5. Organize migrations chronologically - Use numbered prefixes
  6. Document service dependencies - In service README files

Common Operations

Adding a New Service

  1. Define in .env file using CS_N variables:
    CS_1=my_service:express-js:8001
  2. Run nself build to generate from template
  3. Customize code in services/my_service/
  4. Rebuild with nself start

Environment Promotion

# Development to Staging
cp .env.dev .env.staging
# Edit .env.staging with staging values

# Staging to Production
cp .env.staging .env.prod
# Edit .env.prod with production values

Backup Critical Data

# Backup database
docker exec [project]_postgres pg_dump -U postgres [database] > backup.sql

# Backup volumes
tar -czf volumes-backup.tar.gz .volumes/

# Backup configurations
tar -czf config-backup.tar.gz nginx/ postgres/init/ services/

Stack Overview

A complete nself deployment includes 30+ integrated services:

Core Services

  • PostgreSQL - Primary database with extensions
  • Hasura - GraphQL API engine
  • Auth - Authentication and authorization
  • Storage - MinIO S3-compatible object storage
  • Nginx - Reverse proxy and SSL termination

Optional Services

  • Redis - Caching and session storage
  • BullMQ - Job queue and background processing
  • MeiliSearch - Full-text search engine
  • MailPit - Email testing (dev) / SMTP (prod)
  • Functions - Serverless compute

Monitoring Stack

  • Prometheus - Metrics collection
  • Grafana - Visualization and dashboards
  • Loki - Log aggregation
  • Tempo - Distributed tracing
  • Alertmanager - Alert routing and management

Custom Services

  • Up to 10 custom microservices from templates
  • Support for any programming language
  • Automatic integration with infrastructure

Related Documentation

โš ๏ธ **GitHub.com Fallback** โš ๏ธ