Development Setup - jra3/mulm GitHub Wiki

Development Setup

This guide will help you set up a local development environment for the Mulm Breeder Awards Program platform.

Prerequisites

Before you begin, ensure you have the following installed:

Required Software

Software Version Installation
Node.js 20.x or higher nodejs.org
npm 10.x or higher Comes with Node.js
Git Latest git-scm.com
SQLite3 Latest Usually pre-installed on macOS/Linux, sqlite.org for Windows

Optional Software

Software Purpose
VS Code Recommended IDE with TypeScript support
Docker Desktop For testing production build locally
Postman For testing API endpoints

Verify Installation

node --version    # Should be v20.x.x or higher
npm --version     # Should be v10.x.x or higher
git --version     # Any recent version
sqlite3 --version # Any version

Quick Start

1. Clone Repository

git clone https://github.com/jra3/mulm.git
cd mulm

2. Install Dependencies

npm install

This installs all dependencies from package.json, including:

  • Runtime dependencies: Express, SQLite, Sharp, Pug, etc.
  • Dev dependencies: TypeScript, ESLint, tsx, testing tools

Expected time: 2-5 minutes depending on your internet connection.

3. Create Configuration File

# Create config from template
cat > src/config.json << 'EOF'
{
  "databaseFile": "./database/database.db",
  "domain": "http://localhost:4200",
  "googleClientId": "",
  "googleClientSecret": "",
  "adminsEmail": "[email protected]",
  "bugReportEmail": "[email protected]",
  "fromEmail": "[email protected]",
  "smtpPassword": "",
  "smtpHost": "smtp.example.com",
  "smtpPort": 587,
  "smtpSecure": false,
  "s3AccessKeyId": "",
  "s3Secret": "",
  "s3Url": "",
  "s3Bucket": "",
  "r2PublicUrl": ""
}
EOF

Note: Most features work without OAuth/SMTP/R2 configured. See Optional Configuration below.

4. Start Development Server

npm run dev

This starts:

  • Nodemon - Auto-reloads TypeScript on file changes
  • PostCSS - Watches and rebuilds CSS
  • Express server on http://localhost:4200

Expected output:

[nodemon] starting `tsx src/index.ts`
Server is running on http://localhost:4200
Migration 001-init.sql applied successfully
Migration 002-add-tank-presets.sql applied successfully
...

5. Open Application

Visit http://localhost:4200 in your browser.

You should see the Mulm homepage with:

  • Navigation menu
  • Activity feed (empty initially)
  • "Sign In" button

Configuration Details

Configuration File Structure

The src/config.json file contains all application settings:

interface Config {
  databaseFile: string;          // SQLite database path
  domain: string;                // App URL (for OAuth redirects)
  googleClientId: string;        // Google OAuth client ID
  googleClientSecret: string;    // Google OAuth secret
  adminsEmail: string;           // Admin notifications recipient
  bugReportEmail: string;        // Bug report recipient
  fromEmail: string;             // Sender email for notifications
  smtpPassword: string;          // SMTP password
  smtpHost: string;              // SMTP server hostname
  smtpPort: number;              // SMTP port (usually 587)
  smtpSecure: boolean;           // Use TLS (usually false for port 587)
  s3AccessKeyId: string;         // R2/S3 access key
  s3Secret: string;              // R2/S3 secret key
  s3Url: string;                 // R2 endpoint URL
  s3Bucket: string;              // R2 bucket name
  r2PublicUrl: string;           // R2 public URL (custom domain)
}

Minimal Configuration

For basic local development, you only need:

{
  "databaseFile": "./database/database.db",
  "domain": "http://localhost:4200",
  "googleClientId": "",
  "googleClientSecret": "",
  "adminsEmail": "[email protected]",
  "bugReportEmail": "[email protected]",
  "fromEmail": "[email protected]",
  "smtpPassword": "",
  "smtpHost": "smtp.example.com",
  "smtpPort": 587,
  "smtpSecure": false,
  "s3AccessKeyId": "",
  "s3Secret": "",
  "s3Url": "",
  "s3Bucket": "",
  "r2PublicUrl": ""
}

What works:

  • ✅ Database and migrations
  • ✅ Member accounts (email/password)
  • ✅ Submissions
  • ✅ Admin approval workflow
  • ✅ Testing

What doesn't work:

  • ❌ Google OAuth login
  • ❌ Email notifications
  • ❌ Image uploads

Optional Configuration

Google OAuth (Optional)

To enable Google sign-in:

  1. Create Google OAuth App:

    • Visit Google Cloud Console
    • Create new project or select existing
    • Enable "Google+ API"
    • Create OAuth 2.0 credentials
    • Add authorized redirect URI: http://localhost:4200/oauth/google
  2. Add to config.json:

    {
      "googleClientId": "your-client-id.apps.googleusercontent.com",
      "googleClientSecret": "your-client-secret"
    }

SMTP Email (Optional)

To enable email notifications:

  1. Use Gmail (easiest for testing):

    • Enable 2FA on your Google account
    • Generate app-specific password
    • Or use Mailtrap for dev email testing
  2. Add to config.json:

    {
      "smtpHost": "smtp.gmail.com",
      "smtpPort": 587,
      "smtpSecure": false,
      "smtpPassword": "your-app-password",
      "fromEmail": "[email protected]",
      "adminsEmail": "[email protected]",
      "bugReportEmail": "[email protected]"
    }

Cloudflare R2 (Optional)

To enable image uploads:

  1. Create R2 bucket:

    • Log in to Cloudflare dashboard
    • Navigate to R2
    • Create bucket (e.g., "mulm-dev")
    • Generate API token with R2 read/write permissions
  2. Add to config.json:

    {
      "s3Url": "https://<account-id>.r2.cloudflarestorage.com",
      "s3AccessKeyId": "your-access-key-id",
      "s3Secret": "your-secret-key",
      "s3Bucket": "mulm-dev",
      "r2PublicUrl": "https://mulm-dev.<account-id>.r2.dev"
    }

Database Setup

Automatic Migration

The database is created and migrated automatically when you run:

npm run dev

What happens:

  1. Application checks if database/database.db exists
  2. If not, creates empty SQLite database
  3. Runs all migrations from db/migrations/ in order
  4. Records applied migrations in migrations table

Migration files:

db/migrations/
├── 001-init.sql                    # Initial schema
├── 002-add-tank-presets.sql        # Tank presets
├── 003-species-name-tracking.sql   # Species catalog
├── ...
└── 015-add-submission-notes.sql    # Latest migration

Manual Database Reset

To start with a fresh database:

# Stop dev server (Ctrl+C)

# Delete database
rm database/database.db

# Restart dev server (recreates database)
npm run dev

Inspecting Database

# Open SQLite CLI
sqlite3 database/database.db

# Common queries
sqlite> .schema members           # View members table
sqlite> SELECT * FROM members;    # View all members
sqlite> .tables                   # List all tables
sqlite> .quit                     # Exit

GUI Tools:


Development Workflow

Running the Application

npm run dev

What runs:

  • concurrently - Runs multiple commands in parallel
  • nodemon - Watches TypeScript files, restarts on changes
  • postcss --watch - Watches CSS files, rebuilds on changes

Hot reload enabled:

  • Edit any .ts file → Server restarts automatically
  • Edit any .css file → CSS rebuilds automatically
  • Edit any .pug file → Changes visible on next page load

Available Commands

# Development
npm run dev              # Start dev server with hot reload
npm run build            # Build TypeScript + CSS for production
npm start                # Start production build (requires build first)

# Testing
npm test                 # Run all tests
npm run test:watch       # Run tests in watch mode
npm test -- path/to/test # Run specific test file

# Code Quality
npm run lint             # Check code with ESLint
npm run lint:fix         # Auto-fix ESLint issues

# Utilities
npm run script <file>    # Run TypeScript script (e.g., npm run script scripts/seed-data.ts)
npm run postcss          # Build CSS once (no watch)

Development Server Ports

Port Service URL
4200 Express app http://localhost:4200

Note: Port 4200 is hardcoded in src/index.ts. Change if needed.

File Watching

Auto-reload triggers:

File Type Trigger Action
src/**/*.ts Nodemon Restart server
src/**/*.css PostCSS Rebuild CSS
src/views/**/*.pug None Manual refresh browser
db/migrations/*.sql None Restart server

If changes don't appear:

  1. Check terminal for errors
  2. Hard refresh browser (Cmd+Shift+R / Ctrl+Shift+R)
  3. Restart dev server

Project Structure

mulm/
├── db/
│   └── migrations/              # SQL migration files (auto-applied)
│
├── infrastructure/              # AWS CDK code for production deploy
│   ├── bin/                     # CDK app entry point
│   ├── lib/                     # CDK stack definitions
│   └── test/                    # Infrastructure tests
│
├── nginx/                       # Nginx config for production
│   ├── nginx.conf               # Main config
│   └── conf.d/                  # Site configs
│
├── scripts/                     # Utility scripts
│   ├── seed-data.ts             # Generate test data
│   └── init-letsencrypt.sh      # SSL cert setup
│
├── src/
│   ├── __tests__/               # Test files (*.test.ts)
│   │   ├── testDbHelper.helper.ts  # Test utilities
│   │   └── *.test.ts            # Test suites
│   │
│   ├── db/                      # Database layer
│   │   ├── conn.ts              # Connection management
│   │   ├── members.ts           # Member queries
│   │   ├── submissions.ts       # Submission queries
│   │   └── ...
│   │
│   ├── forms/                   # Zod validation schemas
│   │   ├── submission.ts        # Submission form validation
│   │   ├── member.ts            # Member form validation
│   │   └── ...
│   │
│   ├── routes/                  # Express route handlers
│   │   ├── submission.ts        # /submissions routes
│   │   ├── admin.ts             # Admin route handlers
│   │   ├── adminRouter.ts       # /admin/* router
│   │   └── ...
│   │
│   ├── types/                   # TypeScript type definitions
│   │   ├── config.d.ts          # Config interface
│   │   └── api-responses.ts     # API response types
│   │
│   ├── utils/                   # Utility functions
│   │   ├── logger.ts            # Logging utility
│   │   ├── r2-client.ts         # R2 storage client
│   │   ├── image-processor.ts   # Sharp image processing
│   │   └── ...
│   │
│   ├── views/                   # Pug templates
│   │   ├── layout.pug           # Base layout
│   │   ├── index.pug            # Homepage
│   │   ├── submit.pug           # Submission form
│   │   └── admin/               # Admin templates
│   │
│   ├── config.json              # Config file (git-ignored)
│   ├── index.ts                 # App entry point
│   └── sessions.ts              # Session management
│
├── database/                    # SQLite database (created at runtime)
│   └── database.db              # Development database
│
├── public/                      # Static assets (generated)
│   └── index.css                # Compiled CSS
│
├── dist/                        # Compiled TypeScript (generated)
│
├── .gitignore                   # Git ignore rules
├── package.json                 # Dependencies and scripts
├── tsconfig.json                # TypeScript configuration
├── tailwind.config.js           # Tailwind CSS configuration
├── postcss.config.js            # PostCSS configuration
└── CLAUDE.md                    # Comprehensive project documentation

Key Files

File Purpose
src/index.ts Application entry point, Express setup
src/db/conn.ts Database connection management
src/sessions.ts Cookie session management
CLAUDE.md Comprehensive project documentation
package.json Dependencies and npm scripts
tsconfig.json TypeScript compiler options

Creating Your First Account

1. Start Server

npm run dev

2. Visit Homepage

Open http://localhost:4200

3. Click "Sign In"

Click the "Sign In" button in the top navigation.

4. Create Account

Click "Create Account" tab, then fill in:

5. Log In

You're now logged in! You should see:

  • Your name in top navigation
  • "Submit Breeding" button
  • "My Profile" link

6. Make Yourself Admin (Optional)

To access admin features:

sqlite3 database/database.db
sqlite> UPDATE members SET is_admin = 1 WHERE contact_email = '[email protected]';
sqlite> .quit

Refresh the page. You should now see "Admin" menu items.


Common Development Tasks

Adding a New Route

1. Create route handler in src/routes/:

// src/routes/myfeature.ts
import { MulmRequest } from '@/sessions';
import { Response } from 'express';

export const showMyFeature = async (req: MulmRequest, res: Response) => {
  res.render('myfeature', {
    title: 'My Feature'
  });
};

2. Register route in src/index.ts:

import { showMyFeature } from './routes/myfeature';

// Add route
app.get('/myfeature', showMyFeature);

3. Create view in src/views/:

// src/views/myfeature.pug
extends layout

block content
  h1 My Feature
  p This is my new feature!

4. Visit route:

http://localhost:4200/myfeature

Adding a Database Migration

1. Create migration file:

# Create new migration in db/migrations/
touch db/migrations/016-add-my-feature.sql

2. Write migration SQL:

-- db/migrations/016-add-my-feature.sql
-- Up

ALTER TABLE submissions ADD COLUMN my_field TEXT DEFAULT NULL;
CREATE INDEX idx_submissions_my_field ON submissions(my_field);

-- Down

DROP INDEX IF EXISTS idx_submissions_my_field;
-- Note: SQLite doesn't support DROP COLUMN

3. Restart server:

# Ctrl+C to stop
npm run dev

Migration applies automatically on startup.

4. Verify:

sqlite3 database/database.db
sqlite> .schema submissions
# Should see my_field column

Running Tests

All tests:

npm test

Specific test file:

npm test -- src/__tests__/waitingPeriod.test.ts

Watch mode (re-run on file changes):

npm run test:watch

Expected output:

✔ getRequiredWaitingDays > returns 30 days for marine fish (1.2ms)
✔ getRequiredWaitingDays > returns 60 days for freshwater fish (0.4ms)
...
All tests passed (15/15)

Seeding Test Data

To add test submissions and members:

// scripts/seed-data.ts
import { createMember } from '../src/db/members';
import { db } from '../src/db/conn';

async function seed() {
  // Create test member
  const memberId = await createMember('[email protected]', 'Test User');

  // Create test submission
  await db().run(`
    INSERT INTO submissions (member_id, species_type, species_class, ...)
    VALUES (?, 'Fish', 'Catfish', ...)
  `, [memberId]);

  console.log('Seed data created!');
}

seed();

Run:

npm run script scripts/seed-data.ts

Troubleshooting

Port Already in Use

Error:

Error: listen EADDRINUSE: address already in use :::4200

Solution:

# Find process using port 4200
lsof -ti:4200

# Kill process
kill -9 $(lsof -ti:4200)

# Or use different port in src/index.ts

Database Locked

Error:

Error: SQLITE_BUSY: database is locked

Solution:

# Close all database connections
# Close sqlite3 CLI if open
# Close DB Browser / TablePlus
# Restart dev server

Migration Failed

Error:

Migration 016-my-feature.sql failed: ...

Solution:

  1. Check SQL syntax in migration file
  2. Ensure migration file name follows pattern: NNN-description.sql
  3. Verify -- Up and -- Down section markers exist
  4. Reset database and retry:
    rm database/database.db
    npm run dev

Module Not Found

Error:

Cannot find module '@/utils/logger'

Solution:

# Rebuild TypeScript
npm run build

# Check tsconfig paths are correct
# Restart dev server

CSS Not Updating

Problem: Tailwind CSS changes don't appear

Solution:

  1. Check PostCSS is running (should see in terminal output)
  2. Hard refresh browser (Cmd+Shift+R / Ctrl+Shift+R)
  3. Restart dev server
  4. Check public/index.css was generated

Tests Failing

Error:

AssertionError: Expected 5 to equal 10

Solution:

  1. Read error message carefully
  2. Check if database schema changed (migrations)
  3. Check if test helpers need updating
  4. Run single test to isolate issue:
    npm test -- src/__tests__/failing-test.test.ts

IDE Setup

VS Code (Recommended)

Recommended Extensions:

  • ESLint (dbaeumer.vscode-eslint)
  • Prettier (esbenp.prettier-vscode)
  • TypeScript Hero (rbbit.typescript-hero)
  • Tailwind CSS IntelliSense (bradlc.vscode-tailwindcss)
  • SQLite Viewer (alexcvzz.vscode-sqlite)
  • Pug (amandeepmittal.pug)

Settings (.vscode/settings.json):

{
  "editor.formatOnSave": true,
  "editor.defaultFormatter": "esbenp.prettier-vscode",
  "eslint.validate": ["typescript"],
  "typescript.tsdk": "node_modules/typescript/lib"
}

Debugging in VS Code

Create .vscode/launch.json:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug Dev Server",
      "type": "node",
      "request": "launch",
      "runtimeExecutable": "npm",
      "runtimeArgs": ["run", "dev"],
      "console": "integratedTerminal",
      "internalConsoleOptions": "neverOpen"
    },
    {
      "name": "Debug Tests",
      "type": "node",
      "request": "launch",
      "runtimeExecutable": "npm",
      "runtimeArgs": ["test"],
      "console": "integratedTerminal"
    }
  ]
}

Usage:

  1. Set breakpoints in TypeScript files
  2. Press F5 or Run > Start Debugging
  3. Debugger pauses at breakpoints

Next Steps

Now that your development environment is set up:

  1. Explore the codebase:

    • Read CLAUDE.md for comprehensive documentation
    • Browse src/routes/ to understand route structure
    • Check src/views/ for template examples
  2. Make your first change:

    • Pick a small feature or bug
    • Write a test first (see Testing Guide)
    • Implement the change
    • Run tests and lint
  3. Learn the patterns:

  4. Join development:


Getting Help

Documentation

Community

Common Resources

⚠️ **GitHub.com Fallback** ⚠️