LDAP Server Administration - thebaulab/onramp GitHub Wiki

LDAP Server Administration

This page documents the LDAP server that provides centralized authentication for the baulab.us cluster.

Quick Reference

Property Value
Server baunames (10.200.205.143)
Base DN dc=baulab,dc=us
Admin DN cn=admin,dc=baulab,dc=us
phpLDAPadmin Port 8877 (via SSH tunnel)

What is LDAP and Why Do We Use It?

The Problem: Managing Users Across Many Machines

The cluster has 25+ GPU workstations. Without LDAP, adding a new lab member would require SSHing into each machine, running adduser, and setting the password - 25 times. Password changes and account deletions would require the same.

The Solution: One Central User Database

LDAP (Lightweight Directory Access Protocol) is essentially a specialized database optimized for storing organizational data like user accounts, groups, and contact information. Think of it as a shared phonebook that all computers in the cluster can read.

With LDAP:

  • Add a user once → they can log into any machine
  • Change a password once → it works everywhere immediately
  • Delete a user once → they're locked out of everything

Our LDAP server runs on baunames (the "names" server for the baulab cluster).

How It's Different from a Regular Database

You might wonder: why not just use MySQL or PostgreSQL? LDAP is designed specifically for:

  1. Read-heavy workloads: User lookups happen constantly (every login, every ls -l), but changes are rare
  2. Hierarchical data: Users belong to groups, which belong to organizational units
  3. Standardized protocol: Any Linux system can query LDAP out of the box

Understanding the Login Process

When someone types ssh arnab@karakuri, here's what happens behind the scenes:

┌─────────────────────────────────────────────────────────────────────────┐
│                    USER'S LAPTOP                                        │
│                                                                         │
│   $ ssh arnab@karakuri                                                  │
│                                                                         │
└───────────────────────────────┬─────────────────────────────────────────┘
                                │ SSH connection
                                ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                    KARAKURI (GPU Workstation)                           │
│                                                                         │
│   1. "Who is 'arnab'?" → NSS checks /etc/passwd                         │
│      Not found locally (arnab isn't in /etc/passwd)                     │
│                                                                         │
│   2. NSS asks SSSD → SSSD queries LDAP server                           │
│                                │                                        │
└────────────────────────────────┼────────────────────────────────────────┘
                                 │ LDAP query: "Give me info for uid=arnab"
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                    BAUNAMES (LDAP Server)                               │
│                                                                         │
│   OpenLDAP looks up arnab in its database:                              │
│   - uidNumber: 1007                                                     │
│   - gidNumber: 1025 (research group)                                    │
│   - homeDirectory: /share/u/arnab                                       │
│   - userPassword: {SSHA}encrypted...                                    │
│                                                                         │
└────────────────────────────────┬────────────────────────────────────────┘
                                 │ Returns user info
                                 ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                    KARAKURI (continued)                                 │
│                                                                         │
│   3. SSSD receives: uid=1007, gid=1025, home=/share/u/arnab             │
│                                                                         │
│   4. PAM (Pluggable Auth Module) asks SSSD to verify password           │
│      → SSSD tries to "bind" to LDAP as arnab with the password          │
│      → LDAP checks password hash, returns success/failure               │
│                                                                         │
│   5. If password correct:                                               │
│      - Create session with uid 1007, gid 1025                           │
│      - Change to home directory /share/u/arnab                          │
│      - If home doesn't exist, pam_mkhomedir creates it                  │
│                                                                         │
│   $ whoami                                                              │
│   arnab                                                                 │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

The Key Players

Component What It Does Where It Runs
OpenLDAP (slapd) The actual database server storing user/group info baunames only
SSSD Daemon that queries LDAP and caches results Every client machine
NSS System that maps usernames ↔ user IDs Every client machine
PAM System that handles authentication (password checking) Every client machine

Why SSSD caches results: If baunames goes down briefly, users who recently logged in can still work because SSSD remembers their info. New users or password changes won't work until LDAP is back.

The Directory Tree: How Data is Organized

LDAP stores data in a tree structure (called a DIT - Directory Information Tree). Each entry has a unique path called a Distinguished Name (DN):

dc=baulab,dc=us                              ← Root (like / in filesystem)
│
├── cn=admin                                 ← The super-admin account
│
├── ou=people                                ← "Organizational Unit" for users
│   │
│   ├── uid=davidbau                         ← A user entry
│   │   ├── cn: David Bau                    ← Full name
│   │   ├── uidNumber: 1000                  ← Unix user ID
│   │   ├── gidNumber: 1025                  ← Primary group (research)
│   │   ├── homeDirectory: /share/u/davidbau
│   │   ├── loginShell: /bin/bash
│   │   └── userPassword: {SSHA}...          ← Hashed password
│   │
│   ├── uid=arnab
│   │   └── ... (same attributes)
│   │
│   └── uid=newstudent
│       └── ... (~23 users total)
│
└── ou=groups                                ← Organizational Unit for groups
    │
    ├── cn=research                          ← Default group for all users
    │   ├── gidNumber: 1025
    │   ├── memberUid: davidbau
    │   ├── memberUid: arnab
    │   └── memberUid: ...
    │
    └── cn=admin                             ← Admin group (special privileges)
        └── memberUid: davidbau

Understanding Distinguished Names (DNs)

The DN is like a file path, but read from leaf to root:

  • uid=arnab,ou=people,dc=baulab,dc=us means "the arnab entry, in the people unit, in baulab.us"

The prefixes mean:

  • dc = Domain Component (baulab.us becomes dc=baulab,dc=us)
  • ou = Organizational Unit (a folder/container)
  • uid = User ID (the username)
  • cn = Common Name (human-readable name)

Schema: What Makes Up a User or Group

User Entries

Each user entry uses several "object classes" that define what attributes it can have:

Object Class Purpose Required Attributes
posixAccount Unix user info uid, uidNumber, gidNumber, homeDirectory
shadowAccount Password aging (optional shadow* attributes)
account Basic account uid
top Base class (inherited by all)

Here's what a complete user entry looks like in LDIF format (LDAP Data Interchange Format):

dn: uid=arnab,ou=people,dc=baulab,dc=us
objectClass: account
objectClass: posixAccount
objectClass: shadowAccount
objectClass: top
uid: arnab
cn: Arnab Sen Sharma
uidNumber: 1007
gidNumber: 1025
homeDirectory: /share/u/arnab
loginShell: /bin/bash
userPassword: {SSHA}xxxxxxxxxxxx
shadowLastChange: 19658
shadowMax: 99999
shadowWarning: 7

Important fields when adding users:

Attribute What to Set Notes
uid username Lowercase, no spaces
cn "First Last" User's real name
uidNumber next available Look at last user, add 1
gidNumber 1025 Always use research group
homeDirectory /share/u/USERNAME Must match uid
userPassword initial password phpLDAPadmin will hash it

Group Entries

Groups are simpler - they just list their members:

dn: cn=research,ou=groups,dc=baulab,dc=us
objectClass: posixGroup
objectClass: top
cn: research
gidNumber: 1025
memberUid: davidbau
memberUid: arnab
memberUid: newstudent

Current groups:

Group GID Purpose
research 1025 Default group for all lab members
admin - Administrative privileges
lighting - Lighting research project

Accessing phpLDAPadmin

phpLDAPadmin is a web interface for managing LDAP. It makes adding users and changing passwords much easier than command-line tools.

Why SSH Tunneling is Required

phpLDAPadmin isn't exposed to the internet for security reasons. Instead, you create an SSH tunnel that forwards a port from your laptop to baunames:

Your Laptop                 Khoury Jump Host              baunames
localhost:8877  ──────────►  login.khoury  ──────────►   localhost:8877
     │                         (relay)                        │
     └─────────────── SSH tunnel connection ──────────────────┘

Setting Up the SSH Tunnel

Add this to your laptop's ~/.ssh/config:

Host baunames
    ProxyJump login.khoury.northeastern.edu
    User localdavidbau
    LocalForward 8877 localhost:8877

Host login.khoury.northeastern.edu
    User YOUR_KHOURY_USERNAME

What this means:

  • ProxyJump: Connect through Khoury's jump host first (required - baunames isn't directly reachable)
  • User localdavidbau: Log in as the local admin (not your LDAP user)
  • LocalForward 8877 localhost:8877: Make baunames port 8877 appear as your localhost:8877

Connecting to phpLDAPadmin

  1. Open a terminal and run: ssh baunames

    • Keep this terminal open (the tunnel only works while connected)

  2. Open your browser to: http://localhost:8877/

  3. Click "login" in the left sidebar

  4. Enter credentials:

    • Login DN: cn=admin,dc=baulab,dc=us
    • Password: Ask David or Arnab

You'll see a tree view of the directory on the left. Navigate by clicking the triangles to expand sections.

Common Administrative Tasks

Adding a New Lab Member

Before you start: Collect their preferred username and full name.

  1. In phpLDAPadmin, expand dc=baulab,dc=us → ou=people

  2. Click on the last user that was added (to see the highest uidNumber)

    • Note their uidNumber - you'll use the next number

  3. Click "Copy or move this entry" (at the top of the entry view)

  4. Set the new DN: uid=NEWUSERNAME,ou=people,dc=baulab,dc=us

    • Replace NEWUSERNAME with their actual username

  5. Click "Copy"

  6. Now edit the copied entry - change these fields:

    • uid: their username (should already be correct from the DN)
    • cn: their full name ("Jane Smith")
    • uidNumber: increment by 1 from the user you copied
    • homeDirectory: /share/u/NEWUSERNAME
    • userPassword: click to set their initial password
    • Leave gidNumber as 1025 (research group)

  7. Click "Update Object"

  8. Important: Also add the same user to the baukit.org cluster (David's home network uses the same user list)

Changing Someone's Password

  1. Navigate to the user in ou=people
  2. Click on the userPassword attribute value
  3. Enter the new password
  4. Select "ssha" as the hash type (Salted SHA - most secure option)
  5. Click "Update Object"

Best practice: Set both baulab.us and baukit.org passwords at the same time so they match.

Adding a User to a Group

Why would you do this? Maybe you have a shared project directory that only certain people should access.

  1. Navigate to ou=groups → click on the group (e.g., cn=lighting)
  2. Click "Add new attribute"
  3. Select memberUid from the dropdown
  4. Enter the username (just arnab, not the full DN)
  5. Click "Update Object"

Command-Line Administration

Sometimes the command line is faster, especially for quick queries.

Searching the Directory

# List all users (shows just uid and cn)
ldapsearch -x -H ldap://localhost -b "ou=people,dc=baulab,dc=us" uid cn

# Find a specific user (shows all their attributes)
ldapsearch -x -H ldap://localhost -b "dc=baulab,dc=us" "(uid=arnab)"

# List all groups
ldapsearch -x -H ldap://localhost -b "ou=groups,dc=baulab,dc=us" cn memberUid

# Find highest uidNumber (useful when adding users)
ldapsearch -x -H ldap://localhost -b "ou=people,dc=baulab,dc=us" uidNumber | grep uidNumber | sort -t: -k2 -n | tail -1

What the flags mean:

  • -x: Use simple authentication (not SASL)
  • -H ldap://localhost: Connect to LDAP on this machine
  • -b "...": Search starting from this DN (the "base")

Changing a Password via Command Line

ldappasswd -x -D "cn=admin,dc=baulab,dc=us" -W -S "uid=arnab,ou=people,dc=baulab,dc=us"
  • -D "cn=admin,...": Authenticate as admin
  • -W: Prompt for admin password
  • -S: Prompt for new user password

Backup Procedures

WARNING: As of this writing, no automated backups are configured! This should be fixed.

Why Backups Matter

The LDAP database contains all user accounts. If it's corrupted or lost:

  • Nobody can log into any machine
  • All user IDs become "unknown"
  • File permissions become meaningless (files owned by uid 1007 instead of "arnab")

Manual Backup

Run these commands on baunames:

# Backup the server configuration (schemas, access rules, etc.)
sudo slapcat -n 0 -l /share/backup/config/ldap-config-$(date +%Y%m%d).ldif

# Backup the actual user/group data
sudo slapcat -n 1 -l /share/backup/config/ldap-data-$(date +%Y%m%d).ldif

# Backup TLS certificates
sudo cp /etc/ldap/baunames_slapd_*.pem /share/backup/config/

What slapcat does: Exports the LDAP database to a text file (LDIF format) that can be reimported on a new server.

Setting Up Automated Backups

Add to root's crontab on baunames:

sudo crontab -e
# Add this line:
0 3 * * 0 /home/localdavidbau/admin/lab/scripts/backup_ldap_full.sh >> /var/log/ldap-backup.log 2>&1

This runs the backup script every Sunday at 3 AM.

Troubleshooting

User Can't Log In

Start from the user's machine and work backward:

Step 1: Check if the system knows the user exists

getent passwd username

If this returns nothing, the machine can't reach LDAP or the user doesn't exist.

Step 2: Check SSSD (the LDAP client daemon)

sudo systemctl status sssd

If it's not running, start it: sudo systemctl start sssd

Step 3: Test LDAP connectivity directly

ldapsearch -x -H ldap://baunames -b "dc=baulab,dc=us" "(uid=username)"

If this fails, either the network is down or slapd isn't running on baunames.

Step 4: Clear the SSSD cache and retry

sudo sss_cache -E
sudo systemctl restart sssd

Sometimes SSSD caches stale data.

slapd (LDAP Server) Not Running

On baunames:

# Check status
sudo systemctl status slapd

# View recent logs
sudo journalctl -u slapd -n 50

# Start if stopped
sudo systemctl start slapd

Common causes:

  • Database corruption (rare) - restore from backup
  • Config error after manual changes - check journalctl logs
  • Disk full - check df -h

phpLDAPadmin Issues

Problem Likely Cause Solution
Browser can't connect to localhost:8877 SSH tunnel not running Make sure ssh baunames is connected
"Unable to connect to LDAP server" slapd not running sudo systemctl start slapd on baunames
Login fails with correct password Typo in DN Use exactly: cn=admin,dc=baulab,dc=us

Server Technical Details

Key Files on baunames

Path Purpose
/etc/ldap/slapd.d/ Server configuration (cn=config format)
/var/lib/ldap/ Database files (LMDB format, ~220KB)
/etc/ldap/baunames_slapd_cert.pem TLS certificate
/etc/ldap/baunames_slapd_key.pem TLS private key (readable only by openldap user)
/etc/phpldapadmin/config.php phpLDAPadmin configuration
/etc/ldap/ldap.conf LDAP client configuration

Services

# LDAP server daemon
sudo systemctl {start|stop|restart|status} slapd

# Apache (serves phpLDAPadmin)
sudo systemctl {start|stop|restart|status} apache2

Local Admin Accounts

baunames (like all machines) has local admin accounts that work even if LDAP is broken:

  • localdavidbau
  • localarnab

These are in /etc/passwd and have passwordless sudo. Always use these for admin work.

Why Two Clusters (baulab.us and baukit.org)?

You may have noticed references to two LDAP servers. The lab maintains parallel infrastructure:

Aspect baulab.us (Khoury) baukit.org (David's home)
LDAP Server baunames (10.200.205.143) names (192.168.0.2)
Base DN dc=baulab,dc=us dc=thevisible,dc=net
phpLDAPadmin Port 8877 Port 8876
Purpose Lab GPU cluster Personal development

Important: When adding users, add them to both clusters so they have consistent access everywhere.

Related Pages

Further Reading