Systemctl Troubleshooting - ryzendew/Linux-Tips-and-Tricks GitHub Wiki
Systemctl Troubleshooting for Beginners
Table of Contents
- :pencil: What is systemctl?
- :zap: Basic Commands
- :mag: Checking Service Status
- :desktop: Listing Services
- :wrench: Controlling Services
- Starting a Service
- Stopping a Service
- Restarting a Service
- Reloading a Service
- Try-Restart (Restart Only If Running)
- Reload-or-Restart
- Enable Service to Start at Boot
- Disable Service from Starting at Boot
- Enable and Start Together
- Disable and Stop Together
- Masking a Service
- Scenario 1: Service Won't Start
- Scenario 2: Service Keeps Crashing
- Scenario 3: Service Not Starting at Boot
- Scenario 4: Finding What's Using a Port
- Scenario 5: Service Takes Too Long to Start
- Scenario 6: Understanding Service Dependencies
- Status Commands
- Control Commands
- Enable/Disable Commands
- Listing Commands
- Information Commands
- Maintenance Commands
:pencil: What is systemctl?
systemctlis the command-line tool for managing systemd services and units- It allows you to control, monitor, and troubleshoot system services
- systemd is the system and service manager used by most modern Linux distributions
- Services are called "units" in systemd terminology
What systemctl can do:
- Start, stop, restart, and reload services
- Check service status and health
- Enable/disable services to start at boot
- View service logs
- Check service dependencies
- Troubleshoot failed services
:zap: Basic Commands
Getting Help
systemctl --help
Shows all available systemctl commands and options.
systemctl status
Shows the overall system status, including:
- System state (running, degraded, etc.)
- Number of loaded units
- Number of failed units
- System uptime
- systemd version
Example output:
● fedora
State: running
Units: 746 loaded (incl. loaded aliases)
Jobs: 0 queued
Failed: 0 units
Since: Thu 2025-11-13 03:33:58 AST; 11h ago
systemd: 258.1-1.fc43
:mag: Checking Service Status
Viewing a Specific Service Status
systemctl status servicename
What this does:
- Shows detailed information about a service
- Displays current state (active/inactive, running/stopped)
- Shows recent log entries
- Indicates if the service is enabled to start at boot
Common service names:
sshd- SSH servernginx- Nginx web serverhttpdorapache2- Apache web serverlibvirtd- Virtualization daemonNetworkManager- Network managementdocker- Docker daemonpostgresql- PostgreSQL database
Example:
systemctl status sshd
What you'll see:
- Service state (active/inactive, running/stopped)
- Whether it's enabled at boot
- Recent log entries
- Process information
- Loaded configuration file path
Quick Status Checks
Check if a service is active (running):
systemctl is-active servicename
Output:
active- Service is runninginactive- Service is not running- Exit code 0 = active, non-zero = inactive
Check if a service is enabled (starts at boot):
systemctl is-enabled servicename
Output:
enabled- Service will start at bootdisabled- Service will not start at bootmasked- Service is prevented from starting (even manually)- Exit code 0 = enabled, non-zero = disabled/masked
Check if a service has failed:
systemctl is-failed servicename
Output:
failed- Service has failed- Empty output - Service is not in failed state
- Exit code 0 = not failed, non-zero = failed
:desktop: Listing Services
List All Running Services
systemctl list-units --type=service --state=running
What this does:
- Shows all services that are currently running
- Displays service name, load state, active state, and description
Example output:
UNIT LOAD ACTIVE SUB DESCRIPTION
abrt-journal-core.service loaded active running ABRT coredumpctl message creator
abrt-oops.service loaded active running ABRT kernel log watcher
abrt-xorg.service loaded active running ABRT Xorg log watcher
abrtd.service loaded active running ABRT Daemon
What each column means:
- UNIT: Service name
- LOAD: Whether the unit file is loaded
- ACTIVE: High-level activation state
- SUB: Low-level activation state (running, dead, etc.)
- DESCRIPTION: What the service does
List All Failed Services
systemctl list-units --type=service --state=failed
What this does:
- Shows only services that have failed
- Critical for troubleshooting system issues
- Helps identify what's broken
When to use it:
- System is behaving strangely
- Something isn't working
- After system updates
- Troubleshooting boot issues
If no failed services:
UNIT LOAD ACTIVE SUB DESCRIPTION
0 loaded units listed.
List All Services (All States)
systemctl list-units --type=service --all
What this does:
- Shows all services regardless of state
- Includes inactive, failed, and running services
- Useful for finding a specific service
List Enabled Services (Start at Boot)
systemctl list-unit-files --type=service --state=enabled
What this does:
- Shows all services that are enabled to start at boot
- Displays the unit file state and preset state
Example output:
UNIT FILE STATE PRESET
abrt-journal-core.service enabled enabled
abrt-oops.service enabled enabled
abrt-vmcore.service enabled enabled
abrt-xorg.service enabled enabled
:wrench: Controlling Services
Starting a Service
sudo systemctl start servicename
What this does:
- Starts a service immediately
- Does not enable it to start at boot (use
enablefor that) - Requires root privileges (sudo)
Example:
sudo systemctl start sshd
When to use it:
- Service is stopped and you need it running
- Testing if a service works
- Temporarily starting a disabled service
Stopping a Service
sudo systemctl stop servicename
What this does:
- Stops a running service immediately
- Does not disable it from starting at boot
- Requires root privileges
Example:
sudo systemctl stop sshd
When to use it:
- Service is misbehaving and needs to be stopped
- Temporarily disabling a service
- Before making configuration changes
Restarting a Service
sudo systemctl restart servicename
What this does:
- Stops and then starts a service
- Applies configuration changes
- Useful after editing service configuration files
Example:
sudo systemctl restart nginx
When to use it:
- After changing service configuration
- Service is acting strange and needs a fresh start
- Applying updates
Reloading a Service
sudo systemctl reload servicename
What this does:
- Reloads service configuration without stopping it
- Service continues running during reload
- Not all services support reload (some require restart)
Example:
sudo systemctl reload nginx
When to use it:
- After configuration changes
- When you want to avoid service downtime
- Services that support reload (check with
systemctl status)
Try-Restart (Restart Only If Running)
sudo systemctl try-restart servicename
What this does:
- Restarts the service only if it's currently running
- If the service is stopped, it does nothing
- Safer than regular restart
When to use it:
- Restarting a service that may or may not be running
- Scripts where you're not sure of the service state
Reload-or-Restart
sudo systemctl reload-or-restart servicename
What this does:
- Tries to reload first (if supported)
- Falls back to restart if reload is not supported
- Best of both worlds
When to use it:
- Applying configuration changes
- When you're not sure if the service supports reload
:key: Enabling and Disabling Services
Enable Service to Start at Boot
sudo systemctl enable servicename
What this does:
- Configures the service to start automatically at boot
- Does not start the service immediately (use
startfor that) - Creates symlinks to enable the service
Example:
sudo systemctl enable sshd
When to use it:
- You want a service to start automatically on boot
- Setting up a new service
- After installing a service that should run at boot
Disable Service from Starting at Boot
sudo systemctl disable servicename
What this does:
- Prevents the service from starting at boot
- Does not stop the service if it's currently running
- Removes symlinks that enable the service
Example:
sudo systemctl disable sshd
When to use it:
- Service should not start automatically
- Reducing boot time
- Disabling unnecessary services
Enable and Start Together
sudo systemctl enable --now servicename
What this does:
- Enables the service to start at boot
- Starts the service immediately
- Combines
enableandstartin one command
Example:
sudo systemctl enable --now sshd
When to use it:
- Setting up a new service that should run now and at boot
- Quick way to enable and start in one step
Disable and Stop Together
sudo systemctl disable --now servicename
What this does:
- Disables the service from starting at boot
- Stops the service immediately
- Combines
disableandstopin one command
When to use it:
- Completely disabling a service
- Removing a service from the system
Masking a Service
sudo systemctl mask servicename
What this does:
- Prevents a service from being started (even manually)
- Creates a symlink to
/dev/null - Stronger than disable - service cannot be started at all
When to use it:
- Completely preventing a service from running
- When disable isn't enough
- Troubleshooting service conflicts
Unmasking:
sudo systemctl unmask servicename
:file_folder: Viewing Service Information
View Service Properties
systemctl show servicename
What this does:
- Shows all properties of a service
- Displays detailed configuration information
- Useful for troubleshooting
Example:
systemctl show sshd
What you'll see:
- Service ID and names
- Dependencies (Requires, Wants, After, Before)
- Documentation links
- Description
- Load state
- Active state
- And much more
View Service Dependencies
systemctl list-dependencies servicename
What this does:
- Shows what services this service depends on
- Displays the dependency tree
- Helps understand service relationships
Example:
systemctl list-dependencies sshd
What you'll see:
- Services that must be running before this service
- Services that this service wants (optional)
- Target units (like network.target, sysinit.target)
Reverse dependencies (what depends on this service):
systemctl list-dependencies --reverse servicename
View Service Configuration File
systemctl cat servicename
What this does:
- Shows the service unit file content
- Displays the actual configuration
- Includes drop-in files (overrides)
When to use it:
- Understanding service configuration
- Checking what a service does
- Before making changes
View Service Logs
systemctl status servicename
Shows recent log entries in the status output.
For more detailed logs:
journalctl -u servicename
For real-time log following:
journalctl -u servicename -f
See the Journalctl Troubleshooting Guide for more log viewing options.
:alarm_clock: Working with Timers
List All Timers
systemctl list-timers
What this does:
- Shows all systemd timer units
- Displays when timers will run next
- Shows last run time
Example output:
NEXT LEFT LAST PASSED UNIT ACTIVATES
Thu 2025-11-13 16:04:14 AST 44min Thu 2025-11-13 12:08:03 AST 3h 11min ago dnf-makecache.timer dnf-makecache.service
Fri 2025-11-14 00:00:00 AST 8h Thu 2025-11-13 07:34:00 AST 7h ago unbound-anchor.timer unbound-anchor.service
What each column means:
- NEXT: When the timer will run next
- LEFT: Time until next run
- LAST: When it last ran
- PASSED: Time since last run
- UNIT: Timer unit name
- ACTIVATES: Service that the timer activates
When to use it:
- Checking scheduled tasks
- Understanding system maintenance schedules
- Troubleshooting automated tasks
:warning: Troubleshooting Failed Services
Check for Failed Services
systemctl --failed
What this does:
- Lists all failed services
- Quick way to see what's broken
- Equivalent to
list-units --state=failed
Reset Failed State
sudo systemctl reset-failed servicename
What this does:
- Clears the "failed" state from a service
- Useful after fixing a service issue
- Resets the service status
When to use it:
- After fixing a service configuration
- After resolving dependency issues
- Cleaning up service state
Reset all failed services:
sudo systemctl reset-failed
Reload systemd Configuration
sudo systemctl daemon-reload
What this does:
- Reloads systemd manager configuration
- Rereads all unit files
- Required after creating or modifying unit files
When to use it:
- After creating a new service file
- After modifying a service unit file
- After installing a new service
- Before starting a newly created service
Important: Always run this after editing service files in /etc/systemd/system/
:mag_right: Advanced Troubleshooting
View System State
systemctl status
Shows overall system health.
Check if system is degraded:
systemctl is-system-running
Possible outputs:
running- System is running normallydegraded- Some units failed but system is operationalmaintenance- System is in maintenance modeinitializing- System is still bootingstarting- System is starting upstopping- System is shutting down
View Service Dependencies Graph
systemctl list-dependencies servicename --all
What this does:
- Shows complete dependency tree
- Includes all dependencies recursively
- Shows both required and wanted dependencies
Check Service Logs with journalctl
journalctl -u servicename
Filter by priority:
journalctl -u servicename -p err
Follow logs in real-time:
journalctl -u servicename -f
Since a specific time:
journalctl -u servicename --since "1 hour ago"
See the Journalctl Troubleshooting Guide for comprehensive log viewing options.
:bulb: Common Troubleshooting Scenarios
Scenario 1: Service Won't Start
Problem: A service fails to start.
Solution:
-
Check service status:
systemctl status servicename -
Check for errors in logs:
journalctl -u servicename -p err -
Check service dependencies:
systemctl list-dependencies servicename -
Verify service file:
systemctl cat servicename -
Try starting with verbose output:
systemctl start servicename systemctl status servicename
Scenario 2: Service Keeps Crashing
Problem: Service starts but immediately stops.
Solution:
-
Check service status for error messages:
systemctl status servicename -
Follow logs in real-time:
journalctl -u servicename -f -
Check for configuration errors:
systemctl cat servicename -
Check system resources:
free -h df -h
Scenario 3: Service Not Starting at Boot
Problem: Service works when started manually but doesn't start at boot.
Solution:
-
Check if service is enabled:
systemctl is-enabled servicename -
If disabled, enable it:
sudo systemctl enable servicename -
Check service dependencies:
systemctl list-dependencies servicename -
Verify service file for After/Before directives:
systemctl cat servicename
Scenario 4: Finding What's Using a Port
Problem: Port is already in use, preventing service from starting.
Solution:
-
Find what's using the port (example for port 80):
sudo ss -tlnp | grep :80 -
Check the service status:
systemctl status servicename -
Stop the conflicting service if needed:
sudo systemctl stop conflicting-service
Scenario 5: Service Takes Too Long to Start
Problem: Service is slow to start.
Solution:
-
Check service status and startup time:
systemctl status servicename -
Check dependencies:
systemctl list-dependencies servicename -
Check system load:
uptime systemctl status -
Review logs for slow operations:
journalctl -u servicename --since "boot"
Scenario 6: Understanding Service Dependencies
Problem: Need to understand why a service requires another service.
Solution:
-
View dependencies:
systemctl list-dependencies servicename -
View reverse dependencies:
systemctl list-dependencies --reverse servicename -
View service properties:
systemctl show servicename | grep -i depend
:floppy_disk: Service File Locations
System service files:
/etc/systemd/system/- Local system service files (highest priority)/run/systemd/system/- Runtime service files/usr/lib/systemd/system/- Package-installed service files (lowest priority)
User service files:
~/.config/systemd/user/- User service files
Drop-in files (overrides):
/etc/systemd/system/servicename.service.d/- Service-specific overrides/etc/systemd/system/servicename.service.d/override.conf- Common override file
After editing service files: Always run:
sudo systemctl daemon-reload
:keyboard: Quick Reference
Status Commands
systemctl status # Overall system status
systemctl status servicename # Service status
systemctl is-active servicename # Quick active check
systemctl is-enabled servicename # Quick enabled check
systemctl is-failed servicename # Quick failed check
systemctl --failed # List failed services
Control Commands
sudo systemctl start servicename # Start service
sudo systemctl stop servicename # Stop service
sudo systemctl restart servicename # Restart service
sudo systemctl reload servicename # Reload service
sudo systemctl reload-or-restart servicename # Reload or restart
Enable/Disable Commands
sudo systemctl enable servicename # Enable at boot
sudo systemctl disable servicename # Disable at boot
sudo systemctl enable --now servicename # Enable and start
sudo systemctl disable --now servicename # Disable and stop
sudo systemctl mask servicename # Mask service
sudo systemctl unmask servicename # Unmask service
Listing Commands
systemctl list-units --type=service # All services
systemctl list-units --type=service --state=running # Running services
systemctl list-units --type=service --state=failed # Failed services
systemctl list-unit-files --type=service --state=enabled # Enabled services
systemctl list-timers # All timers
Information Commands
systemctl show servicename # Service properties
systemctl list-dependencies servicename # Service dependencies
systemctl cat servicename # Service file content
journalctl -u servicename # Service logs
Maintenance Commands
sudo systemctl daemon-reload # Reload systemd config
sudo systemctl reset-failed servicename # Reset failed state
sudo systemctl reset-failed # Reset all failed
Summary
This guide covered:
- Basic Commands:
- Getting help with
--help - Checking overall system status
- Viewing service status
- Status Checking:
status- Detailed service statusis-active- Quick active checkis-enabled- Quick enabled checkis-failed- Quick failed check
- Listing Services:
- Running services
- Failed services
- Enabled services
- All services
- Controlling Services:
- Start, stop, restart, reload
- Try-restart and reload-or-restart
- Enable and disable at boot
- Mask and unmask services
- Service Information:
- Viewing service properties
- Checking dependencies
- Viewing configuration files
- Viewing service logs
- Timers:
- Listing scheduled tasks
- Understanding timer output
- Troubleshooting:
- Finding failed services
- Resetting failed state
- Reloading systemd configuration
- Common troubleshooting scenarios
- Service File Locations:
- Where service files are stored
- Drop-in files and overrides
Next Steps:
- Practice with services on your system
- Combine with
journalctlfor comprehensive troubleshooting - Learn about creating custom service files
- Explore timer units for scheduling tasks
For more advanced topics, see the Journalctl Troubleshooting Guide for detailed log analysis.