[ Lab 2.1 ] Deploy and Validate Systemd Services - smitja21/group-a-oe2 GitHub Wiki

[!NOTE] Ticket #11: Deploy and Validate Custom Systemd Services
Ticket #16: Deploy and Validate Custom Systemd Services

Task 1: Observe the Process Tree (pstree -p)

What is PID 1? List three services you can see in the tree that are direct or indirect children of systemd. What does this tell you about how Linux manages running processes?

PID 1, is process id 1. ModemManager, agetty and chronyd is a direct child. systemd is the first process to be started by the linux kernel, seen by the process is set to 1. All running processes are given ids and children from systemd.

Task 2: Explore Existing Units

  • How many service unit files are installed on your server?

    • 247 out of 405 files are service unit files.

  • Look at ssh.service. What does the After= directive do? What unit does ssh depend on?

    • The service cannot execute until the units in After= are reached. For example, ssh.service cannot execute until after network.target is reached.

  • What is the difference between enabled, disabled, and static in the unit file listing?

Task A: Your First Service — Hello World

Testing output of script

(a) Loaded — what does this line tell you? Where the service is located/that it exists and was loaded. If it's enabled or disabled on start up. The preset tells what the status should be by default.

(b) Active — what state is the service in and since when? Active since Mon 2026-03-02 20:06:21 UTC

(c) Main PID — what is the process ID of hello.sh? 51581

(d) The log lines at the bottom — what do they show? That the hello word script is running if it is logging the systemd hello world message.

4.4 Step 4: Explore the Logs

(a) What does the SyslogIdentifier= directive in your unit file do? Try changing it to MyFirstService, reloading, and restarting. What changes in the journal output?

The SyslogIdentifier is the tag/prefix used for logs sent to syslog, journal or kmsg. Before

Can see that the prefix has changed in the log

(b) Where is the output of echo in hello.sh being sent? Why does it appear in the journal even though you did not configure any log file?

Because echo outputs to the standard output which systemd‑journald is capturing.

4.5 Step 5: Control the Service

(a) What is the difference between systemctl start and systemctl enable? Can a service be started but not enabled? Can it be enabled but not currently started? Give an example of when each situation would occur in practice.

Starting the service just starts the service but once the machine is restarted it won't automatically restart. While if it's enabled it will start automatically. It can be enabled if it not currently started. You would want to enable and start if you would like it to run after the next restart. Where if you weren't worried if it doesn't need to start itself on boot you don't need to enable it.

(b) After running systemctl enable hello.service, check what changed on the filesystem: What did systemd create, and why?

It created a symbolic link /etc/systemd/system/hello.service, so the system automatically the service up.

Task B: Creating a Production-Style Service

Setting up a monitoring agent that checks disk usage every 30 seconds and writes a structured log entry

  1. Write the monitoring script

Expected output:

  1. Create a Dedicated Service User
  1. Create environment file
  1. Write production unit file

An error Mar 02 22:51:56 backup-a systemd[1]: /etc/systemd/system/diskmon.service:3: Failed to resolve unit specifiers in '[https://github.com/smitja21/group-a-oe2/wiki/%5B-Lab-2.1-%5D-Systemd'](https://github.com/smitja21/group-a-oe2/wiki/%5B-Lab-2.1-%5D-Systemd%27), ignoring: I> was caused by the link to this Github documentation page and the % symbol.

  1. Enable, Start and Verify the Service

Verify it working:

  1. Test the restart policy

find pid

(a) How long did it take for systemd to restart the service after the kill?

Took around 10 seconds

(b) What would happen if you changed Restart=on-failure to Restart=always? When would each be the better choice?

on-failure: only restarts if he service exits with a non zero status or the user kills the process. Better for services that are expected to stop, ie. nightly backup.

always: restarts the service regardless of the exit code. Better for things that should always be running like a web server.

(c) What is the purpose of RestartSec=10s? Why not set it to 0?

Defines how long it waits before attempting to restart after a crash, if the service was failing over and over it would be restarting every second if no restartsec was set. This could flood the CPU and logs.

Task D aka Task C: Mastering journalctl

  1. Show all logs from diskmon since it started sudo journalctl -u diskmon.service --since "30 minutes ago"

Shows me disk usage from the last 30mins

  1. Show only WARNING entries from any service in the last boot sudo journalctl -b -p warning -e

Found that had an issue with formatting in one of the services, which we then fixed.

  1. Count how many error-level entries occurred in the last hour sudo journalctl --since "1 hour ago" -p err --no-pager | wc -l
  1. Show kernel messages from the current boot (useful for hardware issues) sudo journalctl -b -k -e
  1. View journal entries in JSON format for a single diskmon entry sudo journalctl -u diskmon.service -n 1 -o json-pretty
  1. Show all boots recorded in the journal sudo journalctl --list-boots
  1. Check how much disk space the journal is using sudo journalctl --disk-usage

(a) How many error-level entries occurred in the last hour on your server? What services generated them? 12, issues in the diskmon service, caused it.

(b) Examine the JSON output for one diskmon entry. List five metadata fields besides the message text that the journal captures automatically.

_HOSTNAME, _BOOT_ID, _EXE, _CMDLINE, PRIORITY

(c) How many boots does your server’s journal have recorded? How much disk space is the journal using?

Only 1 boot and the journal Has taken up 54.8M

Teardown

Step 1 — Stop and disable the diskmon service

sudo systemctl disable --now diskmon.service

sudo systemctl status diskmon.service

Step 2 — Remove the unit files

sudo rm -f /etc/systemd/system/diskmon.service sudo rm -f /etc/systemd/system/hello.service

Step 3 — Remove the boot symlinks

ls /etc/systemd/system/multi-user.target.wants/ | grep -E "diskmon|hello"

Step 4 — Remove scripts, directories, and config files

sudo rm -rf /opt/diskmon sudo rm -f /etc/diskmon.env sudo rm -f ~/hello.sh

Step 5 — Remove the service user

sudo userdel --remove diskmon

id diskmon

Step 6 — Final verification

No diskmon or hello units should appear systemctl list-unit-files | grep -E "diskmon|hello"

No diskmon processes should be running ps aux | grep diskmon

Confirm the service user no longer exists grep diskmon /etc/passwd