This guide expects the reader to know about basic Linux servers, Unix command-line tools and networking. This guide will be focused on covering the steps taken in a way that should make them reproducible, without going in depth on the reasoning behind the decisions taken. Supplementary documentation will be provided where deemed necessary.

---

To keep an standard system across our infrastructure we will aim to keep all our machines running a standard operating system distribution. For Linux we are going to use the latest Ubuntu LTS server. At the time of writing this is 24.04.

• Download the latest Ubuntu LTS Server ISO: https://ubuntu.com/download/server
• Flashing tool. We recommend balena-etcher: https://etcher.balena.io/#download-etcher. In linux and macos you can use dd directly but this guide will focus on using balena-etcher for simplicity.
• A target computer with a network connection and power source.
• A Github account with at least one ssh key.
• Time and patience.

1. Download the latest Ubuntu LTS Server ISO from the link provided above.
2. Flash the ISO to a USB drive using balena-etcher.
3. Boot the target computer from the USB drive. This will depend on the computer's BIOS/UEFI settings.
4. Follow the installation wizard. We recommend using the default settings for simplicity.
5. Set the installation language.
6. Set the keyboard layout.
7. Choose the Ubuntu Server option.
8. Set the network configuration. We recommend using DHCP.
9. You will not need a proxy configuration.
10. No need to set any alternative mirrors, defaults are good.
11. Set the disk partitioning. We recommend using the default settings.
    • The default partitioning will erase the entire disk and create a new partition table.
    • The installation will use LVM to configure the partitions.
    • Encryption is not needed so lets keep it disabled.
12. Set a username. Let's use manager for this guide. The full name can be Manager.
    • Set a simple password for now. We will change it later when we configure security.
    • Set a hostname. Let's use node1 for this guide.
    • Continue on to the next step.
13. This is a very important step. When prompted about installing OpenSSH server, select Yes.
    • Ensure that Allow password authentication is enabled. We will disable this later.
    • Select Import SSH Keys and select from Github.
    • Ensure that your usename has at least one ssh key in your Github account. Otherwise follow the steps here: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account
    • Input your Github username and click Done.
    • A list of keys will load. If they look correct, click Yes to import them.
    • Click Done to continue to the next step.
14. You will be asked about the snap store. We recommend skipping this step.
15. Now the installation will start. This will take some time.
16. Once the installation is complete, you will be prompted to restart the computer. Click Reboot Now.
17. Remove the USB drive and let the computer boot from the hard drive.

1. If you are using a laptop, start by disabling the lid close action. This will prevent the computer from suspending when the lid is closed. This is important for servers as they are usually headless and you will not be able to wake them up.
   • https://askubuntu.com/questions/15520/how-can-i-tell-ubuntu-to-do-nothing-when-i-close-my-laptop-lid
2. Retrieve important information:
   • The hostname: hostname
   • The IP address: ip a
3. Now connect to the server using ssh:

   ssh manager@<ip-address>

4. Update the system:

   sudo apt update && sudo apt upgrade -y

5. Change the password for the manager user:

    passwd

6. If you have been able to update the password and are currently connected to the server, you can disable password authentication:

   sudo nano /etc/ssh/sshd_config

   • Find the line PasswordAuthentication yes and change it to PasswordAuthentication no.
   • Save the file and restart the ssh service:

   sudo systemctl restart ssh

7. Enable automatic package updates. This will ensure that the server is always up to date with the latest security patches.

   sudo apt install unattended-upgrades
   sudo dpkg-reconfigure unattended-upgrades

8. Enable automatic updates for all packages. This will ensure that all packages are updated automatically.

   sudo nano /etc/apt/apt.conf.d/50unattended-upgrades

   • Uncomment the line //"${distro_id}:${distro_codename}-updates"; by removing the //.
   • Save the file.
   • Restart the unattended-upgrades service:

    sudo systemctl restart unattended-upgrades

   • Verify that the service is running correctly: sudo systemctl status unattended-upgrades
   • For more information: https://github.com/mvo5/unattended-upgrades
9. Create a private/public key pair in this host. This will allow hosts to connect to each other if needed. For instructions on how to do this, follow the steps here: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent.

Sometimes you will see that a service systemd-networkd-wait-online.service is not starting. This may happen if one of the interfaces is configured as non-optional but it is not used anymore. We encountered this issue a lot when a computer is installed by using one service but it runs on another one once configured.

Some resources can be found here:

• https://askubuntu.com/questions/1118283/50-cloud-init-yaml-optional-true-will-the-network-interface-get-initialized
• https://askubuntu.com/questions/1445221/permanently-disable-network-interface-in-ubuntu-22-04
• https://netplan.readthedocs.io/en/stable/netplan-yaml/#properties-for-device-type-ethernets

Another cause for this problem is that the system could be using NetworkManager instead of netplan. In that case disable NetworkManager by deleting the file /etc/netplan/01.netcfg.yaml.

1. To manage all the computers we will use Cockpit. This is a web-based interface that allows you to manage all the computers in your network.

   sudo apt install cockpit

2. Verify that the service is running:

   sudo systemctl status cockpit

   and then going to the url https://<HOST_IP>:9090/ in your browser.
3. You may get a warning about the certificate. This is normal as the certificate is self-signed. You can ignore this warning and proceed.
4. Sign in using the manager username and password.
5. You may see a warning about the Web console is running in limited access mode. Remediate it by following the instructions on the screen, you may need to input your password again.
6. Since we are going to be managing multiple computers we will have a machine works as the host of the web gui while the other machines will be clients.
7. Apply this fix: https://www.reddit.com/r/Actualfixes/comments/1cek3rg/fix_cockpit_cannot_refresh_cache_whilst_offline/

1. Run the following in a client to copy the public key from the cockpit host to this client:

   ssh-copy-id manager@<cockpit-host-ip>

1. Once the web console is running with Administrative access you can add the other machines as clients.
2. Click on the host switcher in the top left corner and then select Add new host.
3. Provide the hostname and not the ip address.
4. You will be asked to verify the fingerprint of the host. Click on Trust and add host.

1. Coolify needs to use the root account(this should be changed once Coolify supports this).
2. Ensure that every server that will be managed by Coolify has ssh access by using ssh keys. Follow the steps here: https://coolify.io/docs/knowledge-base/server/openssh
3. If you are setting up a server managed by Coolify, skip to step XXX.
4. If you are setting up the Coolify host continue with these steps.
5. Open the Coolify dashboard at http://:8000
6. Configure the Coolify host by following the steps here: https://coolify.io/docs/knowledge-base/server/coolify-host
7. To add new servers to be managed by Coolify, go to http://:8000/servers/ and select Add +.
8. Provide the requested information and continue.
9. Coolify will want to verify the server. This can be done by clicking on the Verify button. All the packages should be installed automatically.

1. Go to the notifications tab in the Coolify dashboard.
2. Go to the Discord tab
3. Follow the instructions to add a new webhook to your Discord server.

1. Sign in to Cloudflare and go to Zero Trust.
2. Go to Networks and then Tunnels.
3. Click on Create a tunnel and select Cloudflared.
4. Name your tunnel something that makes it clear to which server it is connected. Then select Save Tunnel.
5. Choose your right OS and architecture and follow the instructions to download and install the Cloudflared binary.
6. The Coolify instructions for Cloudflare mentions how to configure it for accessing server. We will NOT use this, make sure that the server in Coolify is not configured to use Cloudflare.
7. To use Cloudflared for an application, follow the instructions for the One domain -> One resource configuration.

1. Create a tunnel to the coolify host. You can use a separate subdomain(for example coolify) to ensure that there is a convenient URL to connect.
2. The service would be http and the URL would be localhost:8000
3. Save the changes. Now Coolify will be accessible from https://coolify..com/
4. Now sign in and go to Settings, General and set Instance's Domain to the https URL from above.