HackWeek 20: Uyuni SUSE Manager containerization project - uyuni-project/uyuni GitHub Wiki

Logistics

• This is the right place to record progress and plans
• Rocket.Chat for discussions
• Official HackWeek page for announcements
• A concluding Lightning Talk Slot was booked for a presentation of results

Objectives

1. have a lot of fun!
2. learn about container building (buildah)
3. learn about container orchestration (podman with Docker Compose support, k3s)
4. learn about containerized application deployment (helm)
5. learn about roadblocks in delivering Uyuni as containers

Conventions

• all development happens on the Uyuni containers branch/PR
• all new files are to be added in the containers/ directory, for now
• we use Dockerfiles (built with docker build or buildah), locally for now
• we explicitly do not care about traditional clients at least for now

Current plans

1. create a "fat" container with everything needed for a Proxy

• :green_circle: start from https://github.com/SUSE/spacewalk/wiki/Proxy-container
• :green_circle: find out which directories need to be mounted externally
• :green_circle: find out how it is best to specify configuration parameters (environment variables? answer files?)
• :green_circle: pass the machine-id as parameter
• :green_circle: add a startup (Salt) script. Figure out registration to the Server, activation of Proxy functionality (configure-proxy.sh), certs
• :green_circle: add a Server-side Python script to prepare configuration to onboard a Proxy
• :green_circle: ensure the Proxy works (manual test)
• :green_circle: ensure the Proxy container can be killed and restarted. With the same mount points and parameters, it should come back to full functionality ("proto-HA")

2. try to slim down the Proxy container

• :green_circle: remove traditional stack processes/packages, if possible
• :green_circle: split out a first component (eg. salt-broker) into another container. In parallel:
  • :green_circle: try orchestration with Podman
  • :green_circle: try orchestration with k3s/Helm
• :white_circle: try to skip packaging for one of the packages (eg. salt-broker) - sources straight from git to image

3. create a "fat" container with everything needed for a Server

• :green_circle: start from https://gitlab.suse.de/mbologna/sumadocker/-/tree/saltcontainer
  • uyuni server is starting. Instructions on how to deploy in podman are in: https://github.com/uyuni-project/uyuni/tree/containers/containers/srv/uyuni-container
  • An initial attempt to deploy to a k3s cluster are in the file uyuni.yaml
• :white_circle: find out which directories need to be mounted externally. Starting point: HA paper
• :green_circle: add a startup (Python?) script
• :white_circle: test until it breaks!

4. try to slim down the Server container

• :white_circle: carve PostgreSQL out. Try Postgres-in-containers or outside of them
• :white_circle: disable Cobbler. What needs to be done in order to make Cobbler "optional"?
• :white_circle: disable or remove the traditional stack

5. other research

• :white_circle: we will need a solution about commandline tools. Would it be possible to create a UI around them like Rancher does?

What did we learn?

• Dockerfile syntax, best practices
• K8s init containers
• How to preseed a minion with an accepted key
• For permanent storage use volumes. They are managed in docker/podman and get automatically the content of the directory in the image where it is mounted to. You need to update the owner of the mountpoint. Permission are taken from the original dir, but the ownership not.
• UID/GID are not static! To prevent having different in different containers or changing them after a rebuild we need to force the numbering.

Important Proxy directories and files

• configuration files changed by configure-proxy.sh:

/etc/apache2/conf.d/cobbler-proxy.conf
/etc/apache2/vhosts.d/ssl.conf

/etc/jabberd/c2s.xml
/etc/jabberd/router-users.xml
/etc/jabberd/router.xml
/etc/jabberd/s2s.xml
/etc/jabberd/sm.xml

/etc/squid/squid.conf

/etc/ssh/sshd_config

/etc/sysconfig/rhn/up2date
/etc/rhn/rhn.conf

• configuration files related to the mgrsshtunnel user:

/etc/group
/etc/passwd
/etc/shadow
/var/spacewalk/mgrsshtunnel/

• Key material

/etc/salt/pki/minion/minion.pem
/etc/salt/pki/minion/minion.pub

/etc/ssh/ssh_host_*_key.*

/etc/apache2/ssl.crt/server.crt
/etc/apache2/ssl.csr/server.csr
/etc/apache2/ssl.key/server.key
/etc/pki/spacewalk/jabberd/server.pem
/etc/jabberd/server.pem

/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT

/var/lib/ca-certificates/*

/var/spacewalk/gpgdir

• Identifiers

/etc/salt/minion_id
/etc/machine-id
/etc/sysconfig/rhn/systemid
/var/lib/dbus/machine-id

• "permanent" directories

/srv/www/htdocs/pub/
/var/cache/squid/
/var/spool/rhn-proxy
/var/log/

Ongoing Questions

• what does configure-proxy.sh do?
  • it requests, via Salt events received by the fetch-certificate script, the /etc/sysconfig/rhn/systemid file. systemid is generated on the Server side by Java code and essentially contains the system ID (duh) and other basic system infos, with a signature (example). Should be only useful for traditional clients
  • performs "proxy activation" via XMLRPC calls (rhn-proxy-activate script)
  • installs the spacewalk-proxy-management package
  • configures jabberd
  • changes Squid config files
  • changes Cobbler config files
  • optionally generates SSL certs
  • configures SSL certs for Apache and Jabberd
  • configures (and optionally generates) an SSH key for salt-ssh proxying. Configures sshd via the mgr-proxy-ssh-push-init script
  • uninteresting parts: optionally setting up a configuration channel for Proxy configuration, opening firewall ports, activating SLP, enabling services (code)

Open Questions

• does it make sense to send traceback emails? Specifically from the Proxy?

Summary

A short summary can be found here