Setup Katello with remote databases - theforeman/foreman_maintain GitHub Wiki

Table of content

High level
Prepare remote Postgres
Prepare remote Mongo
Fresh install in detail
Migration of existing Katello in details
Full list of options

Setup Katello with remote databases

High level

There are two ways to deplay Katello/Foreman/Satellite with remote databases:

Fresh install
- prepare Postgres server with databases for Foreman and Candlepin and dedicated users owning them
- prepare Mongo DB with user owning the pulp_database
- prepare box where the Katello will be installed and make sure the databases are accessible from the box
- run foreman-installer with right parameters pointing to the databases
Migration of exiting DB to remote instances
- prepare Postgres server with databases for Foreman a Candlepin and dedicated users owning them
- prepare Mongo DB with user owning the pulp_databse
- make sure the databases are accessible from the box where Katello is installed
- shut down the services except the dbs you want to move (mongod, postgresql)
- dump the DBs
- restore the DBs on remote servers
- run foreman-installer with right parameters pointing to the databases. It re-configures the databases and start all the services with new DB locations

Prepare remote Postgres

GOAL: To use remote Postgres database with Katello we have to:

be able to access the databases from katello box
the database user we use to connect to the database needs to own the database, i.e. it can create, alter and delete the tables, indexes and constraints. Note it is not required to be able to create the database itself.

Install Postgres

Warning: This is just minimal testing setup which is not suitable for production.

Assume our postgres server has hostname postgres.example.com.

  yum -y localinstall https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
  yum install postgresql-server postgresql-contrib
  postgresql-setup initdb
  systemctl start postgresql
  systemctl enable postgresql

Now we need to make Postgres listen to inbound connections, please adjust these parameters to your own networking and security requirements.

Edit /var/lib/pgsql/data/postgresql.conf. Uncomment listen_address and modify its value to look like:

  listen_address = "*"

The next step we need to take is to add a proper client authentication for remote client to our postgres server. To achieve the same, edit /var/lib/pgsql/data/pg_hba.conf. Append the following line at the end of the file

  host          all          all          <katello.example.com ip>/24              md5

Now restart the postgres service for changes to take effect

  systemctl restart postgresql

Create the databases

Switch the user role to postgres and start postgres client

  su - postgres -c psql

Once inside the client, we need to create two databases and dedicated roles, one for foreman and one for candlepin

  CREATE USER "foreman" WITH PASSWORD '<FOREMAN_PASSWORD>';
  CREATE USER "candlepin" WITH PASSWORD '<CANDLEPIN_PASSWORD>';
  CREATE DATABASE foreman OWNER foreman;
  CREATE DATABASE candlepin OWNER candlepin;

Test it works

From katello.example.com test the DB is accessible:

PGPASSWORD='<FOREMAN_PASSWORD>' psql -h postgres.example.com  -p 5432 -U foreman -d foreman -c "SELECT 1 as ping"

PGPASSWORD='<CANDLEPIN_PASSWORD>' psql -h postgres.example.com  -p 5432 -U candlepin -d candlepin -c "SELECT 1 as ping"

If there are no errors we are done with database preparation.

Prepare remote Mongo

GOAL: To use remote Mongo database with Katello we have to:

be able to access the databases from katello box
the database user we use to connect to the database needs to own the database

Install Mongo DB

Warning: This is just minimal testing setup which is not suitable for production.

Assume our Mongo server has hostname mongo.example.com. Install and enable Mongo server

  yum -y localinstall https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
  yum install -y mongodb mongodb-server

Enable authentication in /etc/mongod.conf

auth=true

Enable and start the service

  systemctl start mongod
  systemctl enable mongod

Create Pulp user and database

  mongo admin -u admin -p admin --eval "db.createUser({user:'pulp',pwd:'<PULP_PASSWORD>',roles:[{role:'dbOwner', db:'pulp_database'},{ role: 'readWrite', db: 'pulp_database'}]})"

Test it works

From katello.example.com test the mongo DB is accessible:

  mongo --host mongo.example.com -u pulp -p <PULP_PASSWORD> --port 27017 --eval 'ping:1' pulp_database

If there are no errors we are done with database preparation.

Fresh install in detail

Install katello package

We assume the box where the Katello server will be installed has hostname katello.example.com.

Follow the documentation to install the katello package and do not run the foreman-installer yet.

For Centos the steps could look like this

  yum -y localinstall https://fedorapeople.org/groups/katello/releases/yum/3.5/katello/el7/x86_64/katello-repos-latest.rpm
  yum -y localinstall https://yum.theforeman.org/releases/1.16/el7/x86_64/foreman-release.rpm
  yum -y localinstall https://yum.puppetlabs.com/puppetlabs-release-pc1-el-7.noarch.rpm
  yum -y localinstall https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
  yum -y install foreman-release-scl python-django
  yum -y update
  yum -y install katello

Prepare remote databases

Follow the instructions in Prepare remote Mongo and Prepare remote Postgres to make the remote database servers ready for installation.

Run the installer

To install and configure Katello we just need to run

  foreman-installer --scenario katello \
    --foreman-db-host postgres.example.com \ 
    --foreman-db-password <FOREMAN_PASSWORD> \
    --foreman-db-database foreman \
    --katello-candlepin-db-host postgres.example.com \
    --katello-candlepin-db-name candlepin \
    --katello-candlepin-db-password <CANDLEPN_PASSWORD> \
    --katello-candlepin-manage-db false \
    --katello-pulp-db-username pulp \ 
    --katello-pulp-db-password <PULP_PASSWORD> \ 
    --katello-pulp-db-seeds “mongo.example.com:27017” \
    --katello-pulp-db-name pulp_database

Note: for more related options and tips on SSL configuration see Full list of options

Migration of existing Katello in details

We assume that Katello was installed and is running on katello.example.com.

Prepare remote databases

Follow the instructions in Prepare remote Mongo and Prepare remote Postgres to make the remote database servers ready for migration.

Stop the Katello server

Stop the Katello related services to minimize risk of the data changes during the migration

  katello-service stop
  systemctl start postgresql
  systemctl start mongod

Dump databases

Dump the local databases

  katello-backup --online --skip-pulp-content --preserve-directory -y /tmp/migration_backup

Restore data in remote databases

You can restore the SQL dumps to the remote databases from the katello system.

PGPASSWORD='<FOREMAN_PASSWORD>' pg_restore -h postgres.example.com -U foreman -d foreman < /tmp/migration_backup/foreman.dump
PGPASSWORD='<CANDLEPIN_PASSWORD>' pg_restore -h postgres.example.com -U candlepin -d candlepin < /tmp/migration_backup/candlepin.dump
mongorestore --host mongo.example.com --db pulp_database --username pulp --password <PULP_PASSWORD> /tmp/migration_backup/mongo_dump

Now the copy of the local database is also at the remote locations.

Update the configuration

To update existing configuration of Katello we just need to run

  foreman-installer --scenario katello \
    --foreman-db-host postgres.example.com \ 
    --foreman-db-password <FOREMAN_PASSWORD> \
    --foreman-db-database foreman \
    --foreman-db-manage false \
    --katello-candlepin-db-host postgres.example.com \
    --katello-candlepin-db-name candlepin \
    --katello-candlepin-db-password <CANDLEPN_PASSWORD> \
    --katello-candlepin-manage-db false \
    --katello-pulp-db-username pulp \ 
    --katello-pulp-db-password <PULP_PASSWORD> \ 
    --katello-pulp-db-seeds “mongo.example.com:27017” \
    --katello-pulp-db-name pulp_database \
    --katello-pulp-manage-db false

Installer also starts the services and everything should be up and ready at this point.

Full list of remote database related options in the installer

Foreman database related:

    --foreman-db-manage           if enabled, will install and configure the database server on this host
    --foreman-db-database         Database 'production' database (e.g. foreman)
    --foreman-db-host             Database 'production' host
    --foreman-db-password         Database 'production' password, default is randomly generated
    --foreman-db-pool             Database 'production' size of connection pool (current: 5)
    --foreman-db-port             Database 'production' port
    --foreman-db-root-cert        Root cert used to verify SSL connection to postgres
    --foreman-db-sslmode          Database 'production' ssl mode (disable|allow|prefer|require|verify-full)
    --foreman-db-username         Database 'production' user (e.g. foreman)

Candlepin database related:

    --katello-candlepin-db-host   Host with Candlepin DB
    --katello-candlepin-db-name   Name of the Candlepin DB
    --katello-candlepin-db-password  Candlepin DB password
    --katello-candlepin-db-port   Port accepting connections to Candlepin DB
    --katello-candlepin-db-ssl    Boolean indicating if the connection to the database should be over
    --katello-candlepin-db-ssl-verify  Boolean indicating if the SSL connection to the database should be verified
    --katello-candlepin-db-user   Candlepin DB user
    --katello-candlepin-manage-db  Boolean indicating whether a database should be installed, this includes db creation and user

Mongo database related:

    --katello-pulp-db-ca-path     The ca_certs file contains a set of concatenated "certification authority" certificates,
    --katello-pulp-db-name        Name of the database to use
    --katello-pulp-db-password    The password to use for authenticating to the MongoDB server
    --katello-pulp-db-replica-set  The name of replica set configured in MongoDB, if one is in use
    --katello-pulp-db-seeds       Comma-separated list of hostname:port of database replica seed hosts
    --katello-pulp-db-ssl         Whether to connect to the database server using SSL.
    --katello-pulp-db-ssl-certfile  The certificate file used to identify the local connection against mongod.)
    --katello-pulp-db-ssl-keyfile  A path to the private keyfile used to identify the local connection against mongod. If
    --katello-pulp-db-unsafe-autoretry  If true, retry commands to the database if there is a connection error.
    --katello-pulp-db-username    The user name to use for authenticating to the MongoDB server
    --katello-pulp-db-verify-ssl  Specifies whether a certificate is required from the other side of the connection, and
    --katello-pulp-db-write-concern  Write concern of 'majority' or 'all'. When 'all' is specified, 'w' is set to number of

The actual option names may vary between versions. Check the actual naming with foreman-installer --full-help.

SSL configuration

Here is sample installer command that sets up Postgres databases with SSL verification. The Postgres server has its own CA. The CA cert used by Candlepin needs to be stored in system trust (/etc/pki/java/cacerts) as there is no other way to pass it to Candlepin

  foreman-installer -S katello \
    --foreman-admin-password changeme \
    --foreman-db-host postgres.example.com \
    --foreman-db-password foreman \
    --foreman-db-database foreman_2 \
    --foreman-db-root-cert /etc/pki/ca-trust/source/anchors/ca-chain.cert.pem \
    --foreman-db-sslmode verify-full \
    --katello-candlepin-db-host postgres.example.com \
    --katello-candlepin-db-name candlepin_2 \
    --katello-candlepin-db-password candlepin \
    --katello-candlepin-db-ssl true \
    --katello-candlepin-manage-db false