How to Install CKAN 2.9 on CentOS 7 - ckan/ckan GitHub Wiki

1. Install the required packages

  • Set up repositories Postgresql available on native centos repository is 9.2, and will not work with latest ckan

vi /etc/yum.repos.d/CentOS-Base.repo

Find the [base] and [updates] sections, enter insert mode by pressing i, and insert the exclude=postgresql* line in both sections. As a result, your file will look like the following, with new lines:

...
[base]
name=CentOS-$releasever - Base
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

exclude=postgresql*

#released updates
[updates]
name=CentOS-$releasever - Updates
mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates&infra=$infra
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
exclude=postgresql*
...

The PostgreSQL repository includes information for all available PostgreSQL releases. You can see all available packages and versions using the following command:

yum list postgresql*

Choose and install the desired version of PostgreSQL. In this guide, you will use the PostgreSQL 9.6 release.

yum install -y epel-release
yum update -y
shutdown -r now
  • Install packages
yum install -y wget policycoreutils-python python3-devel libpq-dev python3-pip python3-venv git-core java-1.8.0-openjdk maven lsof redis gcc gcc gcc-c++ cmake automake  gmp-devel boost
  • Install Official Postgresql repository
yum install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
  • Install global python packages
pip3 install psycopg2-binary
pip3 install pylons
pip3 install uwsgi
pip3 install supervisor
pip3 install PasteScript
pip3 install PasteDeploy

2. Setup and Configure a PostgreSQL

To install the PostgreSQL server and required packages use the following command:

yum install -y postgresql96 postgresql96-server postgresql96-contrib postgresql96-libs -y

Enable and start PostgreSQL service

systemctl enable postgresql-96.service
systemctl start postgresql-96.service
systemctl status postgresql-96.service

The installation procedure created a user account called postgres that is associated with the default Postgres role. In order to use Postgres, you can log in to that account.

Switch over to the postgres account on your server by typing:

sudo -i -u postgres

Check that PostgreSQL was installed correctly by listing the existing databases:

psql -l

Create a new PostgreSQL user called ckan_default, and enter a password for the user when prompted. You’ll need this password later:

createuser -S -D -R -P ckan_default

Create a new PostgreSQL database, called ckan_default, owned by the database user you just created:

createdb -O ckan_default ckan_default -E utf-8

3. Install CKAN

First, create a CKAN User. The ckan user is created with a shell of /sbin/nologin and a home directory of /usr/lib/ckan to mirror what is shown in the CKAN Deployment documentation.

useradd -m -s /sbin/nologin -d /usr/lib/ckan -c "CKAN User" ckan

Open the newly created directory up for read access so that the content will eventually be able to be served out via httpd.

chmod 755 /usr/lib/ckan

Switch to the ckan user.

su -s /bin/bash - ckan
  • Install an isolated Python environment, called default, to host CKAN from.
mkdir -p /usr/lib/ckan/default
chown `whoami` /usr/lib/ckan/default
python3 -m venv /usr/lib/ckan/default
. /usr/lib/ckan/default/bin/activate

For more details consult the official documentation at https://docs.ckan.org/en/2.9/maintaining/installing/install-from-source.html

  • Install the recommended setuptools version and up-to-date pip:
pip3 install setuptools==44.1.0
pip3 install --upgrade pip
  • Install the CKAN source code into your virtualenv.
pip install -e 'git+https://github.com/ckan/[email protected]#egg=ckan[requirements]'
  • Create a directory to contain the site’s config files:
sudo mkdir -p /etc/ckan/default
sudo chown -R `whoami` /etc/ckan/
  • Create the CKAN config file:
ckan generate config /etc/ckan/default/ckan.ini

Edit the ckan.ini file in a text editor, changing the following options:

sqlalchemy.url

This should refer to the database and password that you created above.

sqlalchemy.url = postgresql://ckan_default:pass@localhost/ckan_default

site_id

Each CKAN site should have a unique site_id, for example:

ckan.site_id = default

site_url

Provide the site’s URL (used when putting links to the site into the FileStore, notification emails etc). For example:

ckan.site_url = http://demo.ckan.org

solr_url

Point to your Solr server, for example:

solr_url = http://127.0.0.1:8983/solr/ckan

Deactivate and reactivate your virtualenv, to make sure you’re using the virtualenv’s copies of commands like ckan rather than any system-wide installed copies:

deactivate
. /usr/lib/ckan/default/bin/activate

4. Setup Apache SOLR

This part is a resume of "Install and use Solr 6.5 with CKAN", you can check there if need a deep explanation about this step.

The latest version compatible with CKAN that do not need extra modification on ckan side is the version 6.5.1, you can go to the Apache Solr website and download Solr, in this tutorial we are going to use the .tgz format, but it works with .zip as well.

Download the package from the web and put it somewhere in your file-system, we are going to use the installer soon

cd /opt
wget https://archive.apache.org/dist/lucene/solr/6.5.1/solr-6.5.1.tgz

When the download is finished, unzip the package's install script:

tar xzf solr-6.5.1.tgz solr-6.5.1/bin/install_solr_service.sh --strip-components=2

Run the install script with the default settings:

bash ./install_solr_service.sh solr-6.5.1.tgz

Enable and check for solar status

systemctl enable solr
systemctl status solr -l

Create and configure the ckan core

Switch to the solr user and go to the bin directory and create the ckan core :

sudo su solr
cd /opt/solr/bin
./solr create -c ckan

If this command fails because "Failed to determine the port of a local Solr instance, cannot create ckan!" then it's because the main 'solr' service has errors. Check logs for errors: /var/solr/logs/solr.log and /var/solr/logs/solr-8983-console.log

When successful it will have created all the configuration files and directories. At this point, we can see the core listed in our solr admin http://localhost:8983/solr/ and we can proceed to edit the configuration files

Now edit solrconfig.xml to make it compatible with SOLR6 syntax. Simply run these commands:

sed -i '/<config>/a <schemaFactory class="ClassicIndexSchemaFactory"/>' solrconfig.xml
sed -i '/<initParams path="\/update\/\*\*">/,/<\/initParams>/ s/.*/<!--&-->/' solrconfig.xml
sed -i '/<processor class="solr.AddSchemaFieldsUpdateProcessorFactory">/,/<\/processor>/ s/.*/<!--&-->/' solrconfig.xml

This will configure Solr to use Classic Index Schema Factory and disable /update processor.

Next, remove the managed-schema file and symlink the schema.xml from CKAN:

rm managed-schema
ln -s /usr/lib/ckan/default/src/ckan/ckan/config/solr/schema.xml schema.xml

Exit from being the solr user:,

exit

Finally, restart solr

systemctl restart solr

Check there are no errors when you browse: http://localhost:8983/solr/#/ckan

curl -s http://localhost:8983/solr/admin/cores?action=STATUS |     python -c 'import sys;import xml.dom.minidom;s=sys.stdin.read();print(xml.dom.minidom.parseString(s).toprettyxml())'

5. Create the Database Tables

Switch back to running as the ckan user, activate the isolated Python environment, and change to the CKAN source directory.

su -s /bin/bash - ckan
cd default/src/ckan
. /usr/lib/ckan/default/bin/activate

Now that you have a configuration file that has the correct settings for your database, you can create the database tables:

ckan -c /etc/ckan/default/ckan.ini db init

6. Set up the DataStore

Setting up the DataStore is optional. However, if you do skip this step, the DataStore features will not be available and the DataStore tests will fail.

7. You’re (almost) done!

You can now run CKAN from the command-line. This is a simple and lightweight way to serve CKAN that is useful for development and testing:

kan -c /etc/ckan/default/ckan.ini run --host 0.0.0.0

Open http://127.0.0.1:5000/ in a web browser, and you should see the CKAN front page.

This command will probably fail (https://github.com/ckan/ckan/issues/5235) , as it does for me due to some unsolved issues, so you can continue to the next step to set up a uWSGI service to run ckan.

8. Setup uWSGI service for CKAN

We already installed installed uwsgi above, if you do not did, you can install it using pip

Copy the default uwsgi from repository to ckan configs

cp /usr/lib/ckan/default/src/ckan/wsgi.py /etc/ckan/default/
cp /usr/lib/ckan/default/src/ckan/ckan-uwsgi.ini /etc/ckan/default/

Edit the ckan-uwsgi.ini:

vi /etc/ckan/default/ckan-uwsgi.ini

I changed some properties from default, such as uid and guid to ckan instead data-www

[uwsgi]

http            =  0.0.0.0:8080
uid             =  ckan
guid            =  ckan
wsgi-file       =  /etc/ckan/default/wsgi.py
virtualenv      =  /usr/lib/ckan/default
module          =  wsgi:application
master          =  true
pidfile         =  /tmp/%n.pid
harakiri        =  50
max-requests    =  5000
vacuum          =  true
callable        =  application
buffer-size     =  32768
vi /etc/systemd/system/ckan.service

Paste and edit(if needed)

[Unit]
Description=CKAN 2.9
After=syslog.target

[Service]
ExecStart=/usr/lib/ckan/default/bin/uwsgi --ini /etc/ckan/default/ckan-uwsgi.ini
# Requires systemd version 211 or newer
RuntimeDirectory=ckan
Restart=always
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all

[Install]
WantedBy=multi-user.target
systemctl enable ckan
systemctl start ckan
systemctl status ckan -l

to check logs:

journalctl -u ckan.service #-f to tail
⚠️ **GitHub.com Fallback** ⚠️