CKAN 2.9 Deploying a source install - New Web Server options

Once you've installed CKAN from source by following the instructions in you can follow these instructions to deploy your CKAN site using a rudimentary web server, so that it's available to the Internet.

Because CKAN uses WSGI, a standard interface between web servers and Python web applications, CKAN can be used with a number of different web server and deployment configurations including:

NGINX with uwsgi
NGINX with gunicorn
NGINX with the modwsgi Apache module

This guide explains how to deploy CKAN using a WSGI web server and proxied with NGINX on an Ubuntu server. These instructions have been tested on Ubuntu 18.04.

The preferred option is NGINX with uwsgi

1. Install Nginx

Install NGINX (a web server) which will proxy the content from one of the WSGI Servers and add a layer of caching

sudo apt-get install nginx

2. Create the WSGI script file

Create your site's WSGI script file wsgi.py with the following contents:

import os
from ckan.config.middleware import make_app
from ckan.cli import CKANConfigLoader
from logging.config import fileConfig as loggingFileConfig
config_filepath = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'ckan.ini')
abspath = os.path.join(os.path.dirname(os.path.abspath(__file__)))
loggingFileConfig(config_filepath)
config = CKANConfigLoader(config_filepath).get_config()
application = make_app(config)

The WSGI Server (configured next) will redirect requests to this WSGI script file. The script file then handles those requests by directing them on to your CKAN instance (after first configuring the Python environment for CKAN to run in).

3. Create the WSGI Server

There is a choice of 3 different WSGI Servers. We recommend the first option (uwsgi) Make sure you have activated the Python virtual environment before running these commands

a. uwsgi

Run pip install uwsgi Create the uwsgi configuration file /etc/ckan/default/ckan-uwsgi.ini

[uwsgi]
http            =  127.0.0.1:8080
uid             =  www-data
guid            =  www-data
wsgi-file       =  /etc/ckan/default/wsgi.py
virtualenv      =  /usr/lib/ckan/default
module          =  wsgi:application
master          =  true
pidfile         =  /tmp/%n.pid
harakiri        =  50
max-requests    =  5000
vacuum          =  true
callable        =  application

Run: /usr/lib/ckan/default/bin/uwsgi -i /etc/ckan/default/ckan-uwsgi.ini

b. gunicorn

Run pip install gunicorn Create the gunicorn configuration file /etc/ckan/default/ckan-gunicorn.ini

pidfile = '/tmp/gunicorn.pid'
chdir = '/etc/ckan/default'
errorlog = '/etc/ckan/default/gunicorn.ERR'
accesslog = '/etc/ckan/default/gunicorn.OUT'
loglevel = 'debug'
bind = '127.0.0.1:8080'
daemon = True
workers = 4
worker_class = 'sync'
threads = 2
user = 'www-data'
group = 'www-data'

Run: /usr/lib/ckan/default/bin/gunicorn -i /etc/ckan/default/ckan-gunicorn.ini

c. Apache

Install Apache (a web server), modsgi (an Apache module that adds WSGIsupport to Apache), and modrpaf (an Apache module that sets the right IP address when there is a proxy forwarding to Apache)

sudo apt-get install apache2 libapache2-mod-wsgi-py3 libapache2-mod-rpaf

Note: For python 2 change this to: sudo apt-get install apache2 libapache2-mod-wsgi libapache2-mod-rpaf

Create your site's Apache config file at /etc/apache2/sites-available/ckan_default.conf, with the following contents:

<VirtualHost 127.0.0.1:8080>
    ServerName default.ckanhosted.com
    ServerAlias www.default.ckanhosted.com
    WSGIScriptAlias / /etc/ckan/default/apache.wsgi

    # Pass authorization info on (needed for rest api).
    WSGIPassAuthorization On

    # Deploy as a daemon (avoids conflicts between CKAN instances).
    WSGIDaemonProcess ckan_default display-name=ckan_default processes=2 threads=10 python-home=/usr/lib/ckan/default

    WSGIProcessGroup ckan_default

    ErrorLog /var/log/apache2/ckan_default.error.log
    CustomLog /var/log/apache2/ckan_default.custom.log combined

    <IfModule mod_rpaf.c>
        RPAFenable On
        RPAFsethostname On
        RPAFproxy_ips 127.0.0.1
    </IfModule>

    <Directory />
        Require all granted
    </Directory>

</VirtualHost>

Replace default.ckanhosted.com and www.default.ckanhosted.com with the domain name for your site.

This tells the Apache modwsgi module to redirect any requests to the web server to the WSGI script that you created above. Your WSGI script in turn directs the requests to your CKAN instance.

Open /etc/apache2/ports.conf. We need to replace the default port 80 with the 8080 one.

On Apache 2.4 (eg Ubuntu 18.04 or RHEL 7):

Replace this line: Listen 80 with this one: Listen 8080

Finally create a softlink for Apache to the WSGI script ln -s /etc/ckan/default/wsgi.py /etc/ckan/default/apache.wsgi

To prevent conflicts, disable your default nginx and apache sites. Finally, enable your CKAN site in Apache:

sudo a2ensite ckan_default
sudo a2dissite 000-default
sudo rm -vi /etc/nginx/sites-enabled/default
sudo ln -s /etc/nginx/sites-available/ckan /etc/nginx/sites-enabled/ckan_default
sudo service apache reload
sudo service nginx restart

4. Install Supervisor for the uwsgi or gunicorn servers

Install Supervisor (a Process Control System) used to control starting, stopping the uwsgi or gunicorn servers::

sudo apt-get install supervisor sudo service supervisor restart

a. uwsgi

Create the /etc/supervisor/conf.d/ckan-uwsgi.conf file

[program:ckan-uwsgi]

command=/usr/lib/ckan/default/bin/uwsgi -i /etc/ckan/default/ckan-uwsgi.ini

; Start just a single worker. Increase this number if you have many or
; particularly long running background jobs.
numprocs=1
process_name=%(program_name)s-%(process_num)02d

; Log files - change this to point to the existing CKAN log files
stdout_logfile=/etc/ckan/default/uwsgi.OUT
stderr_logfile=/etc/ckan/default/uwsgi.ERR

; Make sure that the worker is started on system start and automatically
; restarted if it crashes unexpectedly.
autostart=true
autorestart=true

; Number of seconds the process has to run before it is considered to have
; started successfully.
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

; Required for uWSGI as it does not obey SIGTERM.
stopsignal=QUIT

b. gunicorn

Create the /etc/supervisor/conf.d/ckan-uwsgi.conf file

[program:ckan-uwsgi]

command=/usr/lib/ckan/default/bin/uwsgi -i /etc/ckan/default/ckan-uwsgi.ini

; Start just a single worker. Increase this number if you have many or
; particularly long running background jobs.
numprocs=1
process_name=%(program_name)s-%(process_num)02d

; Log files - change this to point to the existing CKAN log files
stdout_logfile=/etc/ckan/default/uwsgi.OUT
stderr_logfile=/etc/ckan/default/uwsgi.ERR

; Make sure that the worker is started on system start and automatically
; restarted if it crashes unexpectedly.
autostart=true
autorestart=true

; Number of seconds the process has to run before it is considered to have
; started successfully.
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600

; Required for uWSGI as it does not obey SIGTERM.
stopsignal=QUIT

5. Create the Nginx config file

Create your site's NGINX config file at /etc/nginx/sites-available/ckan.conf, with the following contents:

proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=cache:30m max_size=250m;
proxy_temp_path /tmp/nginx_proxy 1 2;

server {
    client_max_body_size 100M;
    location / {
        proxy_pass http://127.0.0.1:8080/;
        proxy_set_header X-Forwarded-For $remote_addr;
        proxy_set_header Host $host;
        proxy_cache cache;
        proxy_cache_bypass $cookie_auth_tkt;
        proxy_no_cache $cookie_auth_tkt;
        proxy_cache_valid 30m;
        proxy_cache_key $host$scheme$proxy_host$request_uri;
        # In emergency comment out line to force caching
        # proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
    }

}

Now restart the nginx server: sudo systemctl restart nginx

6. Access your CKAN site

You should now be able to visit your server in a web browser and see your new CKAN instance.

CKAN 2.9 New Web Server options - ckan/ckan GitHub Wiki

CKAN 2.9 Deploying a source install - New Web Server options

1. Install Nginx

2. Create the WSGI script file

3. Create the WSGI Server

a. uwsgi

b. gunicorn

c. Apache

4. Install Supervisor for the uwsgi or gunicorn servers

a. uwsgi

b. gunicorn

5. Create the Nginx config file

6. Access your CKAN site

⚠️ GitHub.com Fallback ⚠️

CKAN 2.9 New Web Server options - ckan/ckan GitHub Wiki

CKAN 2.9 Deploying a source install - New Web Server options

1. Install Nginx

2. Create the WSGI script file

3. Create the WSGI Server

a. uwsgi

b. gunicorn

c. Apache

4. Install Supervisor for the uwsgi or gunicorn servers

a. uwsgi

b. gunicorn

5. Create the Nginx config file

6. Access your CKAN site

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️