These (brief and potentially incomplete) instructions describe how to generate OpenStreetMap (OSM) tiles for serving to IRIS clients. The process is somewhat different now than it had been in the past. IRIS has moved from using the Apache web server to nginx, which doesn't provide the mod_tile extension used by OSM to render tiles as requests are received. Because of this, it is easiest to use a different server for generating tiles. You can use this separate server to serve tiles directly, or you can force render a set of tiles and copy them to your IRIS server.

The instructions in this guide have been tested on Ubuntu 22.04 LTS, as the most up-to-date OSM guides target Ubuntu or Debian distributions. Theoretically you should be able to translate the instructions to Fedora/RHEL/CentOS or another distribution, but that has not been tested. You could also set this up as a Docker instance, but that has also not been tested.

First, we must set up a tile server, generally following one of the guides at Switch2OSM. If you are also using Ubuntu 22.04, use this guide. The guide will have you download a small sample of map data from Geofabrik. Use whatever region is most relevant to your application, but it is recommended to start small for testing.

I typically choose to follow a modified version of these instructions to install a newer version of PostgreSQL and PostGIS. Here are the basic instructions I used to set up a tile server on a new Ubuntu 22.04 server.

First, make sure the server is updated:

sudo apt-get update && sudo apt-get -y upgrade && sudo apt-get -y dist-upgrade

Reboot the server if needed after installing updates.

Install some dependencies (note that PostgreSQL/PostGIS will be installed in the next step):

sudo apt install screen locate libapache2-mod-tile renderd git tar unzip wget bzip2 apache2 lua5.1 mapnik-utils python-is-python3 python3-mapnik python3-psycopg2 python3-yaml gdal-bin npm fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted fonts-unifont fonts-hanazono osm2pgsql net-tools curl

Now, set up the PostgreSQL repository to install the latest version, following these instructions. Note that PostgreSQL currently is still using the deprecated apt-key package signing method, but that's not our problem and presumably (hopefully) will be fixed soon.

# Create the file repository configuration:
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'

# Import the repository signing key:
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -

# Update the package lists:
sudo apt-get update

Install the latest PostgreSQL and PostGIS (15 and 3, respectively, at the time of writing):

sudo apt-get install postgresql-15 postgresql-15-postgis-3

Once the installation is finished, set up a database user for yourself.

Start a psql shell as the postgres user:

sudo -u postgres psql

Now create a user for yourself (named whatever you like, but I'm using dbadmin here). Using the same name as your system user has some convenient benefits.

CREATE ROLE dbadmin WITH SUPERUSER LOGIN;

Exit the shell by pressing Ctrl + D, then enter again using the new user:

psql -u dbadmin postgres

Create a role and database for generating tiles. We will use _renderd to match the user that runs the renderd daemon.

CREATE ROLE _renderd WITH LOGIN;
CREATE DATABASE osm WITH OWNER _renderd;

Connect to the new database:

\c osm

Now, enable the postgis and hstore extensions and make sure everything is owned by the osm role:

CREATE EXTENSION postgis;
CREATE EXTENSION hstore;
REASSIGN OWNED BY dbadmin TO _renderd;

Exit with Ctrl+D and return to the system shell.

You will need to download some files and make them available to the _renderd user. The Switch2OSM guide has you do this in your home directory, but it is somewhat better to put them in a separate directory. Here we will use a directory we create in /opt/, but you could easily put this somewhere else (e.g., a separate partition mounted at /mnt/data).

Make the directory and set the permissions:

sudo mkdir /opt/osm
sudo chown _renderd:_renderd /opt/osm/

Add your user to the _renderd group and change the permissions on the directory so you can write to it:

sudo usermod -a -G _renderd $USER
sudo chmod g+w /opt/osm

After this, log out and back in again to apply the group change.

We will use the same stylesheet as the one recommended by Switch2OSM (and the same ~/src directory for containing files). Install that from the OpenStreetMap Carto repository.

mkdir /opt/osm/src
cd /opt/osm/src
git clone https://github.com/gravitystorm/openstreetmap-carto
cd openstreetmap-carto

Install the carto compiler:

sudo npm install -g carto

Then convert the carto project into the Mapnik format:

carto project.mml > mapnik.xml

Note that if you want to change the database connection parameters for your system (e.g., to run on a different server or use a different database name, etc.), you will need to change the parameters in project.mml before running the above command. For information on the parameters to change, see the MML file structure documentation.

Now, we will download some data from Geofabrik, which is a good repository of up to date OSM data. Download whatever dataset makes sense for your application. We will use the entire US here, though note that this may be a little large and a smaller area (e.g., a single state) is recommended if you are just testing. We will put the data in a separate data folder in the home directory.

mkdir /opt/osm/data
cd /opt/osm/data
wget https://download.geofabrik.de/north-america/us-latest.osm.pbf

Make the _renderd user as the owner of all the files it will need so we can load the data:

sudo chown -R _renderd:_renderd /opt/osm/

Now we can run osm2pgsql to load the data. This can take a long time, so it's recommended to run it in a persistent screen shell. The following command will create a screen session named osm with a 10,000 scrollback buffer:

screen -h 10000 -S osm

In this session, run the following command. Note that you will likely want to change the numbers after the -C and --number-processes argument, which are the amount of memory allocated to the program and the number of processes used, to match the memory and CPU cores available on your server.

sudo -u _renderd osm2pgsql -d osm --create --slim  -G --hstore --tag-transform-script /opt/osm/src/openstreetmap-carto/openstreetmap-carto.lua -C 2500 --number-processes 1 -S /opt/osm/src/openstreetmap-carto/openstreetmap-carto.style /opt/osm/data/us-latest.osm.pbf

Once the process is running, you can detach from the session by typing Crtl+A then D. You can then close your command prompt/SSH session. To check on the process, open a command prompt or log back into the server, then enter the following command to reattach:

screen -x osm

Depending on the size of the data you downloaded, this may take several hours. Loading the entire US, for example, took about 5 hours when I tested it (allocating 10 GB of RAM and using 4 processes).

To render larger areas, you may choose to load data from multiple areas into your database (e.g., multiple states or countries) rather than picking a broader region (e.g., an entire country or continent). This is generally fine, however there is an additional step if you'd like to do this (in addition to downloading the extra files).

The osm2pgsql tool's --append mode does not perform very well on large datasets, because if runs a bunch of update checks for each updated node that comes in (see [this question] for more info). Instead, you are better off combining the two files, then loading the combined file in one go. One way to do this is using osmium, which you can install on Ubuntu with:

sudo apt install osmium-tool

You can then merge the files with (replacing with your filenames as needed):

osmium merge area1.osm.pbf area2.osm.pbf -o area1and2.osm.pbf

Note that you can combine as many files as you want with this, just add the filenames before the -o <output_file> argument.

Some indexes need to be created manually:

cd ~/src/openstreetmap-carto/
sudo -u _renderd psql -d gis -f indexes.sql

You also need to download some shapefiles for some additional miscellaneous things:

cd ~/src/openstreetmap-carto/
mkdir data
sudo chown _renderd data
sudo -u _renderd scripts/get-external-data.py

And finally, fonts:

cd ~/src/openstreetmap-carto/
scripts/get-fonts.sh

Note that I think you need to actually install these files somewhere too, but I'm not entirely sure. I think the most logical place would be /usr/share/fonts/truetype/noto, but again I'm not sure. It also looks like most of them are also there already. I think it also only matters for non-latin characters.

Renderd requires a little extra configuration. On Ubuntu 22.04, the configuration file is in /etc/renderd.conf. Start editing the file with:

sudo nano /etc/renderd.conf

(or another suitable text editor). Add a section like this at the end:

[osm]
URI=/hot/
XML=/opt/osm/src/openstreetmap-carto/mapnik.xml
HOST=localhost
TILESIZE=256
MAXZOOM=20

You may also wish to configure renderd to show debug messages. Edit the systemd service file with:

sudo nano /usr/lib/systemd/system/renderd.service

(or another suitable text editor) and add:

Environment=G_MESSAGES_DEBUG=all

after the [Service] line. Then reload and restart the service with:

sudo systemctl daemon-reload
sudo systemctl restart renderd
sudo systemctl restart apache2

You can now test that it works by opening a web browser and pointing it to http://<your-ip-address>/0/0/0.png (substituting your server's IP address or domain name). You can also try out the sample leaflet map in the "Viewing Tiles" section of the Switch2OSM guide (note that the sample map will initially be centered on Azerbaijan as the guide used for its example).

Once you have completed this process, your server should render tiles when it receives requests from clients. Because tiles can take a long time to generate, you probably want to pre-render a number of tiles, particularly at low zoom levels. (These instructions were taken from here.)

To do this, mod_tile provides the "render_list" command that can be used to request renderings of specific tiles. For instance:

render_list -n 1 -s /var/run/renderd/renderd.sock -z 0 -Z 7 -m osm -a

will use one thread to render all tiles from zoom level 0 to zoom level 10 in the "osm" named map style defined in renderd.conf. You can add the -f parameter to force re-rendering of tiles that already exist. This command can take a while, so it is recommended to use a screen session like in Loading Data so you can disconnect from the session while it is running.

TODO left off here...

If you want to pre-render tiles in a particular area, you can download another script to do this (assuming it remains available):

cd ~/src
git clone https://github.com/alx77/render_list_geo.pl
cd render_list_geo.pl

You can call this script with latitude and longitude parameters. For example, this will generate tiles for zoom level 11 for the area around Lincoln, NE:

./render_list_geo.pl -n 1 -z 11 -Z 11 -x -96.836396 -X -96.498908 -y 40.690743 -Y 40.941440 -m ajt

Tiles will be rendered in the .meta format, which works for web applications, however IRIS requires PNG images. To create these, you can use the meta2tile program.

First, clone the repository:

cd ~/src
git clone https://github.com/geofabrik/meta2tile
cd meta2tile

Then, install some dependencies. If you are on Ubuntu, this command will do it:

sudo apt-get install libzip-dev libgd-dev libsqlite3-dev libgdal-dev

Finally, build the program:

make

Now you can run meta2tile to create PNG files. For instance, if you used the ajt map name as suggested by the Switch2OSM guide, you can use this command:

./meta2tile /var/cache/renderd/tiles/osm/ /var/www/html/tiles

This command will put the PNG tiles on the server so they can be accessed at http://<server_ip>/tiles. If you want to host the tiles on your IRIS server, all you need to do is copy them to that server and place them in your webroot directory (likely also /var/www/html). In either case, once the tiles are in place, make sure you set the correct URL for the map.tile.url property of your iris-client.properties file.

PNG tiles require a lot of files, something like 50x the number of files at high zoom levels compared to .meta tiles. Because of this, there is a high likelihood of running out of inodes on a typical filesystem, even if it has plenty of space in bytes. To deal with this, you will either have to reduce the number of zoom levels and/or size of the area you use to generate tiles (or a combination), or use a filesystem that doesn't have this limitation (like btrfs). Choose whatever approach is best for your situation.

Following these instructions will give you tiles that match the default OSM style. This is probably fine for most cases, but you may want something different. In particular, road operators (i.e. IRIS users) may wish to have the map use MUTCD shield symbols to display highway numbers, rather than the generic format that OSM uses. There is an open request for OpenStreetMap to do this by default, but it seems likely to require customization for some time. This project is probably a good place to start, and this page may have some useful information too. Note that it is definitely possible to do this, but it hasn't been verified.