Design of the Makefile - GetPoplog/Seed GitHub Wiki

Aims of the Seed Makefile

The purpose of the Seed Makefile is to automate the key tasks in building and installing a fully working Poplog system. I also wanted it to help bring Poplog into the wider Linux/Unix packaging ecosystem and a Makefile, rather than a raw shell script, is one of the stepping stones for doing this.

It is intended to operate in several "pre-build" scenarios:

  • Where the Seed repo has been cloned (or downloaded) along with its supporting scripts. No pre-build step is required: make build
  • Where the set of resources have been downloaded in advance: make download && make build
  • Where the Seed, Base and Corepops repo have been cloned together. In this case we acquire the set of build resources from the others repos: make use-repos && make build

In each case, the Makefile supports the following targets:

  • build - downloads the sources and builds a Poplog tree in _build/poplog_base
  • install - installs the newly built-tree as a new version into the POPLOG_HOME_DIR, defaulting to /usr/local/poplog
  • clean - removes all downloaded and built artefacts
  • jumpstart - installs the required packages; a one-time convenience.
  • uninstall - removes the entire Poplog installation (all-versions) and symlink but leaves a backup in /tmp
  • really-uninstall-poplog - removes the entire Poplog installation (all-versions) and symlink, no backup

Declarative Targets

We aim to make our internal targets independent, restartable and monotonic. Independent means that the targets can be made in any order and automatically sort out their own dependencies. Restartable means that when a target fails, it is safe to continue from where it left off. We manage the latter by arranging that any file-updates are deferred until all potentially failing operations are complete. Monotonic means that once a target succeeds it is not re-executed (at least not until make clean).

Building a fully working Poplog-tree requires multiple build phases that imperatively operate in-place. However Makefiles work best when they exclusively add new files rather than update files through multiple, hard-to-identify stages. So one of the tricks used was to create declarative targets that represented the successful completion of imperative operations. These are implemented by empty "proxy flag" files and have the form _build/NAME.proxy.

Another trick we use is to copy entire directory-trees to safely isolate file-updates. If some part of the update fails the targets are designed to re-copy the trees when restarted. This does mean that the build process uses more disk space than might be expected but we think this is the right tradeoff. Besides, Poplog is a very small system by today's standards.

Build Variables

We wanted to make it possible to install the Poplog-tree in an arbitrary folder. The installation folder is defined by the make-variable POPLOG_HOME_DIR, which plays a similar role to PREFIX in many other build scripts. By default this is /usr/local/poplog but another good choice would be /opt/poplog. By overriding the POPLOG_HOME_DIR this is straightforward. Note that this variable is only referenced during installation.

# Install the Poplog-tree into /opt/poplog (the poplog commander will still be in /usr/local/bin)
make build
sudo make install POPLOG_HOME_DIR=/opt/poplog

We also wanted to make it possible to put the symlink to the poplog commander tool in another bin directory apart from /usr/local/bin. And this can be done by overriding the EXEC_DIR variable during installation.

# Install the Poplog-tree into /opt/poplog and the commander into /usr/bin
make build
sudo make install POPLOG_HOME_DIR=/opt/poplog EXEC_DIR=/usr/bin

It is also important to be able to use an experimental branch. This is intended to be done via the MAIN_BRANCH variable. However with the separation into different repos we aren't too sure this is what will be needed - when we get CI working we will have to revisit this.

Side-by-Side Versions

It is not that unusual to have multiple Poplog installations on the same machine. To support this configuration, the install-target will not remove different earlier (or later) versions. So the $POPLOG_HOME_DIR might look like this:

$ ls -l /usr/local/poplog
total 7
drwxrwxrwx 3 root  root  4096 May 17 10:40 V15
drwxrwxrwx 3 root  root  4096 May 17 10:40 V16
lrwxrwxrwx 1 steve steve   33 May  9 05:06 current_usepop -> V16/poplog_base

When adding a new version of Poplog, a new folder is added to $POPLOG_HOME_DIR and the current_usepop symlink is updated to that new folder. By contrast, uninstall make no attempt to act on a specific version but removes all versions of Poplog.

Overwriting a Previous Installation - or Not?

If the same version is installed a second time, the install-target will preserve the existing Poplog-tree. For example if there was already a V16 installed then re-installing it will rename it to V16.orig and then create a new Poplog-tree in V16. A further attempt to install will preserve the V16.orig if it exists - instead it will remove any V16.prev, rename the existing tree to V16.prev and then create the new Poplog-tree in V16. The effect is like this:

After 1 install:  V16
After 2 installs: V16 V16.orig
After 3 installs: V16 V16.prev V16.orig 
After more:       V16 V16.prev V16.orig

The general idea behind this scheme is not to lose a long-standing version (hence V16.orig does not get overwritten) but if you run multiple installations in a short space of time, not to clog up the $POPLOG_HOME_DIR with lots of backups.