A new default for $poplocal - GetPoplog/Seed GitHub Wiki

Background

The Poplog system is delivered as a collection of files (or "tree") that is loosely organised but delivers some rich capabilities:

  • libraries that are loaded explicitly (via lib)
  • libraries that are loaded if-needed (via uses)
  • variables whose definitions are loaded on demand (aka autoloaded)
  • hyperlinked help files in an idiosyncratic markdown organised into tutorials (teach), a programmers guide (help), reference information (ref) and long articles (doc)
  • saved images that enable fast start-up for common use cases
  • libraries to customise the development environment
  • and this structure is repeated for different programming languages

This rich capability is delivered not through a tightly organised file structure or some declarative configuration - but through the repeated use of search-paths. In fact it makes use of two types of search-paths, one based on environment variables (e.g. $popsavepath, $popcomppath) but more commonly list variables (e.g. popautolist, popliblist, popuseslist). These lists are mostly just names of folders but can also be functions that perform dynamic lookup.

(See HELP SEARCH_LISTS for more details)

A Poplog-tree

Although easy to understand and manipulate, search-lists are a very unstructured way of providing extensibility. So in addition there is a loose folder structure that we call a Poplog-tree. It has roughly the following shape

<tree-root>
    auto/    files that correspond to autoloadable variables
    lib/     libraries
    help/    documentation
    teach/   ditto
    ref/     ditto
    doc/     ditto

The search-lists are set-up to respect this structure. For example vedhelplist includes both $poplocal/local/help and $poplocal/local/teach. And this pragmatic structure brings a rough and ready order.

Only a fool would imagine that this tames search-lists. There is no warning when one search list shadows another and there is no protection against one library masking another and accidentally loading into the wrong project. They are a flexible, practical but flawed method.

What Gets Installed in $poplocal/local?

The $poplocal/local tree is intended to allow an individual or organisation to extend Poplog with additional packages or customisation as if they were part of the base system. In other words, the additions here will not just affect a particular user, they will also affect any Poplog script that compiles and/or runs on that system.

This has several implications. The first and simplest is that $poplocal/local must not be on read-only media. That is trivial on a standard installation. However, in a snap or appImage or any other structured delivery, the main Poplog-tree is mounted on a read-only area. The default definition of $poplocal is $usepop/.. which is read-only. So we definitely need to override the default definition in those cases. We go through this argument in more detail in the next section.

It also means that $poplocal/local should not be part of a user's $HOME directory. There is an alternative Poplog-tree for user-oriented customisations, namely $poplib. But because $poplocal is intended to be a system-wide, pervasive, unavoidable customisation that affects all scripts for all users on the system, it has to be located somewhere else. And this also suggests that it has to be off to one side of the main Poplog-tree e.g. /usr/local/poplocal.

Unavailable Locations for $poplocal

The default location of $usepop/.. is unavailable

Because GetPoplog aims to deliver Poplog in a convenient way, we give particular priority to "1-click" installers such as snap, appImage and in the future FlatPak and more exotic environments such as Jupyter and Sandstorm. And as a general principle we want to minimise the variations between different installations, so that an installation via (say) snap is difficult to distinguish from an installation via dpkg.

This means that we have to be prepared to accept that the base of the Poplog-tree ($usepop) is a read only tree. However, the GetPoplog installer has some additional super-structure to accommodate side-by-side versions and also to maintain equivalence with the FreePoplog distribution which it builds upon. That super-structure looks like this:

/some/path
    current_usepop -> V16             <-- $usepop
    V15/
        pop/...
    V16/
        pop/...

This super-structure is also captured as read-only. And it is that capture gives rise to a conflict. It conflicts with the usual default for $poplocal being $usepop/.. since $poplocal/local must be writeable. (Of course we could make $poplocal/local a symlink to somewhere else but that barely kicks the can down the road since we immediately ask that somewhere else will be.) And since $poplocal must be writeable for a sysadmin, this unfortunately rules out the default location.

$poplib is inappropriate

The $poplib environment variable names a third, under-developed Poplog-tree. This tree is primarily intended for users to customise their Poplog interactive-development experience rather than extend Poplog with additional libraries. There is no pre-defined method for the latter, although it is easy for users to implement roll-your own.

Because $poplib defaults to a folder in the user's $HOME folder, it varies depending on which user account is running Poplog. Since $poplocal is supposed to be a system-wide extension for all accounts, including script accounts, it is inappropriate to use any reference relative to $poplib.

/etc/poplog, /etc/opt/poplog, /usr/share and /usr/local/share are problematic

Because we need to store local saved images in $poplocalbin and saved images are binary objects, we should not be using any architecture independent location for saved images. Now $poplocalbin does not have to be inside $poplocal but that is preferred. In addition, $poplocal may contain object files, which is the nail in the coffin. None of these architecture-independent folders are appropriate.

The GetPoplog Preferred Location

Ultimately the Linux Filesystem Hierarchy Standard gives insufficient guidance on where local customisations of base applications that have a few architecture-dependent files should be kept. However it is important that the it should be located in an area that requires root access to prevent hijack attacks. There are two main candidates however:

  • In $usepop/../../poplocal aka $PREFIX/poplocal
  • In /usr/lib/poplocal or /usr/local/lib/poplocal (or use poplog rather than poplocal)

We prefer the first of these. So, for example, where Poplog is installed in /opt/poplog the extensions will be installed in /opt/poplocal. Our reasons for preferring this location are (1) the two folders are side-by-side visually in most file browsers, which is a good reminder to sysadmins and (2) the lib locations are usually used for base installations and would be slightly misleading.

⚠️ **GitHub.com Fallback** ⚠️