Python Packaging, Repository Tools and Github Actions - AST-Course/AST5110 GitHub Wiki

This page covers the extra files and tools which are in the library repository.

These are not things you need to know or will be tested on but provide context on basics of creating a Python package and the kind of tools which are used to format and check for common mistakes in Python code.

Python Packaging

These are files you should not have to edit

To create a Python library is now done with justpyproject.toml. setup.py and setup.cfg are deprecated and will be unsupported in the future. pyproject.toml is distributed in blocks (following the TOML specification), and one can find the following:

[build-system block]

Here, this pyproject.toml will use "setuptools" to "compile" the Python library.

[project]

Describes the name of the library, description, authors, and keywords; readme and license files; required Python, classifier, and dependencies.

On top of these two main blocks, other additional blocks are:

    [project.urls]
    [project.optional-dependencies]
    [tool.setuptools_scm]
    [tool.setuptools]
    [tool.setuptools.packages.find]
    [tool.towncrier]

These are designed to separate the configuration for either a specific tool or provide extra information for the build system. If you want to see more information, the Python Packaging Authority has a breakdown of the specification.

Automated testing and formatting.

Other files relate to running pre-commit or tox.

The former is a tool which runs other tools, avoiding one having to run multiple tools separately. The latter is a tool which enables the automatic testing of Python code in a consistent manner.

For the "pre-commit", it has one file which lists the tools it will run and the configuration for those tools are:

_typos.toml contains the list of words that the automatic spell checker will not replace. It is very common that words using in programming are not legitimate english words and will be changed, so this file prevents that.

ruff.toml to run a command-line tool called Ruff which checks for known or common flaws when coding Python. It is able to auto fix some problems but otherwise it will inform you what the problem is which you will have to fix manually. The configuration for this tool is pretty complex, so we suggest looking at the manual if you want to make changes.

To run pre-commit, you will need to make sure it's installed and then you can do the following:

   $ pre-commit run -a

Or you can run this tool via tox as well:

   $ tox -e codestyle

For tox, the configuration file tox.ini is broken into sections. The top section just describes the range of environment names which the user can list by doing:

   $ tox -l

Then you can run any of them by doing:

   $ tox -e py312

The rest of the file is broken into sections which define the "base" environment, the one run when you try "py" or "py313" etc Then the other sections define the specially named ones like "codestyle" or "build-docs", these ones have special commands and require their own section. Each section allows one to say what Python packages to instal, what commands should be run, how those commands can be modified in real time as well. As with Ruff, tox is a complicated tool and has a large manual.

These are files you should not have to edit

GitHub Actions aka Workflows

GitHub Actions is a system that allows people who work on GitHub to automatically run code in the cloud. To do this, configuration files are created in the .github folder: .github/workflows/filename.yml.

If you are working on GitLab, the simplest configuration is one file at the top level called .gitlab-ci.yml.

Going back to GitHub, the workflows are currently setup to run two separate jobs: pre-commit and tox. These ensure that changes made by a user or developer are checked to ensure at least a basic level of functionality. They are using by larger packages to ensure that any code changes don't break the existing codebase.

Much like all configuration files, the layout of these files are also in blocks. They enable one to pick the operation system, the architecture to run on and what scripts or commands should be run in seriel.

The workflows in nm_lib will check for any simple code issues using the codestyle environment within tox. It will also run the unit tests (functions which are designed to ensure that a block of code is working as expected), one of which is a test for exercise 2. In this course, you will learn how to add unit tests via this exercise.

There are other workflow and pipeline tools separated from Gitlab and Github, such as Jenkins. Which should never be used as it is garbage software and when they die, they go to hell.