Denmark: How to Run HWR Models - byuawsfhtl/RLL_computer_vision GitHub Wiki

Guide

This document provides a guide on how to run inference (i.e. predict) using the transcription neural networks developed at SDU and deployed at BYU.

Before running HWR...

Some datasets have a lot of blank images. These images don't need to be run through the entire hwr pipeline. We have a tool that can detect if an image is blank, a dash, quotes, or has words. See ~/fsl_groups/fslg_census/tools/blank_recognition (on the supercomputer) for the project files and the README in that folder for instructions on how to run it.

General Information

The predict packages are located here: ~/fsl_groups/fslg_census1940/compute/denmark_hwr/predict-packages. The file predict.sh is responsible for all packages: No matter which package needs to be used, predict.sh should be called. Calling the correct package is then handled by predict.sh.

Before presenting the general API, note the use of the following abbreviations:

  1. PACKAGE: This is the package used for prediction, e.g., gender.
  2. path/to/output: This is the directory that will be created and where output will be stored.

Whenever executing a call to predict.sh, the output will be saved in path/to/output and consists of a csv-file preds.csv with columns (filename, pred, prob).

Important note: Each run should have its own unique path/to/output. The setup is very strict and will avoid writing to a directory which already exists. This is to minimize the risk of overwriting files.

API

The most important argument to provide, aside from selecting the package and where to store output, is --predict-folders, which controls which folders are searched in for images. Multiple folders can be selected and tarballs (.tar.gz) can also be selected.

Important note: The slurm output will be written to the folder /fslhome/fslcollab280/fsl_groups/fslg_census1940/compute/denmark_hwr/predict-packages/slurm-out.

Minimal

To predict on all images inside the (ordinary) folder path/to/images, use:

sbatch predict.sh PACKAGE \
--output path/to/output \
--predict-folders path/to/images

Important note: The argument PACKAGE must be provided as the first argument. The order of the remaining arguments does not matter.

To predict on all images inside the tarball path/to/images.tar.gz, use:

sbatch predict.sh PACKAGE \
--output path/to/output \
--predict-folders path/to/images.tar.gz

Important note: When predicting on images in a tarball, the tarball is first unpacked and then, after prediction is done, the unpacked version is deleted. It is possible to not delete the unpacked images by adding the argument --keep-unpacked.

Advanced

Often, prediction on images in multiple folders is wanted. Further, in order to speed up prediction, using a large batch size with a sufficient number of workers is crucial. This section explains how to predict on multiple folders and how to specify the batch size and the number of workers.

To predict on all images inside the folders path/to/images-1 and path/to/images-2 and the tarball path/to/images-3.tar.gz, using a batch size X and Y workers, use:

sbatch predict.sh PACKAGE \
--output path/to/output \
--predict-folders \
    path/to/images-1 \
    path/to/images-2 \
    path/to/images-3.tar.gz \
-b X \
-j Y

If the folder path/to/images contains subfolders, each with images, it is possible to automatically include those, adding the argument --predict-on-subfolders, like so:

sbatch predict.sh PACKAGE \
--output path/to/output \
--predict-folders path/to/images \
--predict-on-subfolders

If the tarball path/to/images.tar.gz contains subfolders with images, adding the argument --predict-on-subfolders will also include those.

Examples

Example 1 (simple)

In this example, we will transcribe images using the gender model. Let us predict on the images in the tarball /lustre/scratch/grp/fslg_census/projects/for_torben/genders_for_denmark/genders_only.tar.gz. We will write the output to the folder /fslhome/fslcollab280/fsl_groups/fslg_census1940/compute/denmark_hwr/predict-packages/output-folder-for-examples/example-1. We will use a larger-then-default batch size.

To do this, open the terminal and type:

ssh [email protected]
PASSWORD
VERIFICATION-CODE
cd /fslhome/fslcollab280/fsl_groups/fslg_census1940/compute/denmark_hwr/predict-packages
sbatch predict.sh gender \
--output /fslhome/fslcollab280/fsl_groups/fslg_census1940/compute/denmark_hwr/predict-packages/output-folder-for-examples/example-1 \
--predict-folders /lustre/scratch/grp/fslg_census/projects/for_torben/genders_for_denmark/genders_only.tar.gz \
-b 512

Important note: The folder /fslhome/fslcollab280/fsl_groups/fslg_census1940/compute/denmark_hwr/predict-packages/output-folder-for-examples is only meant for the examples of this guide. We recommend you to not write output to /fslhome/fslcollab280/fsl_groups/fslg_census1940/compute/denmark_hwr/predict-packages. Keep this folder as clean as possible!

Example 2 (advanced)

Let us make an example together.

Details on Packages

Each "predict-package" from SDU consists of five files and a folder:

  1. timmsn: Folder where most of the code is located. Will remain mostly unchanged between packages.
  2. argparser.py: Script responsible for argument parsing to Python. Will remain mostly unchanged between packages.
  3. args.yaml: File stores hyperparameters and the like for the network and some parameters for data loading.
  4. formatter_*.py: Script responsible for formatting values, i.e. translating between "English" and the format used internally by networks. Will differ significantly between packages.
  5. model.pth.tar: Network parameters (or weights).
  6. predict.py: Script responsible for predicting. Will remain mostly unchanged between packages.