Workflow: automated dissemination - thoth-pub/thoth-dissemination GitHub Wiki

This page explains the process for automated dissemination of Thoth works to other platforms, and denotes the steps the Project Manager needs to take to ensure it runs smoothly.

Background

At present, works are automatically disseminated to the Internet Archive and to Loughborough University's Figshare repository. The same process is also used to register DOIs with Crossref. Any individual publisher can opt in to any individual platform (this is controlled via repository variables). The automated process for each platform runs on a scheduled basis. It aims to perform a "catch-up" dissemination of works which have become valid for submission since the previous run.

The process for each platform is controlled via a GitHub Action named [XX]-bulk-disseminate (where XX = ia for Internet Archive, fs for Figshare, cr for Crossref, etc). They can be found under the Actions tab of this GitHub repository. The direct link to the Actions tab is here.

From each Action's page, anyone can view details about times when the process was run, and members of Thoth can enable/disable the process. Successful runs are marked with a green tick, and failed runs are marked with a red cross.

Works which have been successfully disseminated via this process may be made visible on the target platform, sometimes under a dedicated Thoth Archiving Network collection. It may take a few hours before a disseminated work is made fully available.

Dissemination platforms

Platform Publishers Schedule Target Action name Action link Repository variables Collection link
Internet Archive Open Book Publishers, punctum 1st of every month, before UK OOB All works published since last run ia-bulk-disseminate https://github.com/thoth-pub/thoth-dissemination/actions/workflows/ia_bulk_disseminate.yml IA_ENV_PUBLISHERS, IA_ENV_EXCEPTIONS https://archive.org/details/thoth-archiving-network
Loughborough Figshare Open Book Publishers 7th of every month, before UK OOB All works published in preceding month fs-bulk-disseminate https://github.com/thoth-pub/thoth-dissemination/actions/workflows/fs_bulk_disseminate.yml FS_ENV_PUBLISHERS https://repository.lboro.ac.uk/Thoth_Archiving_Network
Crossref Open Book Publishers Hourly, around 45 minutes past All published works updated since last run cr-bulk-disseminate https://github.com/thoth-pub/thoth-dissemination/actions/workflows/cr_bulk_disseminate.yml CR_ENV_PUBLISHERS N/A (check DOI link(s) to confirm success)

Project Manager's responsibilities

Managing publisher opt-ins

  • The list of publishers who have opted in to automatic upload for each platform is stored under the [XX]_ENV_PUBLISHERS repository variable at https://github.com/thoth-pub/thoth-dissemination/settings/variables/actions (where XX represents the platform name, as above).
  • If a new publisher wants to join the scheme, the Project Manager can add their Thoth Publisher ID to the relevant list(s).
  • For platforms where individual publishers share their credentials with Thoth (e.g. Crossref), rather than Thoth having its own credentials (e.g. Internet Archive), additional steps are required:
    • The publisher's credentials need to be added in the appropriate format to the list of repository secrets at https://github.com/thoth-pub/thoth-dissemination/settings/secrets/actions.
    • e.g. for Crossref, the username needs to be added as CROSSREF_USER_[XXXX] and the password as CROSSREF_PW_[XXXX], where XXXX represents the publisher's Thoth ID, with hyphens changed to underscores and letters capitalised.
    • The Thoth development team can handle this if preferred; steps noted here for full reference.

Handling dissemination failures

  • The Project Manager should configure their GitHub settings so that they will receive alert emails if any of the processes fails. For the processes which only run once a month, it is also worth actively checking their status on the day of the run, by viewing the relevant Action page.
  • Detailed instructions on investigating failures are available here.
  • For Crossref, the process itself may succeed, but Crossref may later report back an error via email to [email protected] (an alias of [email protected]). Email filters ensure that any such emails go into the subfolder Crossref submissions > Error reports. This subfolder should be regularly checked in case of errors which the publisher needs to rectify. Once any error report email has been dealt with, it can be moved to the subfolder Crossref submissions > Checked.

Other maintenance

  • Each process can be enabled and disabled at the relevant Action link.
  • Note that "scheduled workflows are automatically disabled when no repository activity has occurred in 60 days". This is unlikely to happen often while Thoth is under active development, but if the repository has been inactive for close to 60 days, the warning banner below will be displayed. Click on "Continue running workflow" for each process in turn to prevent them from being automatically disabled.
  • Some processes, such as Internet Archive, have an additional [XX]_ENV_EXCEPTIONS repository variable, which stores a list of works which should be excluded from the automatic upload process. At the moment, these only comprise legacy publications with no uploadable PDF version. (Attempting to upload them would fail due to the lack of PDF.) The Project Manager can add additional Thoth Work IDs to this list if required.