Digital accessioning workflow - smith-special-collections/sc-documentation GitHub Wiki

May not be accurate, use with caution

Workflow Overview

  • Content is received from donor via electronic transfer or on physical media
  • Accession is recorded
  • Physical materials (if any) are accessioned, computer media is flagged and described
  • Digital content is copied or downloaded to workstation
  • Content is assessed; recommendations are shared with collection steward
  • Files are packaged with metadata and documentation to create Archival Information Package (AIP)
  • AIP is transferred to storage server
  • Access files are created if needed, and transferred to LibAccess
  • Metadata and documentation are completed

See also general Workflow for Accessioning Manuscripts & Archives

1. Acquisition: Digital content is received

  • Collection steward coordinates transfer of digital content to Special Collections via

    • electronic transfer (file sharing service, email attachment, FTP)
      • steward forwards email with attachment or link to Accessioning Archivist
    • computer media with papers/records (drives, CD/DVDs, disks)
    • individual hard drive

    Digital Preservation Archivist may be consulted or asked to assist donor with transfer.

    See Workflow for Accessioning Manuscripts & Archives for details.

2. Accession is recorded

  • Collecting steward creates skeletal accession record and creates card in Accessioning Airtable base
  • Accessioning Archivist creates card in [TBD; previously Trello] Note: this workflow will be reviewed and updated by the incoming Digital Archivist.

3. Physical materials (if any) are accessioned

4. Digital content is copied or downloaded to workstation

Follow transfer workflow for type of material and transfer method:

  • Electronic transfers:

    • Download files to workstation
    • Content is scanned for viruses automatically upon download
      • If file is too large, scan manually after download
    • Create digital file list (or this can be done later upon transfer to storage)
    • Start log file, if appropriate, and attach to Trello card in Digital Accessions board

    Note: e-transfers should be considered high-priority since they may be available online for a limited time.

  • Physical media:

    • See Protocols for computer media (Draft)
    • Start digital accession log, if appropriate, and attach to Trello card in Digital Accessions board
    • Copy media according to Instructions.
    • A file list is created during transfer process

5. Assess digital content

  • Perform general assessment of content, visually inspecting contents and file list(s) without opening files.
  • Optional: run software to identify file formats (or this may be done later during processing (see below))
  • Compare contents to documentation and descriptions provided by donor; notes any discrepancies and notify collection steward to contact the donor.
  • Make recommendations to collection steward regarding disposition of unwanted material (commercial files, software and system files, deleted files, duplicate content, etc.), as well as potentially sensitive materials.
  • Document preservation actions and make processing recommendations:
    • In the log, record unique identifiers for each media item and preservation steps taken; if media could not be copied; special software needs to read files (if known).
    • Additional logs may be automatically generated by tools during accessioning
    • Preservation and processing information is also noted (more generally) in the Processing Information note in the resource record

6. Prepare Archival Information Package (AIP)

  • Organize files, documentation, and metadata in 3 subfolders:

    • Objects/Data: original files; disk image(s) (if applicable); normalized objects
    • Metadata: file manifest
    • Documentation (may include): log file (downloaded from Google Drive), photos of media, checksum file, emails, release forms, other docs received from donor

  • Assign unique identifier (filename for AIP): See Guidelines for Digital Object Identifiers

7. Transfer AIP to storage server: LibPres or LibTemp

Save to temporary storage (LibTemp) if further appraisal is needed, or if there is pending donor documentation, otherwise save to LibPres.

  • Using a file transfer tool such as Data Accessioner, transfer files to preservation storage
  • Checksums are created and verified upon transfer
  • A new file list is generated (AIP-level)
    • Covert xml file to csv/xls to attach to Resource record
    • Save a copy in the metadata subfolder of the AIP and a copy on workstation
  • Save copies of files on workstation if access files are needed. Otherwise they can be moved to trash.

8. Create access files, if needed and transfer to LibAccess

9. Complete metadata and documentation

Further appraisal and processing may happen immediately, or at a later date.

IMPORTANT: all processing should be performed on copies on your workstation, do not attempt to open the original files in preservation storage.

  • Processing and appraisal activities may include:

    • identifying and removing duplicate files
    • weeding unwanted content (commercial files, software, system files, deleted content)
    • normalization of file formats for preservation and access
    • creation of access copies
    • additional forensic activities (e.g., identification of potentially sensitive content or PII)

  • After processing, a final file manifest is created and save in the Metadata folder. The original filelist is saved in the Documentation folder and renamed "filelist-original"

  • If access copied were received with the accession they are moved to a separate directory and a a separate file list is created (for online access, these can be moved to the staging server)

  • Once processing is completed, the original AIP can be replaced

  • Preservation actions are documented in a log file

  • Resource record is updated

    For more information see Digital processing guidelines (draft)