10 16 2023 Tech Team Report - QualitativeDataRepository/TechnicalTeam GitHub Wiki

10-16-2023

Logged Tasks

                            Date             Task Hours (Main) Hours (EOLS) Hours (PII) Hours (QDAS)
9-Oct-2023 Reporting, upload docs to drupal_public, meeting 1
10-Oct-2023 Fixed ORCID messaging, investigate storJ ~rate-limiting issue 3
11-Oct-2023 Continue investigating failing datasets - find #10003, create solution & #10004, deploy, test, deploy to stage (w/orig file fix as well), update drupal webforms to 6.1.7 5
12-Oct-2023 Reindexing failed datasets on stage, investigate one silent failure, reindex perms, lower logging in AuthFilter, deploy to dev/stage, coord re deployment, generate/send list of controlled datasets on prod 4
13-Oct-2023 Investigate/fix/create PR for #10009 re: templates with custom terms, deploy to dev/stage, update email reg module on dev/stage 2

Operations

  • Uploaded docs to drupal_public on prod per request
  • Ran /api/admin/index/status and fixed all issues on dev/stage/prod (after Dataverse fixes for storJ). Stage had one failure due to the 1024mb limit for Dataverse heap, otherwise the issues were storJ, bad geospatial bounding boxes (on dev, fixed last week), and ~random failures that were fixed simply by re-running indexing and/or running the 'remove-orphans' API call (cleans up solr docs not corresponding to existing datasets/files).

SSO

  • Updated ORCID messaging on login page

Drupal

  • Updated webforms module, deployed to dev/stage
  • Updated email reg module, deployed to dev/stage

Dataverse

  • Further investigation of storJ failures led to discovering/fixing issue with conflicting S3 authentication mechanisms - our AWS role-based access control was taking precedence over the storJ profile-based credentials and failing. Fixed on dev/stage and reported/fixed at IQSS (#10003/#10004)
  • Found/fixed issue w.r.t. templates with custom terms breaking dataset creation. This was an issue that was previously fixed but the guestbook-at-request PR, which was created before the fix, moved the affected code to a new file and thus recreated the issue. Fix deployed to dev/stage and in #10009/#10010

AnnoRep

Discussion

  • Scheduling prod deployment - Wed.?

Plans

  • Deploy everything to prod
  • Create rich HTML previewer to replace existing one
  • Fix Stata-14 ingest by allowing file inspection during direct upload or adjusting the Stata ingester.
  • Fix #113 if possible
  • Matomo - investigate event-level tracking via tag manager, remove non-working google scripts
  • AnnoRep - explore round-trip, configure auto-start and log rotation
  • Ops
    • check missing globalidcreationdates and fix via /modifyRegistration or alternative
  • Dataverse
    • Make PR for guestbook adding datasetversion fix
    • Popup info accessibility - IQSS likes the recommendations from the source I linked to, so this can be implemented along those lines.
  • Drupal - v10 - review compatibility and start updates.
  • QDAS Previewer
    • Updates per request
    • Investigate writing aux file/previewing lower-sensitivity version and/or other write options
  • TBD: FRDR Security