10 2 2023 Tech Team Report - QualitativeDataRepository/TechnicalTeam GitHub Wiki

10-2-2023

Logged Tasks

                            Date             Task Hours (Main) Hours (EOLS) Hours (PII) Hours (QDAS)
25-Sep-2023 GBAR - fix preview terms dialog issue, create fix for tools terms dialog issue, fix missed query updates from review, test/update IQSS PR, dev, and stage branches, investigate fail msg for archiving on prod, determine that archiving worked, update db, reporting, meeting. 4
26-Sep-2023 Investigate issue with page refresh from #9954 on QDR forks. 1
27-Sep-2023 Investigate #7956 - change displays of name/size/mimetype in file table, file page, file citation to use original, explore options for handling name changes for ingested files, consolidate two flyways in gbar PR, update honeypot module, cleanup, deploy page-refresh fixes to dev/stage. 5
28-Sep-2023 Coord re: original vs tab name editing, update #7956, Help debug storJ upload fails on prod - create dvwebloader issue re: not reusing UI. 3
29-Sep-2023 Update keycloak-orcid plugin to use member API - change to scope, URL, Keycloak config; update qdr-keycloak dev config to ~match stage, fix doc issue seen in GBAR QA, update composer to 2.2.22 for sec fix on dev/stage 4

Operations

  • Help debug folder upload problems on prod - added dvwebloader repo issue re: avoiding allowing reuse of page (which doesn't work)
  • Investigate failure message on prod archiving - looks like initial archiving worked, but a manual repeat (prior to the original completing) gave a fail message. Updated db to show success.
  • Discovered/reported that stata-14 files aren't being ingested at QDR due to the use of direct-upload (which stops checking the file contents when assessing the mime type) and the fact that the Stata ingest code relies on the mimetype rather than checking for the version during ingest.

SSO

  • Update keycloak-orcid module to use member API, deploy to stage, update configuration, test.

Drupal

  • Updated honeypot, deployed to dev/stage
  • Updated composer version on dev/stage for security

Dataverse

  • GB-at-request PR
    • fixed issue displaying terms dialog when launching previewer
    • discovered/fixed old issue re: showing terms dialog when launching other tools
    • fixed missed query updates found in review
    • merged two flyway scripts into one
    • update IQSS PR, fixed doc example typos, coord re: final merge
    • deployed updates to dev/stage
  • Updated dev/stage with fix for page refresh - an earlier optimization change had broken it in 5.14
  • Investigated/created initial fix for #7956 - displaying original file instead of .tab version. Deployed to dev/stage. TBD - how to handle changing file name extensions (Dataverse currently only allows changing the extension for the tab version.)

AnnoRep

Discussion

  • FYI: The GB-at-request PR has been merged at Harvard
  • Added fixing Stata-14 ingest to future work
  • Using the ORCID Member API causes users to be asked to trust QDR, and login will now work if email is public or available to trusted parties, but will still fail if/when email is private. Need to discuss how/where to add messaging to address this case.
  • FWIW: we noticed in the failed PID reports that old corrupt datasets now show up in the collection /contents API call but they do not have DOIs and we can see the effect of this in the failed PID report as calls to the uppdateCitationsForDataset API with "doi%3A" from localhost. Removing/fixing the corrupt datasets should stop this.

Plans

  • SSO
    • Clean out old corrupt test datasets - required before v5.14 can be deployed to prod.
    • Support testing as needed
  • Fix Stata-14 ingest by allowing file inspection during direct upload or adjusting the Stata ingester.
  • Fix #113 if possible
  • Matomo - investigate event-level tracking via tag manager, remove non-working google scripts
  • AnnoRep - explore round-trip, configure auto-start and log rotation
  • Ops
    • check missing globalidcreationdates and fix via /modifyRegistration or alternative
  • Dataverse
    • Make PR for guestbook adding datasetversion fix
    • Popup info accessibility - IQSS likes the recommendations from the source I linked to, so this can be implemented along those lines.
  • Drupal - v10 - review compatibility and start updates.
  • QDAS Previewer
    • Updates per request
    • Investigate writing aux file/previewing lower-sensitivity version and/or other write options
  • TBD: FRDR Security