7 22 2024 Tech Team Report - QualitativeDataRepository/TechnicalTeam GitHub Wiki

7-15-2024

Logged Tasks

Date	Task	Hours (Main)
15-Jul-2024	Reporting, fix race condition in ORCID script, deply to dev/stage, make cvoc draft PR, coord, update extauth, webform, schema_metatag modules, deploy to dev/stage, investigate file lock on search results, invesstigate/fix broken dataset thumbs (#10686), create PR, meeting	6
16-Jul-2024	Start install of payara6.2024.6 solr 9.6.1 on prod, coord re orcid script, investigate Annorep fail on prod, inclusion of [] in params on stage, misconfigured AR server IP on dev, upate ARServer with newer libraries, configurable timeout for doc to pdf conversion, update chef, deploy new 0.0.7 ARServer to all, investigate report of restricted lock on search for one stage file	4
17-Jul-2024	Coord re: orcid/ror look&feel, investigate file index oddity, discover/fix large memory use in changing permissions, test, deploy to dev/stage, check permission reindexing after role assignment (OK)	4
18-Jul-2024	Adjust ROR icon styling, continue testing permissions indexing, create issue/PR, investigate /perms api call (separately inefficient)	4
19-Jul-2024	Update citation block on dev/stage, update solr schema, debug/update solrconfig.xml changes on dev/stage, pre-pop on prod, add logging to perms indexing to be able to see progress, update #7068 file PID reg at upload PR for QA, merge DataCite/+relationtypes PRs, check for missed QDR changes, create #10705/6 for Search code cleanup	4
20-Jul-2024	Create #10708 with MDC citation/PID case fixes, add test, release notes, #10709/#10710 - index/status improvements, updated Drupal webforms, stable theme	3
21-Jul-2024	Fix ORCID type mgmt, change to logo only links on github/stage. Created issues/PRs for Solr and libs updates, Cvoc changes for author field, ROR updates in DataCite XML #10711-13, updated DataCite XML/relationType PRs with latest, annotated all PRs with relevant tags, update keycloak docker-compose files in github/add 24.0.5 for prod.	4

Operations

Start installation of payara6.2024.6, solr 9.6.1 on prod
Updated citation block on dev/stage to add keywordTermURI field, updated solr schema
Found/fixed typo in github version of solrconfig, redeployed to dev/stage, staged on prod

SSO

Update Keycloak in git to use 24.0.5 for all machines

Drupal

Updated extauth, webform (x2), schema_metatag modules, stable theme

Dataverse

ORCID/ROR
- fixed race condition in script causing managed fields to show
- made cvoc PR with updated scripts
- adjusted styling per requests (only icons are links, no email popup), added ORCID/ROR icons
- deploy latest to dev/stage via github.io
- with GDCC support, presented the ORCID/ROR work to IQSS. (FWIW - I also used some DANS support to handle adding the ability to enter non-ORCIDs.)
Investigate/fix broken dataset thumbnails (#10686/#10687)
Investigate restricted file icon in search - discover that permission indexing failures caused out-of-date info. Tracked to large memory use when adding a role assignment, e.g. in the root collection, developed fix, created #10709/#10710 which reduces memory use by doing incremental updates, reindexed on dev/stage with new code (Also found/did not fix that the /perms API call to reindex all perms is similarly inefficient.)
Updated old file-PIDs-at-upload PR #7068 as part of review/QA at IQSS
Updated DataCite XML/relationType PRs with latest from IQSS develop and changes from QDR over the last ~2 weeks
Created an issue/PR (#10705/#10706) for the search code cleanup I did in merging v.6.3 with QDR code
Created #10708 to cover multiple updates related to making MDC citation counting work with the multiple PIDs code, added test, release notes
Created PR with solr 9.6.1 and other lib updates QDR has done over time that IQSS doesn't yet have (#10713)
Created PR with Dataverse code changes needed to support the new ORCID/ROR scripts (#10711/#10712)
Updated PRs at IQSS with more tags to get them noticed for prioritization/review/QA/merge

HEAL

TKLabels

with GDCC support, attended TKLabels meeting - discussed ext. vocab script and custom one-field block that have been created to allow association of a dataset with a Local Context project to show the Notices or, once updated, the TKLabels.

AnnoRep

Investigated reported failure on prod. Discovered 2 things: an older version of ARServer was running, updates chef code, reported issue is probably due to a timeout in how long a doc->pdf conversion can take. Created a new ARServer version with a configurable timeout and updated libraries, deployed to dev/stage/prod. (Also fixed an old IP address in the AR Client config on dev.)
Also investigate problem on stage - AR Client now sending [] with array params which causes a failure - reported that this is probably from a library upgrade/incorrect config to suppress [] chars.

Discussion

FWIW: During the demo of ORCID/ROR for ICPSR, I suggested they could create a script for the ICPSR thesaurus, or I could potentially get it to a demo stage if they can send a machine-readable version of the terms.

Plans

Work on MFA w.r.t. on authentication issue #43(MFA, etc.)
Shephard relationship type entry in metadata block #44 (more metadata to DataCite, etc.) through testing/review/QA
Background work to change/remove deprecated Drupal modules in prep for 11.0.0
Fix Stata-14 ingest by allowing file inspection during direct upload or adjusting the Stata ingester.
Fix #113 if possible
Matomo - investigate event-level tracking via tag manager, remove non-working google scripts
AnnoRep - explore round-trip, configure auto-start and log rotation
Ops
- check missing globalidcreationdates and fix via /modifyRegistration or alternative
Dataverse
- Make PR for guestbook adding datasetversion fix
- Popup info accessibility - IQSS likes the recommendations from the source I linked to, so this can be implemented along those lines.
QDAS Previewer
- Updates per request
- Investigate writing aux file/previewing lower-sensitivity version and/or other write options
TBD: FRDR Security