8 21 2023 Tech Team Report - QualitativeDataRepository/TechnicalTeam GitHub Wiki

8-21-2023

Logged Tasks

                            Date             Task Hours (Main) Hours (EOLS) Hours (PII) Hours (QDAS)
14-Aug-2023 Reporting, meeting, add submit functionality to new partial reg forms, configure Keycloak for email, test, investigate non-delivery for some addresses. 6
15-Aug-2023 Update DVUploader to use correct fixityAlgorithm, deploy 1.2.0beta3 (while fixing bugs for IQSS) 2
16-Aug-2023 Test, handle missing consentver, reg form fields w/o warnings, reformat, cleanup, add mailOptIn validation, deploy D8 via Jenkins, test; Remove unused Shib code from Dataverse, change settings to :SSO*, test, investigate Google app publish/verification requirements, create v5.14-qdr branch, merge, test, build, create forked version of keycloak-orcid plugin in github. 7
17-Aug-2023 Start stage upgrade, check docker-compose, install keycloak, backup ldap, remove posixAccount from users/schema/accesscontrol, find/fix stage-specific user issues, setup apache proxy, shutdown shib idp/module on dev. 6
18-Aug-2023 Configure keycloak/document settings, investigate making keycloak admin console blocked, deploy drupal, develop bootstrap process, deploy Dataverse/fix flyway issues, add oidc-keycloak provider, set :SSOLogoutUrl, test local login/logout 6

Operations

SSO

  • Finish T&C and social new account form submission logic, test.
  • Configure Keycloak to send mail, coord w/Seba to resolve delivery failures.
  • Suppress warnings about missing info when creating new social accounts
  • Add/fix mail opt-in validation
  • Cleanup code
  • Deploy Drupal via Jenkins, test
  • Remove shib module from Apache, turn of Jetty on dev
  • Make a fork of the keycloak-orcid plugin as a QDR github repo, add modified code (uses email as uid/account name)
  • Deploy to stage while documenting process
    • backup dbs
    • Remove posixAccount class and attributes from LDAP db, find/fix accounts with long uid/email that caused failures in removing class/attributes, remove class from schema after finding/removing ACL entry using posixAccount attribute
    • Check docker-compose install, install keycloak and keycloak-orcid, walk through all config entries for realm, client, client scope, ldap, orcid.
    • setup Apache proxy for Keycloak on stage, investigate options for keeping admin console behind VPN
    • Install new Drupal modules, develop bootstrap process to login via shib, set new Keycloak/SSO module settings, logout and login via Keycloak
    • Deploy Dataverse v5.14-qdr, fixing flyway conflicts, add oidc-keycloak provider via API, set :SSOLogoutUrl, test local login/logout

Drupal

Dataverse

  • Update DVUploader to use correct hash algorithm, make a 1.2 beta2 release (fixed some bugs working for Harvard at the same time).
  • Remove unused Shib code from Dataverse, replace :Shib-related SSO settings with new :SSO* ones
  • Merge with 5.14 code, create v5.14-qdr branch with new SSO code

AnnoRep

Discussion

  • For stage/prod - should have a way to keep Keycloak admin console firewalled
  • For stage - planning to set up Google (for existing accounts/real Google accounts) and ORCID sandbox (@mailinator.com accounts only - easy to test creating new QDR accounts w/o requiring test Google/ORCID accounts)
  • Google policy: Unclear if we need to go through verification process. I found OAuth-based plugins: if you're setting up an OAuth-based plugin for a popular platform, such as SMTP for WordPress, you don't need to go through the verification process. which suggests we don't have to. Verification seems focused on cases where you do more than just authenticate, i.e. you use people's Google data in some way). If we need it, we'll need need a privacy policy that addresses how Google is used, etc.
  • ORCID - Just looks like we need an account from which to create the App credentials needed.
  • FYI: The recent accessibility issues for Dataverse should all have fixes in the release on dev/stage, so checking them could be part of testing.

Plans

  • SSO
    • Finish deployment on stage
    • Support testing as needed
  • Fix #113 if possible
  • Matomo - investigate event-level tracking via tag manager, remove non-working google scripts
  • AnnoRep - explore round-trip, configure auto-start and log rotation
  • Ops
    • Clean out old corrupt test datasets
    • check missing globalidcreationdates and fix via /modifyRegistration or alternative
  • Dataverse
    • Track ADA guestbook branch and merge when working, Make PR for guestbook adding datasetversion fix, deploy to stage
    • Popup info accessibility - IQSS likes the recommendations from the source I linked to, so this can be implemented along those lines.
  • Drupal - v10 - review compatibility and start updates.
  • QDAS Previewer
    • Updates per request
    • Investigate writing aux file/previewing lower-sensitivity version and/or other write options
  • TBD: FRDR Security