8 7 2023 Tech Team Report - QualitativeDataRepository/TechnicalTeam GitHub Wiki

8-7-2023

Logged Tasks

                            Date             Task Hours (Main) Hours (EOLS) Hours (PII) Hours (QDAS)
31-Jul-2023 Reporting, mtg, add sso=true flag in Drupal/DV to handle more out-of-sync cases, add user agent check for monitoring, move all new code to qdr_oidc_sso module, remove custom oidc redirect module, remove hardcoded redirect in DV, remove unused LogoutController, investigate old Drupal sec notices (in theme dev areas), debug/fix Nagios monitoring 7
1-Aug-2023 Coord re: GBAR UI/API settings, investigate Google, ORCID failures, delete partial ldap entry for Google, investigate potential to get error in Drupal, explore parse error/non-JSON response from ORCID. 3
2-Aug-2023 Coord re: GBAR, update email html structure, investigate error w/Google when no local account, add code to provide a clearer custom message, investigate/fix select_or_other issue/warning (see https://www.drupal.org/project/select_or_other/issues/3373641), update matomo module for sec. in dev/oidc branches. 4
3-Aug-2023 Investigate ORCID failures, try /userinfo for email (never provided), return to using /read_public scope, fix URL causing parse issues, explore ORCID/Google mapping issues preventing new accounts, find partial fix (no reg form entries). 6
4-Aug-2023 Investigate select options in keycloak profile, and select_or_other possibilities, merge 5.14, re-merge hotfixes, fix prod banner. 5

Operations

  • Removed extra quote marks in prod banner that broke the link

SSO

  • Add an ?sso=true param to support resync
  • Add user-agent check in passive Dataverse login to support monitoring - monitor now skips passive login check (which would fail) and tries to load the main DV page as before.
  • Refactor all SSO code to allow use of standard OIDC/Keycloak modules and keep all customizations in the qdr_* modules.
  • Replaced hardcoded redirect to DV front page to dynamically redirect to current page (hardcoded link was used to get the mechanism working)
  • Remove unused Logout controller from qdr_oidc_sso module, continue clean-up
  • Investigate Google failure for new account. Find mechanism to succeed in Keycloak and fail in Drupal where a clearer message can be created.
  • Investigate ORCID failure/fix user info retrieval to support existing users
  • Explore supporting Google/ORCID use for new account. Find settings to allow creating partial LDAP entry and ways to add registration questions in Keycloak. (see discussion)
  • Investigate/fix backward-compatibility issue with select_or_other module used in registration form (to allow entry of text after selecting 'other' for things like department).
  • Investigate whether Keycloak profile can support select or other (possibly with custom plugin code).

Drupal

  • Investigate auto-reported sec. issues in QDR theme, merged fix for one. (These are not issues with deployed code, just with some of the code-generation tools linked in by Schema.)
  • Update matomo module for sec issue on dev/oidc branches

Dataverse

  • Coord re: Guestbook-at-request functionality, fix email html/structure issues.
  • Merge 5.14 code, along with hotfixes to DDI export/tabular data tags

AnnoRep

Discussion

  • SSO for current accounts should be working for local and Google or ORCID accounts. ~Final task here is to check registration and terms-and-conditions check still work in all cases
  • For using Google/ORCID for new accounts, there are ~three current options:
    1. Fail, but with a useful message in Drupal telling people they have to register first. I think this is ~working except for wordsmithing the message and perhaps adding some notes about adding a local password (the reg form requires it but you won't need it if you do ORCID or Google login after that. Could either add a note or try to hide it, e.g. a 'will use social login' button that generates a random local password without showing the password input on the form).
    2. ~Succeed and require users to fill out reg form in Drupal. For this, I have been able to get Google and ORCID to create an LDAP entry with email/name info which is enough to get Drupal to allow login - currently with warnings about missing info like organization, etc. If, instead of showing warnings, users were redirected to the ~reg form (same questions except for name/email/password and different workflow (account exists but if they say no to being 18 or in eligible country they get blocked or deleted), this process would work.
    3. Succeed with Keycloak asking all additional questions from reg form (and possibly about accepting the terms-and-conditions). Keycloak has the flexibility to let an admin add additional fields to the user profile so nominally the reg form questions could be asked by Keycloak. It looks like text input as well as selecting from a fixed list can be handled. What may not exist yet are the ability to refuse to create an account if someone answers no, or doesn't check a box for a question like 'I am 18 or older', or to allow a select-or-other capability (select from a list and if selecting other to then have a text type-in). It is possible that these could be developed as there is a UI plugin interface.
  • My guess is that the last option is the best long-term option as it centralizes everything in Keycloak where we'd probably need less custom code and should be able to keep things consistent with MFA, etc. However, for the short-term going with not allowing account creation from Keycloak (option 1 - fastest to finish) or updating the reg process in Drupal to handle this case (option 2) will be faster and less risky.

Plans

  • SSO - continue to explore/build OIDC options
    • Finalize handling for new accounts from Google/ORCID
    • Cleanup old Shib code, document, verify deploy from github
    • Start process to get formal Google/ORCID production creds for our app (needed to go beyond a few test users)
  • Fix #115 if possible
  • Matomo - investigate event-level tracking via tag manager, remove non-working google scripts
  • AnnoRep - explore round-trip, configure auto-start and log rotation
  • Ops
    • Clean out old corrupt test datasets
    • check missing globalidcreationdates and fix via /modifyRegistration or alternative
  • Dataverse
    • Track ADA guestbook branch and merge when working, Make PR for guestbook adding datasetversion fix, deploy to stage
    • Popup info accessibility - IQSS likes the recommendations from the source I linked to, so this can be implemented along those lines.
  • Drupal - v10 - review compatibility and start updates.
  • QDAS Previewer
    • Updates per request
    • Investigate writing aux file/previewing lower-sensitivity version and/or other write options
  • TBD: FRDR Security