7 19 2021 Tech Team Report - QualitativeDataRepository/TechnicalTeam GitHub Wiki

7-19-2021

Logged Tasks

                            Date             Task Hours (Main) Hours (EOLS) Hours (PII)
12-Jul-2021 Report, meeting, check metrics re: BySubject counts 1
13-Jul-2021 Update google libs for security, tika, test archiving, fix prod DV banner 1
14-Jul-2021 Coord re: AnnoRep api use 1
15-Jul-2021 Investigate D9 issues, update composer from 1 to 2.1.3 on dev, install upgrade status module, remove migrate, entity_update, devel_entity_updates modules, update versions of honeypot, login_history, private_files_download_permissions, sitemap, try updates of others, test qdr modules, update qdr_theme to be D9 compatible, fix shib_auth issues, remove ShibCache class, apply me_redirect patch, scan for remaining issues. 7

Summary

Operations:

  • Checked reported metrics, e.g. the collection bySubject plot which includes 14 entries when there are only 7 collections. It looks like DV adds the subject of any subcollection to the main collection (and doesn't remove it if you then delete a test subcollection). So the plot is correct given the db entries and we may want to discuss whether to fix the db (e.g. remove at least the test subjects) and/or look into updating Dataverse or the plot (how should parent collection subjects be handled?)
  • Fixed DV banner that had a broken link due to escaped quote chars.
  • Figured out how to update composer and updated it on dev (from v1->v2.1.3, faster and potentially needed for D9) (Note older reports since last fall incorrectly point to a v2->3 update needed - v3 doesn't yet exist).

Dataverse:

  • Basic testing with new library versions being udpated for security reasons (google, tika)

Anno-Rep

  • Answered questions re: Dataverse API

Drupal:

  • Started digging into update issues:
    • installed latest Update Status module which lists open issues in various modules
    • removed some unused problematic modules e.g. the v6-7 migrate module
    • installed updated versions of D9-compatible modules (list in table above)
    • tried/reverted updates to some modules re: ldap - which either have D9-specific requirements or dependencies where multiple modules must be updated together
    • investigated yet-to-be-merged patches that should provide D9-compatibility - successfully applied such a patch to the me_redirect module we use
    • made minor update required to make qdr_theme D9 compatible
    • ran checks on qdr_* modules - no updates required :-)
    • ran checks on shib_auth module - removed unused ShibCache class, adapted patches for stock shib_auth to out modified version to fix other issues
    • investigated db version issue

Discussion

  • With the work last week, I think I have a better sense that the Drupal v9 upgrade won't be huge and I made a good start working through the updates required. It is good news that the qdr_* didn't require changes and I was able to adapt shib_auth. The only other module involved in our SSO is the ldap module and, while I wasn't able to upgrade that successfully yet, I don't think the changes needed will be too complex. For the remaining modules, most have patches that I can try applying and there are only one or two where the maintainers haven't done much. For those, the status checking tools appear to give a good description of what's needed - often simple 'replace use of call 'x' with call 'y'.
  • The one coordination issue from this is that it appears D9 expects mysql 5.7.8+ (8.0 is current) and we are at 5.6. While there is a module that should allow use of 5.6 with D9, 5.6. is nearing end of life, so it probably makes more sense to just update. (I realize now we talked about getting Dataverse to postgres 13 as well but haven't kept it on Seba's todo list. That's less urgent as there aren't >9.6 dependencies in DV yet, the update was just to avoid the EOL issue.) If updating mysql can't be done soon, I can look into trying the 5.6 module to start.

Plans

  • Anno-Rep work
    • Answer open question re metadata api
  • Drupal 9 - continue going through upgrade processes
  • Dataverse
    • still want to investigate the guestbook responses re version info not being included.