User Testing: Round 2 - NWFWMD-IT/Hydrologic-Monitoring-Project GitHub Wiki

Scope

  • Users

    • Katie
    • Steve
    • Jacob
      • Experienced field user
      • New to app
  • Hardware

    • Use Katie and Steve's existing field laptops
      • Oldest machines; newer ones should work if these do
  • Locations

    • Test various locations with different configurations / conditions

Approach

  • Test in-office first

    • Catch obvious issues
    • Train field test users
    • Broaden scope of users who are testing
      • Not everyone will test in field
      • Include users who haven't participated in design
    • Test infrequent circumstances (e.g. DOA battery)
  • Test locations from easy > hard (roughly)

  • Intentionally introduce problems to see how we can handle them

    • E.g. battery runs out before submitting survey

Unit tests

Administrative configuration

  • Publishing services / maps
  • Installing apps on field devices
  • Sideloading basemaps

User configuration

  • Accessing apps
  • Connecting to portal
  • Going offline

Field maps

  • Find location
  • Launch Survey123 at current location

Locations

  • Prepare final data (e.g. Measuring Point / photo) for these locations

  • Types of locations

    • Test every section/capability of the survey
    • Include manual location(s)
    • Index (ADVM)
      • Currently visited by contractor
      • District will need to visit these sites for testing, between contractor visits
    • Dry surface water

List

ID Parameters Comments
002138 GWL Ott mini logger, battery dies
003340 GWL InSitu Logger
003341 GWL No logger
008434 Stage Sutron logger, needs stage corrections
008459 Stage usually dry
008484 Discharge + Stage + Rain
011313 Rain
011355 Stage WaterLog logger
012796 Groundwater + Conductivity + Temperature Keller logger
012899 Discharge + Stage + Velocity ADVM, will not be completing a discharge
012949 Groundwater + Conductivity + Temperature Keller logger

Preparation Checklist

  • Determine schedule for round 2 user testing: COMPLETE BY JANUARY 15
    • Preliminary testing with local locations (Steve) in December
    • Field testing for January 9 - 11
  • Determine users who will test apps / workflows
  • Determine locations for field testing
  • Review tickets that need to be complete before testing begins (UPDATE LIST AS NECESSARY BEFORE MEETING)
    • Data
      • #82 Measuring Point offsets
        • Complete for Round 2 test locations
      • #87 Measuring Point offsets comments
      • #109 Collect > load photos
      • #24 Finalize domain values
      • #36 Configure offline basemap
        • Current solution is sufficient for offline imagery for limited testing area
        • Focus on Leon and Wakulla counties
      • #114 Configure pop-ups/ service
      • #116 Sync enable feature service
    • Software Installation
      • #108
        • Upgrade field laptops to Windows 11
        • Test other hydro apps on Windows 11
          • To be verified during field testing
      • Use directions to install Field Maps / Survey123 on laptops (using WSA)
    • Documentation
      • #113 Round 2 testing instructions and tracking spreadsheet
  • #125 Develop / present user training Postponed until after Round 2
    • One hour
    • Recorded (video)

Field Testing Feedback

See Survey123 BetaTesting_Jan2024.docx for consolidated testing feedback

Hardware

Generally, field testing revealed issues with:

  • Performance
  • Battery life
  • Usability

Possible Solutions

  1. Run Survey123 on Windows (not WSA)
  • Advantages

    • Faster?
    • Lower battery consumption?
    • Easier to attach files
      • Survey can access Windows file system directly; don't need to transfer from Windows > WSA
    • Single device
  • Disadvantages

    • Can't launch survey from Field Maps
      • Field Maps must run in WSA
      • WSA Field Maps can't trigger Windows Survey123 to run
  • Questions

    • Can Survey123 for Windows work offline?
      • Yes, per Sarah from ad hoc test during 2024-01-18 meeting
  1. Switch to tablets
  • Advantages (per ad hoc field testing with phones)

    • Faster
    • Lower battery consumption
    • Field Maps > Survey123 integration seamless
    • Field Maps works / works better offline?
      • Recall hearing something in 2024-01-18 meeting about FM on WSA not working when offline (e.g. in flood plain and would have to walk back to vehicle to get connectivity)
    • Easier configuration
      • GIS: App configuration
      • IT: Avoid Windows 11 upgrade
      • Users: Avoid reconfiguring desktop after rebuild (and having to switch to new OS in general)
  • Disadvantages

    • Cost
      • Need new hardware
        • Note that tablet is probably cheaper than laptop
        • May migrate to tablets over time
          • Concerns about mixed environment during transition (e.g. some users on tablets, others on laptops)
    • May need multiple devices in-field
      • Some data loggers require Windows
      • Gradually replacing Windows-only data loggers with mobile compatible ones
  • Questions

    • How to get files from laptops (connected to older data loggers) to tablet for attaching to survey?