The purpose of this study is to understand user behaviors and expectations related to the Printing and Downloading PDFs on page 5 of the "create project" form and the "my projects" page of the TDM Calculator.

NOTE: Due to circumstances discussed in the below documentation, this research required 2 rounds of testing. Therefore, readers will find multiple versions of documents distinguished by "round 1" and "round 2." These labels outline the differences between the first and second round of testing.

1. Round 1 of Testing
   1. Round 1 Prerequisites
      1. Test Plan
   2. Round 1 Preparation
      1. Website Overview Page
      2. Target Users
      3. Sample Project Landing Pages
   3. Round 1 Recruitment
      1. End Users
   4. Round 1 Interview Logistics
      1. Interview Materials
      2. Tool Setup
      3. UXR Training
   5. Round 1 Results
      1. Interview Records
      2. Data Analysis
      3. Decision Record
2. Round 2 of Testing
   1. Round 2 Prerequisites
      1. Test Plan
   2. Round 2 Preparation
      1. Website Overview Page
      2. Target Users
      3. Sample Project Landing Pages
   3. Round 2 Recruitment
      1. End Users
   4. Round 2 Interview Logistics
      1. Interview Materials
      2. Tool Setup
      3. UXR Training
   5. Round 2 Results
      1. Interview Records
      2. Data Analysis
      3. Findings
   6. Additional Resources
      1. Guide to Usability Testing
      2. Referenced Resources
      3. Contact Us

• Created test plan for the Printing/Downloading PDFs.
  • RP011: Printing/Downloading PDFs Research Plan
    • Referenced prior usability testing RP004: Reset Button Usability Test to reuse and update relevant testing materials and tools.
    • Discussed recruitment with PM.
    • Created RP011: Participant Tracker - De-Identified - Round 1 from the templates used in previous rounds of testing.

• Reused the overview webpage [webpage needs update], which presents the TDM Calculator project.
  • Introduces LA TDM Calculator Project with details about:
    • TDM and TDM Calculator
    • TDM Program Update
    • Current status of TDM Calculator
    • Resources from LADOT

• Utilized HfLA community to represent users for usability testing.
  • DR Research: Determining who to recruit for usability testing

• Reused the basic landing page structure from RP004: Reset Button Testing to create the RP011: Printing and Downloading PDFs Landing Page.

• Sent out a recruitment message via Slack that allowed participants to sign up for usability testing time slot via a Google Calendar link.
• Recorded participant appointments on the RP011: TDM Participant Tracker - De-Identified - Round 1
  • Full participant tracker for round 1 in the PII drive contains information about each participant’s name, interviewer, testing date and time, and landing page assignment. Contact research lead for access if needed.

• Referenced prior usability testing RP002: Usability Testing 2.
• Created RP011: Printing/Downloading PDFs Testing Script
  • Interview script for 1:1 interviews includes intro, site testing, and closing questions, which helps UXR to follow the same procedures for all interview sessions and make notes accordingly as interviews proceed. A copy of this template is created for each interview and the link placed into the Usability Testing Session Table.
• Used the TDM background image for each interview session.

• Used HfLA Slack channels for recruitment.
• Used the TDM-UX@Hackforla.org email address to set up a Google Calendar event that allowed participants to select the time slot that worked best for them.
• Used HfLA Zoom account to conduct and record interviews.
• Used Zoom's built-in transcription feature to transcribe recordings.

• UXR leads reviewed RP002: TDM UXR Team Lead Responsibilities for Usability Testing to ensure consistency between previous testing procedures and the My Projects page testing.
• Reviewed TDM Calculator: Researcher Instructions for Usability Testing to ensure all interview procedures follow the same format and to address any issues with the process before conducting live interviews.
• Reviewed Guide: How to Use Zoom for User Testing and Guide: How to use Google Calendar for User Testing to provide researchers instructions for accessing the tools and materials needed to conduct interviews.
• Reviewed TDM Calculator: Researcher Instructions for Affinity Diagramming to ensure all researchers followed the same process for analyzing transcripts and contributing to the affinity diagram.

• Saved round 1 interview records in the PII drive. Contact research lead for access if needed.

• Created RP011: Data Analysis Spreadsheet - Round 1

• Due to bias being introduced during the testing in round 1, the following decision was made by the team DR Re‐doing RP011 usability testing. To maintain efficiency on testing and eliminate bias in round 2 of testing, the UXR team completed the following: rewrote testing scripts, recruited new interviewees, and completed the interviews in round 2 of testing.

• Maintained and referenced the same testing plan that was created in round 1 of this research.
  • RP011: Printing/Downloading PDFs Research Plan
    • Discussed recruitment with PM. Decision to utilize all new sample of users for round 2 of testing was agreed upon.
    • Created RP011: Participant Tracker - De-Identified - Round 2 from the templates used in previous rounds of testing.

• Reused the overview webpage [webpage needs update], which presents the TDM Calculator project.
  • Introduces LA TDM Calculator Project with details about:
    • TDM and TDM Calculator
    • TDM Program Update
    • Current status of TDM Calculator
    • Resources from LADOT

• Utilized HfLA community to represent users for usability testing. No users that were interviewed in round 1 of testing could be re-interviewed for round 2 of testing so as to not introduce any bias to the test results.
  • DR Research: Determining who to recruit for usability testing

• Reused the basic landing page structure round 1 of testing while making minor edits to follow the updated script in round 2, creating RP11: Landing Page.

• Sent out the same recruitment message as round 1 of testing via Slack that allowed participants to sign up for usability testing time slot via a Google Calendar link.
• Recorded participant appointments on the RP011: TDM Participant Tracker - De-Identified - Round 2
  • Full participant tracker for round 2 in the PII drive contains information about each participant’s name, interviewer, testing date and time, and landing page assignment. Contact research lead for access if needed.

• Referenced prior usability testing RP002: Usability Testing 2.
• Reused RP011: Printing/Downloading PDFs Testing Script from round 1 of testing
  • Interview script for 1:1 interviews includes intro, site testing, and closing questions, which helps UXR to follow the same procedures for all interview sessions and make notes accordingly as interviews proceed. A copy of this template is created for each interview and the link placed into the Usability Testing Session Table.
• Used the TDM background image for each interview session.

• Used HfLA Slack channels for recruitment.
• Used the TDM-UX@Hackforla.org email address to set up a Google Calendar event that allowed participants to select the time slot that worked best for them.
• Used HfLA Zoom account to conduct and record interviews.
• Used Zoom's built-in transcription feature to transcribe recordings.

• UXR leads reviewed RP002: TDM UXR Team Lead Responsibilities for Usability Testing to ensure consistency between previous testing procedures and the My Projects page testing.
• Reviewed TDM Calculator: Researcher Instructions for Usability Testing to ensure all interview procedures follow the same format and to address any issues with the process before conducting live interviews.
• Reviewed Guide: How to Use Zoom for User Testing and Guide: How to use Google Calendar for User Testing to provide researchers instructions for accessing the tools and materials needed to conduct interviews.
• Reviewed TDM Calculator: Researcher Instructions for Affinity Diagramming to ensure all researchers followed the same process for analyzing transcripts and contributing to the affinity diagram.

• Saved round 2 interview records in the PII drive. Contact research lead for access if needed.

• Conducted the initial organization of the quantitative data in the RP011: Data Analysis Spreadsheet - Round 2
• Utilized FigJam to analyze data:
  • Created a RP011: Sticky Note Grid and organized participant responses to the questions.
  • Created RP011: Affinity Diagram to further code interview responses based on trends and themes in the responses.

• Created RP011: TDM Calculator UXR Presentation to discuss findings and potential solutions with stakeholders.

• GitHub Issue #1662 - EPIC: Printing/Downloading PDFs Usability Testing
• GitHub Wiki page for all prior TDM Calculator usability testing
• TDM Calculator Research Plan Template
• TDM Calculator Participant Tracker Template
• TDM Calculator UXR Presentation Template

• Our Slack channel: #tdm-ux.

Return to Contents