Purpose

Guides/outfitters

To test the temp outfitter login flow, application management screens and review automated emails.

Special use admin

To test login, new features in the e-Permit admin screens, review the content from a few temp outfitter permit sections, and review automated emails.

Test session details

• 3 guide/outfitter testers scheduled with 4 guides
• 3 total guide/outfitter testing sessions performed with 4 guides
• 2 special use admin testers scheduled
• 2 special use admin testing sessions performed

Results and insights

Overall Login.gov flow with guides/outfitters

Results: somewhat successful

• All users expected to go to some sort of account creation or signing process when clicking the “apply” button from the non logged in landing page.
• Most users were a little disoriented when getting to the initial login.gov screens. The most common comment was that the colors and design looked too different.
• Two users suggested adding the Forest Service logo.
• All users commented that, even though it seemed disjointed, they felt confident that they were directed to an official site.
• All users felt that overall, the flow was fine.
• One user thought the sign up process was a bit long and having to do the two factor authentication every time they logged in seemed like it might become annoying.
• One user thought it was really fast and simple.
• All users thought that being able to access and manage applications after submitting was worth any additional time or steps.
• All users quickly and easily figured out how to log out.
• Two users brought up a concern about who the Login.gov account belonged to, the user or the organization. Are they the same? What if that person quits and no one has their email access to do a reset? Is there an option for an organization account that multiple employees have access?

Recommendations:

• Look into adding the Forest Service logo or name to the Login.gov pages.
• The organization vs. individual Login.gov account issue can be solved by the organization when setting up the account. Making sure to use an email account additional users have access to, communicating who signed up for the account and for which applications etc. Consider creating content to put in the learn more pages to address the possible methods for creating and managing a Login.gov account.

Business name optional with guides/outfitters

Results: successful

• All users understood that business name was optional.
• One user mentioned that it seemed odd that an individual could opt to not enter a business but that an organization had to enter a person’s name and may find it confusing as to what name to enter. The applicant may be an employee hired just to fill out the application but not participate in trips or trip planning.

Recommendations:

• Discuss change the name label to something that indicate the person that should be listed. “Official permit holder name” etc.

Client charges feedback with guides/outfitters

Results: somewhat successful

• Most users were still concerned with this section and wording and were unsure what to put and what the purpose was.
• All SUAs mentioned that they sometimes didn’t use this info but, based on other details of the application, may look at the costs and what they covered.
• All SUAs thought it would be better to have applicants provide more info rather than risk them not providing enough.
• All SUAs thought that giving them info suggesting they provide info including, scholarships, equipment fees etc. would be useful and get to the point of what they’re looking for.
• One SUA suggested mentioning that the cost was based on a flat fee, not on what they charged, to try and set the expectation with applicants that SUAs were not making application approval decisions based on what organizations charged or how they structured their charges.

Recommendations:

• Add additional copy, “Provide a description of client charges and fees and what they cover (account for any additional fees, scholarships, etc.) What you enter will not change what you will be charged. Charges for a temporary outfitter permit are based on a flat rate.” and provide a link to the cost table in a new tab.

Acknowledgement of risk feedback with guides/outfitters

Results: not successful

• All users found the acknowledgment of risk section confusing and possibly too easy to skip.
• Most users thought that, after seeing the “optional” label, most applicants would think they could skip this section and just move on without reading the explanation that it was required for some users.

Recommendations:

• Under the acknowledgment of risk form, have an unchecked box that says, “We require our clients to sign and acknowledgment of risk for.” When checked the upload interface is presented and required.

Cancel application with guides/outfitters

Results: successful

• All users quickly and easily found the cancel button.
• One user thought it might be too easy for users to accidentally cancel an application but when they saw the confirmation/warning pop up they thought that would sufficiently solve the issue.

Recommendations:

• None

Application progress indicator with guides/outfitters

Results: successful

• All users noticed the file upload progress animation.
• All users understood the purpose and noticed that it was counting down files uploaded.

Recommendations:

• None

Confirmation page changes with guides/outfitters

Results: successful

• All users thought the new information in the “Next Steps” section of the confirmation screen looked good, would be helpful and was accurate.

Recommendations:

• None

Automated emails with guides/outfitters

Results: successful

• For the submission confirmation email one guide asked if we could add some info about when they could expect to hear about next steps.
• “Seems clear and makes sense to me. I dream of getting this much info when I apply for permits.”

Recommendations:

• None

Overall login flow with special use admins

Results: successful

• All testers were familiar with logging into e-Auth and expected that to be the method for logging into e-Permits
• All users were able to figure out how to log out quickly

Recommendations:

• None

Application types with special use admins

Results: successful

• All users thought the application type filter list covered all application typed they would want and need to view.
• One user mentioned that the special use admin internal labels were sightly different and sent over the list of labels they use.
  • Accepted, as a formal application (passed 2-level screening)
  • Decision made to authorize applied for use
  • Held pending decision on appeal
  • Application denied as a result of decision or applicant request
  • Processing on hold at discretion of authorizing officer
  • Application withdrawn by request of applicant

Recommendations:

• Consider using the application type labels currently used by SUAs.

Temp outfitter detail page with special use admins

Results: successful

• All users thought the detail page was organized well and had the information displayed in a useful way.
• All users thought the revision history was clear and easy to understand. Each user knew what it was communicating immediately.

Recommendations:

• None

Unhold/review with special use admins

Results: successful

• All users figured out how to remove the hold status quickly.
• All users thought the status, “review” made sense as a state after removing the hold status.
• One user noticed that when seeing applications in “Hold” and “Review” status that the applications that were in “Pending” looked confusing, not having a status. They suggested always show the status of an application on the list page.

Recommendations:

• None

Cancelling application with special use admins

Results: successful

• All users were able to figure out how to cancel an application quickly.
• All users thought the alert window was sufficient to prevent accidental cancelation.
• One user mentioned that SUAs refer to canceling an application as “withdrawn”.
• One user suggested being able to add a message to the applicant about why it was being cancelled.

Recommendations:

• None. Cancellation message doesn’t seem necessary at this point. If a SUA does cancels an application it will be due to an applicant request.

Automated emails with special use admins

Results: successful

• All users thought it was difficult to figure out what action the email was referencing.
• One user asked it the submit alert email could reference the amount of time until a response is required.
• One user asked if the cancel email could make it clear it was canceled by applicant.

Recommendations:

• Easier quick reference to topic. new submission in subject line.
• Add info about time to reply to submission alert email.