Introduction

We will be using Think Out Loud and Time-on-Task methods to evaluate both the usability of our system and more importantly, establish a baseline for game balancing.

User testing is important because it helps identify usability issues and improves overall user satisfaction. By testing with real people, we can see how people interact with the system and catch potential design errors and bugs. This process ultimately leads to a better, more efficient user experience.

All three Mini-Games use a medal based system that correspond to in game rewards. A player must also get a bronze medal to make progress in the main game and unlock new quests to defeat the boss. As such, we need the bronze medal to be achievable, without being trivial. As Silver and Gold medals are only for extra benefit to the player in the main game, we have decided to make these much more challenging.

Methodology

We will trial 7 people to play each of our mini-games 3 times, all three trials will be used for a time-on-task analysis with the first trial also being used for a think out loud analysis.

Think Out Loud is a user testing technique in which participants verbalise their thoughts (or 'think out loud'). This is a good method to highlight the usability of an interface and observe how a user interacts with it. Specifically, it highlights how users navigate the system and highlights potenital issues that users face when they first use a system. This is valuable to make the system more user friendly. We have decided to use think out loud along with general observation the first time a participant plays each of the Mini-Game's.

Time-on-Task measures how long it takes a user to complete a specific task. Usually this is meant to help evaluate the learnability of a system. It can be hard to establish a baseline for Time on Task which is why we have choosen to use it in conjunction with think out loud. In our case we will use it to measure the learnability of the mini-game as well as track their score improvement which we will use to help with game balancing. We have decided to use a variation of time on task to time the duration of each mini-game played by the participant and record the score. As games generally has a small learning curve, we will ask the participants to play each game 3 times in total.

We have also decided to ask a few questions to the participants just for some extra feedback.

User Breakdown

• None had played any of the mini-games before
• age range of 20-21 years old
• mix of genders (3 women and 4 men)

Heirarchy Task Analysis

This diagram shows the ideal flow of each of the user test cases.

We will read the following script to the user:

"I am going to time and record you as you play our mini-games in the Attack on Animals game. This is not testing you but rather the system. I want you to navigate to the Mini-Game's menu and play each minigame. I want you to think out loud your decision making process as you complete this task. I will record how long you play the game and your score. Once you have played each mini-game and navigated back to the Main Menu screen, I will get you to play each Mini-game twice more (not thinking out load) in which I will record your time and score for each mini-game play through. I will finally ask you some questions about your experience."

Questions to ask after all of the trials: For each game:

1. Was the game easy to understand?
2. Do you agree with the current medal system?
3. Did you enjoy playing the game?

Results

Trial Raw Results

Think Out Loud Results Summary

• All participants easily navigated to the minigame menu screen
• All participants did not anticipate the game to start as soon as they loaded the game
• 1 participant tried to play the birdie dash game by pressing the trackpad
• 4 participants did not expect to survive touching the pipes in birdie dash but managed to recover quickly
• 3 participants did not realise collected coins increased score in birdie dash (over entering pipes)
• 2 participants highlighted the flappy bird button name on the Mini-Game's menu screen
• 4 participants tried to play the Maze mini-game with arrow keys and when that did not work they switched to WASD
• All participants did not track their health decrease in the Maze Game
• A participant noted that the health decrease form the angler fish did not feel dramatic

Questions Summary

• Question 1
  • All participants said the games were easy to understand and play (after 3 trials)
• Question 2
  • Snake
    • Some participants said the gold and silver for Snake should be decreased, and bronze increased
  • Birdie Dash
    • Most participants were happy with the scoring with a couple participants suggesting higher thresholds for each
  • Maze Game
    • Almost everyone, agreed with the maze scoring system, one person suggested lowering the gold medal threashold
• Question 3
  • All participants said that they enjoyed playing all of the games

Discussion of Results

Positives

We will start out by listing some positives. All participants said the games were fun to play and easy to understand after 3 trials. Each participant generally improved theur game scores over each trial suggesting that the games are easy to learn. Each mini-game trial did not exceed 3 minutes which is an adequate time for a mini-game (not too long that the player is bored and not too short that there is not more to do). All participants easily navigated the system and were able to find out how to play each of the mini-games through the mini-games menu easily.

Potential Issues

One of the biggest issues highlighted through user testing included the immediate start to a mini-game. When the participants would load the game, they did not expect it to start as fast as it did. This is especially evident with the first user test of birdie dash where all but 1 participant immediately died (all highlighted in purple in the table). This is most evident in Birdie Dash over the other games as in this game, the player dies faster than they would the other games wihtout any input.

Despite almost all participants saying it was easy to understand how to play birdie dash, there was some confusion through the think out loud results. Many participants were confused when they did not die touching the pipes and that their score only increased when they collected coins. This is due to similarity with a well known game flappy bird. In saying this, all the participants quickly learnt from their expierience and were easily able to adapt in future trials. Furthermore, one of the users first intinct to play the game was to use trackpad input which had not been implemented.

It was also highlighted by two of the participants that the button to enter birdie dash from the Minigames Menu said "flappy bird" which is incorrect/outdated.

Issues related to the maze game highlighted through observation and think out loud methods included issues with key input and lack of attention to the players health. FOur participant intially tried to play the game with arrow keys which was consistent with birdie dash and snake, almost none of the participants payed any attention to the health of the player with one participant noting that being hit by the angler fish did not feel dramatic for losing 20 health.

Code Improvments

Immediately after user testing, the 'flappy bird' button on the Minigame menu screen was changed to 'birdie dash' to reduce any confusion with the flappy bird game.

The biggest improvement we made to all the mini-games was the implementation of a pre game pop up, each pop up now outlines the rules and controls for each minigame reducing any confusion on how to play the game. Furthermore, this also gives the game a bit more of a buffer from loading in and the game actually starting which should give the participants a bit more anticipation before the game actually starts reducing the results seen in birdie dash trials. Furthermore, we have also added functionality with the left click to play birdie dash which may be more intuitive to some players.

In the maze game, we have added particle effects for when the player takes damage from an enemy fish. This makes it more dramatic when the players health decreases and acts as a small reminder for the player to pay attention to their health status. We have also added functionality for the arrow keys to play the game which may be more intuitive for some players.

From user testing results we have decided to update the medal thresholds as follows:

We believe these thresholds make the aquirement of a bronze medal very achieveable (but not just a given) but also leaves the gold medal quite a challenge for players who want those better in game rewards. At least one participate was able to get a bronze medal for each game as per the new medal thresholds. Accounting for a small learning curve, (the participants have only played the game 3 times), we believe that the new gold medal threshold is difficult but achieveable.

Evaluation and Future Testing

This usetr testing effectlively helped us identify some potential issues and usability flaws in our system which we have fixed. We were able to identify how users interact with our system and adjust our controls and game mechanics accordingly.

Ideally we would like to have some more user testing to effectively evaluate if the pop ups can help both understand the game and anticipate the game starting. Furthermore, more trials for user testing may be helpful to evaluate the achievability of a gold medal with the new threshold as none of the participants from these trials were able to get a gold medal.