Methodology

Overview

User testing will be performed on the HUD of the game to ensure the player can intuitively understand the meanings of the elements, that the elements encompass everything the player needs to know (nothing more/less) and that the placement is natural and easy to look at/quickly understand during gameplay. This will be achieved both through giving player access to the game and questioning their understanding alongside A/B testing for HUD placement and design. This will allow us to both test their intuitive understanding as well as their perspectives on the design decisions and whether they are appropriate.

Justification

The HUD was chosen to be tested as it is an integral part of the users experience and understanding of the game and therefore it's fluency and integration into the game can make or break the users suspension of reality during play. As additional motivation, the game's emotional goals strive to ensure any struggles with the game from player ability/adaption and not from being unable to understand the mechanics of the game therefore this fluency is a must. Additionally by testing immediately following the integration of the elements any issues can be quickly identified to ensure any required changes/updates are made as soon as possible.

Testing Process

User Brief

Before testing the user will be given a brief description of the game and basic descriptions of the player functionality (i.e. the player is capable of attacking and getting hurt) alongside most of the controls. The test administrator will not explain the wound system, or dash cooldown or any other very specific implementation related information that we intend to test whether the player to be able to understand through simply playing the game and observing the feedback given through the HUD (this also includes the hotkey use of items).

User Tasks

The Users tasks will be broken up into two segments: testing the inherent understanding of HUD elements and feedback on HUD placement of element design.

For testing understanding the user will be given ten-to-twenty minutes to play the game and gain an understanding of what the elements are and what they do. During this time the tester will not speak to the player unless telling the player how to return to the game in the case of bugs interrupting play or reminding players of the control keybindings (as a tutorial is intended to be added to the game and therefore these keybindings will be consistently provided information to new players). After this, the user will be asked a series of questions to learn their interpretations of HUD elements, this will include:

• Asking users the meaning of each bar
• Asking them which elements are related to one another
• Asking them whether they felt any information was left out
• Whether they had trouble seeing the HUD when it was over the game area
• Whether it was confusing to have the bullets next to the items (i.e. does the spacing between elements make sense)

Secondarily the player will be shown a series of HUD illustrations and asked to determine which out of a series of options they prefer. They will be further prompted for extra explanations on why certain elements are preferred and whether their understanding of the elements changed depending on which graphic was used.

Illustrations Set #1: HUD Placement

Additionally Users will be asked to, if they have another way they would prefer to arrange it to explain the changes they would make to their choice and why.

Illustrations Set #2: Health Bar

Other elements do not have alternate options so users will be asked to, if they have a personal idea of what it should look like, to describe how they would change it.

Testing

Participant #1: Significant observations and feedback

Assumptions from play:

• Understood it was a health bar, got that it was tiered (colours changing) but didn't quite understand why
• Understood the dash cooldown bar completely
• Understood the item number meanings
• Understood that the number of rocks usable and reloadable were different and which image represented which
• Was confused by the word "left"
• Didn't see the reload message at first
• Understood the relations between elements
• Was hard to see the text elements of the bar on top of the screen

Feedback on HUD Options

• Preferred the HUD in the middle bottom and stacked
• Felt more in the game
• Felt easier to see at a glance
• Would makes words smaller/remove words
• Would stack health & dash bar (close)
• Would use images for bandages instead of numbers

Feedback on Health Bar Options

• Preferred separated bars of same size
• Easier to understand at a glance
• Easier to see at a glance
• Liked it always coloured (easier to see sections)

Other Elements

• Other than previously mentioned changes to items, there were no other comments

Participant #2: Significant observations and feedback

General Feedback based on questions asked

• Current HUD arrangement feels odd. 1 long bar - would make more sense (1 bar changes colour when lower - first option of health bar would make more sense)
• Left should not be there (bullets made sense) [second image]
• Rocks should be round maybe - not flat at the bottom
• Prefer to have HUD arranged at the top in one row (option one of HUD placement)
• Maybe instead of bandage - use potion image instead?
• Dash cooldown might not be necessary (should use it for an ability). Intuitively can tell that dash can't be used all the time when dash can't be used twice in a row during game when shift is clicked.

Participant #3: Significant observations and feedback

Assumptions from play

• Understood the health bar represented health
• Assumed that the changing levels of the health bar that changed the player state (pointed directly to the change in movement speed)
• Did not understand the regeneration gate aspect of the tiered health bar
• Understood the rocks represented the amount you currently have and the current amount you are able to throw
• Understood the Band-Aids represented the number of healing items
• Understood the number of coins represented how many coins you had
• Understood the dash bar showed the players dashing cooldown

HUD options

• Preferred the bar at the bottom
• Liked the dash and health bars stacked together
• Would have liked if the rocks were separated from the items (makes a rock took like an item)
• Felt the dash cooldown as it was so short didn't really need a bar though
• Preferred if the item numbers didn't have the words and only the images (didn't need it and took up more of the screen)

Health Bar Options

• For the states said they would prefer either thicker separations between the bars for different health states or the colours to always be different
• Liked the separated bars more than a single bar
• Felt all designs get used and so none of them were 'bad'

Participant #4: Significant observations and feedback

• Issues with health system
  • unconfident in identifying health bars / something relevant to health
  • After explaining the health system, was still unsure what 'wounds meant'
• Didn't understand the dash system / blue bar
• Has issues with the ammo system
  • Though rocks were an item, was unsure of purpose until prompted about ranged attack.
  • Doesn't understand the ammo vs loaded counters.
  • After firing all ammo, didn't notice the reload message.
• Prefers HUD elements being center bottom
  • lines up with the edge of screen
  • More compact

Participant #5: Significant observations and feedback

• Doesn't understand health system
  • Assumes multiple bars mean the player "bleeds out over time"
  • "Why does health go up to 5 when I get hit?" - doesn't understand the concept of wounds
• Blue bar as "food / water"
• Correctly assumes that bandages heal health
• No idea on ammo system
  • Not a standard magazine count / number of bullets loaded. (eg 5/2 = 5 sets left, 2 bullets remaining in current set)
  • Observed participant struggling in knowing how many bullets until they need to reload, prefers number instead.
• "Everything left out"
• No idea on the items, since they're not in the game.
• Can't dodge enemy bullets on some levels where ranged enemies are too close to each other

Participant #6/#7: Significant observations and feedback

• Stumbles onto the Space, Enter, 1 and Left Shift hotkeys.
• Assumes level title may be part of the debug screen
• Correctly identified health bar
• Correctly identified coin counter as currency system, something to purchase items with
• Projectile meaning isn't clear
• #6: No preference on the layout of the screen, "however the left-side feels normal"
• #7: Prefers a dedicated border for the HUD, as to not overlap with the game terrain, especially if the area goes dark.

Participant #8: Significant observations and feedback

• Notices that you can't close / dismiss the info dialogue box (with prologue)
• Correctly identifies all elements to an extent
  • Blue bar as a mana or relevent to special abilities
  • Prefers having health / wound counter under the green bars
• Correctly uses items with the number hotkeys
• Ammo system makes sense when you stare at it for a bit
• No preference stated, although explains that it seems fairly clustered and hard to quickly recognise from the left hand bar.
• Ammo counter relocated to right hand of screen
• Proposes Blue bar hidden normal, then a small blue bar ontop of player sprite to show use of ability and its cooldown

Participant #9: Significant observations and feedback

• Doesn't understand the health system
  • Thinks health should stay the same when player is hit
  • wounds labeled as lives
  • Visual indicator is not intuitive, doesn't understand the penalty between getting hit by a big enemy vs small enemy
• Items should be labeled with keybind, ie. [1] (BANDAGE IMAGE) (Qty)
• No preference on the interface, although thinks health, items and ammo should be spaced out.

Evaluation of Results

Common Issues & General Options for improvement:

All players preferred a separated design along the x axis as it allowed them to more quickly see the HUD and read what was in it. Two out of three preferred the bottom placement. One user also would have preferred if the rocks were separated from the items as they are shown the same.

• Moving the HUD to the bottom of the screen
• Having the items separated from the health bar and cooldown bar
• Having the projectile counts separated from the rest of the items

Most users felt the dash bar was unnecessary due to the very short dash cooldown.

• Remove the dash cooldown bar
• Repurpose the dash cooldown bar (e.g. for abilities) -> would require design changes to indicate it is for ability
• Increase the dash cooldown

Most users understood the separation of the health bar was to indicate different states but no users understood the regeneration gating.

• Including a tutorial within the game to explain the regeneration
• Including an animation / muted colours to show which bars can regenerate

Didn't reach a consensus on preference in terms of the health bar, though most users wanted more separation between the states, either through constant colour differences or thicker barriers between state bars. Most users did prefer separate bars, but one did prefer the single version.

• Switching to constantly coloured states or thicker barriers
• Looking into different health bar arrangements and animations to better indicate state differences
• Getting rid of the tiered arrangement and using same length bars

Many users thought some of the words on the HUD were unnecessary and cluttering the screen.

• Removing the name identifiers for number indications that already have sprites
• Removing the word left

A user had a problem with visibility for the HUD as it was over the game area (as well as problems seeing the reload message).

• Adding semi-transparent backgrounds to the HUD and reload message
• Moving the reload message to appear where the images for the rocks you can currently throw are

Currently Actionable Improvements:

• Moving the HUD to the bottom of the screen
• Having the items separated from the health bar and cooldown bar
• Removing the name identifiers for number indications that already have sprites
• Removing the word left

For the health bar, due to the lack of consensus and need to produce animations for the alternate options to the original design given, more iteration, and likely testing, will need to be conducted in the future to ensure an effective representation.

Additionally all other changes will be considered in the next sprint when more time is available to incorporate said changes.

Evaluation of Methods

Overall the use of options (i.e. A/B testing) was very appropriate for giving players options on how to design. For arrangement of HUD as an alternate method utilising a paper prototype were users are able to arrange their own HUD could have been beneficial in order to explore more player driven arrangements. Additionally, though the users assumptions feedback was more useful it could have likely been better to give player more time as due to the long regeneration delay most players didn't even allow the player to heal in order to register the regeneration gating. Overall, though it was a generally successful, users could have been giving better language to communicate their feelings and in future more testing will strive to incorporate interactable/freeform elements.