Methodology

Overview

User testing will be performed on the MVP implementation of the game in order to evaluate the basic functionality of the player character including; movement controls, keymapping, player statistics and general combat. In order to achieve this, potential users will be given access to the current build of the game with a test administrator observing. The user will be given a list of activities to perform with little description, before being asked to think aloud, describing all decisions taken when attempting to complete the activities. This will test the players understanding of the game from its presentation and a basic guide of what is possible. During these tasks the test administrator will not give the player any indication as to whether the player is taking the correct actions, only interrupting if the user becomes entirely stuck, reaches a point where they are stuck due to features that are not yet implemented (e.g. player death) or after completion to ask leading questions about specific actions the user took that were highly unexpected in order to gain a deeper understanding as to what drove said action. Following the tasks the user will be asked some broad questions about specific aspects of the game which could not be tested due to the lack of feedback (e.g. through animations) in the current game build.

Justification

These functionalities were chosen to be tested as due to no previous user testing there is no support (other than theoretical research into other games implementing similar features) for the base functionality of the player. Therefore, in order to ensure the main aspects provide an approachable and coherent experience they must be tested early and together. Likewise a thinking aloud approach was deemed best as this allows the developers to gain an understanding of what the overall player experience is, alongside being able to identify as many frustrations as possible early in development.

Testing Process

User Brief

Before testing the user will be given a brief description of the game and basic descriptions of the player functionality (i.e. the player is capable of attacking and getting hurt) alongside the controls. The test administrator will explain how the player is to tell them all their thoughts while completing the given tasks, without asking for guidance during play.

User Tasks

The activities the user will be asked to perform will include both basic concepts (e.g. how does the player attempt to move and attack using the keyboard and mouse) as well as more complex tasks that require the player to better interact with the games systems/rules.

1. Walk around the perimeter of the screen (to test the feel of the basic movement controls)
2. Perform melee and ranged attacks (to test the feel of the player attack methods)
3. Take damage then regain health (to test whether the player expects the regeneration system/how it is implemented as well as their understanding of the players interaction abilities within the environment)
4. Kill an enemy without dying (to test if the player is able to simultaneously interact with the health and combat system)
5. Reach your most wounded state then kill an enemy without dying (to test if the player understands the changes to the system when the player is more wounded, checking the balancing between movement speed and player attack damage)
6. Dash around the game area and enemies (to test the fluidity of it as part of the game and how natural an inclusion it was. Also looking at its distance and delay)

Small Questionnaire

1. How did you find keeping track of the wound level and identifying the player's health?
2. Did you find that in fighting the enemies that if felt easy/hard? Why?
3. Did the controls feel natural? Why?
4. Did the dash feel beneficial to the experience? If not, how could it be improved?
5. Is the difference between wound level and state health distinguishable?

Testing

Participant #1: Significant observations and feedback

Attacking:

• When melee attacking they disliked the lack of visual feedback (spammed the spacebar trying to get it to work)
• Was surprised the ranged attack was set on last player movement but said it was something they'd get used it
• Said that the ranged attack would be easier if the player icon showed the current direction
• Was frustrated that they had no reference for how much damage the enemy was taking
• When using melee they spammed it in place
• Used ranged attacks mostly at first (used more melee later in testing)
• Said they didn't struggle during combat but also didn't find it too easy
• Felt like the knew when they would get hit but wasn't sure where they could melee attack from

Movement & Dash:

• Found the basic move controls smooth and expected, was happy that diagonal movement was allowed
• Mainly used to dash to get away or past enemies
• At first struggled to use dash (wasn't holding direction while doing)
• Noticed delay and was not bothered by it's length

Health & Wounds:

• Had no trouble getting close and noticed health regenerated when they avoided for long enough
• Seemed a bit confused at first when taking damage, weren't sure if they were taking damage or not
• Didn't notice the difference in the character when in a more wounded state
• Didn't look at health often
• When looking at health was surprised at amount of damage taken, not surprised that they had taken damage
• Quickly understood the wounds going down as player health went down
• Liked the regeneration gates (i.e. wound system)

Controls:

• Said they liked ranged attacks being on a separate hand (easier to move and attack at the same time)
• Didn't like "L" as the ranged attack (felt random/hard to remember)

Participant #2: Significant observations and feedback

Attacking:

• Said the projectiles seemed too large compared to the characters size
• Felt lost when melee attacking as there was no noise, couldn't tell what was happening on screen
• Said they felt like the melee attack did nothing
• Died often while fighting
• Majorly used ranged attack
• Felt like fights didn't last very long
• Said it felt easier to use the ranged attack while moving as they knew where it was going

Movement & Dash:

• Said the basic movement felt responsive
• Liked the axis' they could move on but expressed frustration that the character image didn't turn
• Didn't use dash often during fight, mostly used to run away between encounters
• Said they liked the amount of distance travelled in the dash

Health & Wounds:

• Looked at health decently often, noticing regeneration was happening
• A little confused by the wound system, expressed surprise when the players health went up after the first wound
• Noticed when they got more wounded that the player got faster
• When wounded they ran away further from enemies before attacking again
• Didn't notice difference in damage when player was more wounded
• Liked the health regenerated, liked that it didn't regenerate all the way

Controls:

• Liked that the ranged attack used the other hand
• Struggled to remember the key bindings for melee and ranged attacks
• Said the regular movement controls felt natural, same for shift

Participant #3/4 (Done together): Significant observations and feedback

Attacking:

• Melee attack is confusing, can’t tell when its hitting an enemy and how great the range is
• Can’t tell if hurting an enemy or what health they are on
• Enemies a bit too beefy, especially not being able to tell enemy health
• Couldn't really understand the range of melee attack until they saw the collision box in debug mode
• Ranged attack is confusing with the slow bullet and fast movement speed

Movement & Dash:

• Movement overall feels good and makes sense for a game
• Likes the dash but would prefer it to be a bit longer distance
• The cooldown is good but would like a way to see it

Health & Wounds:

• Thinks regenerating is good
• Finds the wound state and health a bit confusing (specifically using numbers to represent the wound)
• Would prefer the health and wounds to be visual, not numerical
• Likes the idea of using items to increase/regenerate
• Wants the sprite to change depending on wound state
• Could balance health by disabling the dash when badly wounded
• Regen timer is at a good point

Controls:

• Realises keybinds aren't final but doesn't mind them
• Left-shift is good for dash
• Takes a bit to get used to shooting in four directions

Overall:

• Wants character customisation
• Overall game is pretty good, especially considering the current limitations (i.e. Sprint 1, team doing Player)

Evaluation of Results

Common Issues & General Options for improvement:

Going through the tester responses common issues were congregated and possible solutions were listed for further consideration.

Testers expressed issues with the keymapping of the ranged attack function, describing it as random or struggling to remember it (however players liked most other key bindings and preferred that the player attack used their second hand).

• Using a different key on the right side of the keyboard for ranged attacks (e.g. enter/directional keys)
• Using the mouse for ranged attacks

Testers had issues with the feedback of the melee attack (unsure what it was doing, how far it went & how to use it)

• Implementing a directional melee attack to allow users better control
• Adding a melee animation to show both response when attacking and attacking range

Testers overall liked the ranged attack but all had issues with knowing the directionality of the attack while playing. Issues with the size and speed of the attack were also mentioned.

• Adding directional changes for the player sprite so the player always knows which direction the next attack will go
• Utilising a mouse system for aiming so the player doesn't have to be aware of direction of the player sprite/last movement
• Reducing the size of the projectile
• Increasing the speed of the projectile

Testers had an issue understanding if they were progressing in the fight (no idea how long it would take/if they were doing well).

• Adding enemy health to the screen so users are aware that their attacks are working & how long a fight is likely to last
• Adding visual damage indicators to show attack effects

Testers had an issue with fighting, feeling that the fights lasted longer than was interesting, with some user dying often before fights were finished.

• Increasing player attack/lowering enemy defence
• Implementing enemies with varied attack/movement sets to add more dynamics to fighting

Testers all mentioned enjoying the basic directional movement of the game and the implementation of a dash with a cooldown timer. However some had issues using it (holding down direction while pressing dash), being aware of the cooldown between dashes and one user disliked the distance.

• Including a tutorial sequence in the game to explain how to use the dash
• Increasing the distance travelled in a dash

Testers all liked the regeneration and wound gate system but were confused by its presentation in the game (didn't understand what the wound state number was, was confused when health increased after damage led to a health of zero in that state)

• Using a visual representation of the health (possibly blocking off sections to show health that can no longer be regenerated once wounded)

Testers all had trouble keeping track of the players health in some respect (dying often, being surprised by current health, not noticing when they were taking damage, etc)

• Adding visual changes to sprite to show when hit is taken
• Adding number indications to screen to show amount of damage taken by player
• Having a sound effect to signify the player taking damage

Most testers noticed the change in the player in different wound states, however it took most players a long time. Players tended to play very similarly when wounded or healthy.

• Having sprite change based on wound state to give visual representation
• Having more wounded states have less health to encourage running away
• Removing the dash in more wounded states to discourage fighting when highly wounded

Currently Actionable Improvements:

In order to improve based on these observations (as much as possible within the remaining time of the sprint), the following modifications were implemented:

• Using a different key on the right side of the keyboard for ranged attacks (enter)
• Increasing the speed of the projectile
• Adding directional changes for the player sprite so the player always knows which direction the next attack will go
• Altering dash speed

Evaluation of Methods

Overall it gave a lot of information however a lot of problems centred around issues with feedback rather than the implementation and experience playing the game with the added player functionalities. In future sprints, where visual feedback is included, this form of testing would be highly successful but is currently less effective. This method could have been made more effective by more support for the tester, possibly by telling the player when the melee attack is being performed correctly (i.e. when feedback would occur), so the actual dynamics of the system would have been better explored.