Methodology

Overview

As part of the testing plan of the 1st sprint it was determined that player designs should be shown to users in order to gauge player opinion on the character & any assumptions that makes on the story depending on that design. They will additionally be asked their opinion on the different player state images (e.g. the dead sprites and hit animations) to ensure they understand their purpose. Alongside this the items and weapons will be shown to the user to gauge their opinion on both the integration of the designs but also their inherent understandings of the items purposes. This will be achieved through showing the user their designs and asking them a set of questions to gauge perception and opinion.

Justification

The player designs were chosen to be tested as it is an integral part of the users experience and their interactions within the game. If the player character and their items do not fit the games narrative and intended perception of the player character and world then the user will be taken out of the experience, and likely not continue to play the game. Additionally, specifically for items and taking damage, if the user does not understand the meaning of these designs easily the player will be unable to interact effectively with the game mechanics. However, as there are no alternate assets currently available testing will rely on direct perception of single images rather than A/B testing to ensure testing is done even with the limited current resources.

Testing Process

User Brief

Before testing the user will be given a brief description of the game including: the world, general concept of the player and what they are capable of.

User Tasks

The Users tasks will involve showing the player designs and requesting their feedback on them. In order to ensure actionable/useful answers are given due to the very freeform nature of such testing, the tester will ask specific questions, including:

• What is your opinion on the player based on the design? Are they strong/weak? Are they frightened/calm? (why do you think this)
• Do the items feel like they come from the same world?
• Do they feel as if they are correctly sized in relation to one another?
• What are each of the items and what do they do?

Additionally users will be shown the character in action (walking around, getting hurt, dashing, shooting) to get user opinions on the animations. Users will be asked the following questions after demonstration of each animation:

• Do the animations feel consistent with one another?
• Do the animations make sense (do they understand what is happening)?
• Do the animations feel natural (or are there some issues with pacing/stuttering)?

Illustrations Set #1: Basic Player:

Illustration Set #2: Player Hurt & Dead

Illustrations Set #3: Item Images

Illustrations Set #4: Player Weapons

Testing

Participant #1: Significant observations and feedback

Player Designs:

• Look like a worker type guy (outdoorsy)
• Cute -> Would care about his wellbeing (friendly)
• Strong, overalls and a hat so manual labour, stocky
• Realism -> but not modern reality, maybe a fantasy version
• Imagine he would go on a quest, nice but can stand up to things when they want to
• Correctly identified died and attacked images
• Felt the alternative player images made sense
• Seen red to be hurt before, felt natural
• Liked the blood in the dead image (extra detail, feels more real)

Items

• Correctly identified that the band aid healed the player (remarked on green pluses)
• Correctly identified the torch as something that gave light
• Correctly identified the coin was sued for buying things
• Correctly identified the rock was for throwing it (an ammo)
• Rock felt a bit too detailed (out of place)
• Had a visibility issue with the coin, hard to see the green (different colours slightly)

Weapons

• Correctly identified the weapon types
• Identified that axe would be slower and stronger
• Identified that the dagger would be faster and weaker
• Identified that the sword would be an in between
• Felt like they were cohesive except that the battleaxe was a little too detailed

Images Overall

• Could see the player using the weapons (but he seemed a bit too regular for a big axe, maybe a club instead?)
• Felt like the items and the player fit into the same world
• Felt like having both a rock and a battleaxe as weapons didn't fit together

Animations

• Felt all the walking animations worked for the player
• Animations seemed low framerate but fit the game
• Seemed fluid between directions
• Player seemed to small for sword
• Liked motion but was confused it was sideways and not vertical strike
• Felt the player stayed red for too long (thought they were still taking damage the whole time)

Participant #2: Significant observations and feedback

General Feedback based on questions asked

• Player should look more tired or scruffy
• Weapons make sense but would be nice to be able to use different weapons
• Pickup items don't feel like they can be picked up. Item feels like they are part of the world or obstacles. Perhaps have some light shining over it which highlights items can be picked up would be better
• A little odd that bushes and cobweb don't do anything
• Animations make sense and size ratio is logical

Participant #3: Significant observations and feedback

Player

• Cute/small/cared about them
• Reminded them of pokemon characters
• Felt the character would more run away from fights then pick them
• Felt the 2D style fit the character well
• Understood the red overlay was hurt, would have preferred the shading to still be present when hurt
• Liked the dead image just thought that having the crosses less pixelated would be better (looked like a square and lines)

Items

• Correctly identified it was a bandage and was for healing
• Correctly identified the rock was a projectile for throwing
• Correctly identified the torch was for spreading light
• Correctly identified the coins were for buying things at shops
• Had an issue with coin colour scheme (red-green colourblind cannot see the coins markers)
• Also general coin shades were too close together
• Felt the coin and rock felt out of place because of the high detail
• Preferred the less detail items -> fit better with player

Weapons

• Correctly identified the axe and felt it would be used for larger attacks
• Correctly identified the sword but thought it would be used more for stabbing
• Identified the knife but felt it would be more a throwing knife then a dagger (did also suggest it could be a dagger though)
• Felt the axe was a little out of place with the higher detail

Animations

• Felt the animations fit the designs well
• Felt the animations made sense in the world
• Though the hurt animation should flicker if it was going to last as long as it does
• Felt the throwing animation was a little weird as the player didn't look like they threw anything

Participant #4: Significant observations and feedback

8e341c9

Player Designs:

• Mildy worried, doesn’t look super confident
• Vibe of scary plot-line
• Imagines development of character throughout the game (ie. Changes to appearance to look stronger / more confident)

Weapons:

• Correctly identified weapon types, and relative strength / speed difference
• Initially identified the dagger as a “small sword” before implying dagger

Items:

• Correctly identified all items, and their purpose in the game.
• Connected story to torch’s purpose
• At first, didn’t notice the rocks as being items / ammo.
• When prompted, identifies purpose of rocks as somewhat related to projectile / ammo system.
• Coins hard to differentiate, due to “visibility”

Sizing & Animations:

• Animations are consistent
• Player faster moving diagonally than straight in one direction
• Possible to dodge

Participant #5: Significant observations and feedback

8e341c9

Player & Items:

• Player and bushes look nice
• Didn't notice the player went red when they got injured.
• “Cobwebs look like a joke”
• Some items fit the game theme, some don’t
  • The size of the rock, as ammo, is too large compared to the player
  • Trees are the right size

Weapons:

• Correctly identified all weapons and their purpose.

Animations:

• At first, didn’t recognise the animations, being very subtle.
• Player melee attack animation stutters / can spam swing.

Participant #6/#7: Significant observations and feedback

8e341c9

• Player looks like a child
• Player design not consistent with the Intro Dialogue’s detailed design
• “hodgepodge of art styles”
  • trees vs bushes.
  • Rock being too high-definition / scaled too big relative to the player
• Weapons were correctly identified, purpose successfully implied
• Items were correctly identified, and fit with game theme
• Animation of player’s feet vs enemies’ feet not consistent with their speed; player moves faster yet feet are slower.
• Identified the house being the objective, due to the size, location and colour

Participant #8: Significant observations and feedback

• Player design fits into the game theme
• Identifies cobwebs and bushes have some meaning, however notice the game doesn’t implement them yet
• In-game items are consistent with overall theme
• DID NOT TEST PROPOSED ITEMS / WEAPONRY
• Animations are suitable
• Successfully identified house as the objective, managed to always make their way to the safehouse, even if unintentional (touching the safehouse hitbox)

Participant #9: Significant observations and feedback

8e341c9

Designs:

• Colors, style, and animations of the player are consistent
• Player looks like a survivor, looks desperate
• Art style of trees significantly differ
• Rocks are way too big to be items

Weapons:

• Axe and sword are suitable for game environment
• Questions use of the dagger, seems somewhat pointless unless coupled with the sword
• Rocks don't collide with enemy fireballs

Items:

• Game items make sense
• Bandages are somewhat hard to identify (ie. on the game HUD)

Animations:

• Animations make sense
• Consistent with all other animations

Evaluation of Results

Common Issues & General Options for improvement:

Most users had an issue with the higher level of detail on the rock and coin and felt they stood out as different (also felt the same for the axe). Additionally, most users also had issues with the colour scheme of the coin and visibility.

• Reduce the pixel counts for both item images (i.e. reduce detail)
• Reduce pixels or amount of shading for the axe
• Make the coin sprite darker
• Make the coin sprite use different colours (e.g. yellow & dark orange, yellow & grey shading)

Some users mentioned the hurt animation either lasted too long or was too static, making them feel as if they were getting hurt the entire time.

• Make the animation only last a short amount of time
• Make the animation flicker between red and normal player during its duration

Some users mentioned slight problems with the player and the setting (of a post apocalypse) thinking they would be in some kind of fantasy realm.

• Altering the player sprite to be more hurt/dirty/darker to signify the dangerous world and how they aren't invincible

A user had trouble distinguishing what was an item and was confused by the purpose of many of the non-items.

• Altering the art style for pickup-able items (e.g. box around them or hard outline)
• Adding animations for items while they are on the ground and pickup-able (e.g. rotation)

Currently Actionable Improvements:

• Making the hurt animation last a shorter length of time
• Changing the coin sprite to a different colour scheme

Other improvements will need to be considered in later sprints as alterations of design elements will take a large amount of time to research/consider/develop.

Evaluation of Methods

Overall gained a lot of good insight from potential users based on their perception of the game, therefore the testing was quite successful. However, some users had trouble articulating problems with assets and therefore it would likely have been beneficial to conduct this design testing in a more comparative style with alternate sprites for future testing. Additionally, within the method it could have been beneficial to include an integrated testing in order to check the coherency between the other current designs of the game world to check overall cohesiveness of the designs. Finally, gaining the perspective of more users would have been highly beneficial.