Methodology

Overview

The latest version of the game will be shown to multiple users to gather feedback on the mid-level cutscenes and NPCs. The user will be briefed about the current stage of development and the target audience. The user will be asked to play the game from start to finish (possibly multiple times if they die), and will be told to speak their thoughts out loud while playing. After the user has completed the game they will be interviewed about their experience. After the interview the user will be given a short questionnaire to fill out for primarily quantitative data.

Justification

One of the main focus for this sprint was create "juicy" game experience. As a game being "juicy" is subjective, user-testing is imperative to check whether the correct experience and emotions are being created from the features. The following things will be focused on in this user testing:

• New enemy designs: Users' thoughts on what they are and the feelings they invoke.
• Enemy flashes when hit: Whether this makes the combat more intuitive/smooth.
• Enemy is knocked back when hit: Whether this lessens frustration when the enemy won't get off the player.
• Enemy visibly changes when wounded: Whether this lessens frustration of not knowing if they are doing damage or not.
• Enemy leaves a corpse when killed: What feelings are evoked by seeing the dead body, is there more of a feeling of accomplishment?
• Enemies in darkness are faster: Whether the player tries to stay in the light more. What feelings this invoked.
• Enemies swarming the player when the player is in darkness: Same as above.
• Enemies dropping loot: Reactions to receiving loot, does this stop users' current strategy of just avoiding all the monsters.
• Sound effects: What feelings they evoke. Does it alleviate any frustration with combat?
• Hitbox sizes: Do users feel they were hit when they shouldn't have been hit.
• General balancing: Do users struggle to get past the enemies?

Testing Process

User Brief

Before testing commences the user will be given an explanation about how "thinking aloud" testing will work and the current stage of game development. The game's genre and target audience will be explained to the user, and the users will be notified that the game may contain disturbing scenes. The user will be notified that the observer will remain silent while the task is being performed but if they have any question then they can ask the observer.

Interview after finishing the game

The following questions will be asked once the user has finished the game:

• Describe your overall feelings that you felt playing the game.
• Explain what you think each enemy type is. (Ask them about the spawner enemy as well if they don't bring it up).
• Describe your emotions when first interacting with each enemy type.
• What was your favourite enemy and why?
• During combat, what were some features that helped make the combat feel smooth/enjoyable? Describe why you feel they helped.
• During combat, what were some features that made the combat feel clunky/rough/frustrating? Describe why you feel they caused this.
• Describe your feelings of seeing the enemies corpses left behind after they are killed.
• Describe feelings of the darkness or being in darkness within the game.
• Describe the strategy you used to complete a level.
• Describe the feelings the you felt when hearing the different enemy sound effects
• What were some of your favourite sound effects? What were some of your least favourite sound effects? Explain why they were your favourite or least favourite.

Small Questionnaire (Quantitative testing)

A small questionnaire will be given to the user to complete after all tasks are complete. These questions will contain a Likert scale.

• I found the slug creature interesting or scary
• I found the eyeball creature interesting or scary
• I found the large shooting creature interesting or scary
• I found the large melee creature interesting or scary
• I found the spawner and the spawner mechanics interesting or scary
• I found the enemy corpses interesting or scary
• I understood that the spawner was creating slugs
• The enemies had a consistent and coherent theme to them.
• The enemies were consistent compared to how Shadow Crawlers were defined at the start of the game.
• I knew when the tougher enemies were almost dead.
• I knew when I had successfully hit an enemy.
• I felt like it was an accomplishment when I defeated a hard enemy
• I found the unlit areas of the game and the enemies glowing eyes scary
• I tried my hardest to stay in the lit up areas.
• I felt that the loot enemies dropped was rewarding.
• I felt that the enemy dropped enough loot for me.
• I felt that the game was too easy
• I found the slug creature easy
• I found the eyeball creature easy
• I found the large shooting creature easy
• I found the large melee creature easy
• I found the spawner and the spawner mechanics easy
• I found the enemy sound effects were scary
• I understood what most of the sound effects meant
• I liked the game
• I felt that the game was too short
• I found the game interesting/fun

Testing

Participant #1: Significant observations and feedback

• Describe your overall feelings that you felt playing the game.

Fast paced, spooky.

• Explain what you think each enemy type is. (Ask them about the spawner enemy as well if they don't bring it up).

Speedy slug, Laser eyeball, spike spider, big monster, slug spawner(at first confused, but once it started spawning slugs it was recognisable as a spawner)

• Describe your emotions when first interacting with each enemy type.

Panic, run away, but it’s too fast. Eye ball is easy just a rock Big spider, dodging the spike is pretty easy, and easy to kill overall Hard to hit when standing next to it, since it would hit you first, but it was easier to hit it from underneath. Spawner was easy to destroy, but slugs spawning made me want to use rocks instead, but since it has high health i needed to use the melee weapon to save the rocks.

• What was your favourite enemy and why?

Slugs are fun and cute. Challenging to fight.

• During combat, what were some features that helped make the combat feel smooth/enjoyable?

Rock was smooth and made the combat easy.

• During combat, what were some features that made the combat feel clunky/rough/frustrating?

Animation for some weapons are a bit slow and it’s hard to tell what its hitting.

• Describe your feelings of seeing the enemies corpses left behind after they are killed.

It was easier to tell when the enemy was dead.

• Describe feelings of the darkness or being in darkness within the game.

The benefit of the torch wasn’t clear. The darkness was spooky, and it was harder to see mobs coming from the edge of the screen.

• Describe the strategy you used to complete a level.

Going to the safe house as fast as possible, and encountering mobs in large clusters. Buying the items in the shops.

• Describe the feelings the you felt when hearing the different enemy sound effects.

The sound effects don’t have consistency. Some sound effects are weird. Some sound effects are too loud. Big monster death sound especially.

• What were some of your favourite sound effects? What were some of your least favourite sound effects? Explain why they were your favourite or least favourite.

Spawner sound is cute Spider shooting sound sounds grating.

Participant #2: Significant observations and feedback

• Was very cautious in approaching the first enemy
• Was highly excited about money that had dropped
• Frustrated that some coins had spwned behind a tree and was inaccessible
• Toggled the torch off to preserve it.
• Was concerned about the large enemy due to its size
• Dashed from light source to light source
• Torch ran out early and they died
• Really wanted more rocks, they felt they had to ration them.
• Felt like they should be able to get new torches, was stuck without a torch.
• Couldn't talk to the person in the safehouse (potential bug?)
• Couldn't talk to the dead pilot body (potential bug?)
• Felt the projectile spikes had too wide hitbox, felt they hit them when they shouldn't of
• Pushed pilots dead body around the map to try to use him as a shield
• Likes that you get rocks as loot from the spawners
• Used rocks against the big enemies (as the tester though they would be more difficult) and ranged enemies, weapon for the slugs
• Figured the big enemies would be difficult so used rocks
• Liked the detection noises (particular the rattle snake noise of the slugs)
• Wondered if they would spawn back at all? Figured they didn't come back as long as all the spawners were destroyed.
• Felt the game was a typical dungeon crawler
• Felt overwhelmed with so many buttons to press
• Felt a bit paniced when realising the torch light was running out
• Thought the eggs sack were gross. Felt they matched the games theme but still thought they were gross.
• Thoughts on the large tough ranged enemy: Oh shit his big!
• Tried to avoid the large snake enemy as they had high health high health.
• When the torch ran out, they felt like they couldnt see. Maybe if there was more fireflies around the place.
• Favourite enemy was the Antlion (tough ranged enemy) as they really liked the design. Thought it was cool for a monster. Antlions are not overused (unique) where snakes are used a lot.
• Combo of ranged and melee weapon was good, felt more control to attack from afar and up close.
• Found it frustrating getting caught on an invisible tree or throwing a rock and it will hit an invisible tree
• "After my first death experience in the darkness I decided to avoid the darkness"
• All sound effects seemed pretty logical. Eyeball was squelchy, that made sense. Wet squishing noise.
• "(in response to favourite noise) Rattlesnake (tester means the small slug enemy detection noise) was my favourite. I liked thinking they were little rattle snakes, or that they were scuttling towards me quicker"
• Squelching noise was least favourite but felt it was suitable... just gross
• Didn't feel that those particular monsters matched with the story of taking the light away. Why are bugs and snakes taking the light?
• Thought the monsters were consistent with each other (snakes and bugs) except for the eye.
• Spawner took longer to kill than I thought it would

Participant #3: Significant observations and feedback

Details

• Tester: Renée
• Time: 18 Oct, 7:07pm
• Commit: 33d0d6

General notes and Feedback

• I deliberately used a tester who has not previously been involved in testing previous sprints of this game to gain an overall feel of the game in a near-completed state.
• For this reason, they did not comment on improvements made since the last sprint.
• They didn't make too many complaints or complements, suggesting that this game was at least not frustrating to play.

Game difficulty

• The game was too easy for the tester, even on their first playthrough.
• There were enemies that could be hurt by rock-throwing while remaining outside of their detection range (and the enemy not reacting).
• Hit-and-run combat tactics were extremely easy to execute.

The Darkness

• They thought the rock collectables were pieces of coal that could be used to refuel the torch and was confused when they tried to re-light the torch after it had run out of fuel.
• Was quick to ask whether the torch had a limited lifespan and took about 30 seconds to notice the torch radius decreasing.
• They liked the fading light radius as a visual indicator, but commented that this would not be useful if there were multiple torches in the inventory (there is currently no HUD display for this)
• The player sprite not becoming dark was a lighting inconsistency.
• The enemies in the darkness could be made even faster to further incentivise players to stay in the light. For example, the large melee enemy could still be outrun.
• The torch had to be manually turned off when in a safehouse to stop it from wasting fuel.

Story Consistency

• The Shadow Crawlers behave like generic monsters that could have belonged to many games of similar genre. There was no mechanic where they took light away, as described in the opening cutscene.
• Making them more aggressive in the dark was not enough to portray this narrative element.

Other

• Found that enemy knockback effects were 'pretty standard' of most 2D games of this style.
• The sound effects added to the game's enjoyment and were not annoying. They were also useful for enemy identification.
• There was an abundant amount of money to be found on the map and from dead enemies and it was too easy to purchase quality items in the early game.
• They found the pixel art 'cartoonish' and couldn't take the horror genre seriously. It felt like horror for children.
• The large melee enemy was likened to the enemies from Alien: Isolation
• Likened the game of Pokémon Sapphire and Don't Starve. It evoked a slight sense of nostalgia for this genre of 2D games.

Participant #4: Significant observations and feedback

The small enemy: "grub"

• they stand out too much in the darkness, cause they're fully lit and they're too bright. It looks like they're overlaid instead of being underneath the cloud of darkness.
• big fan of their animation
• first thought was "squoosh" - just wanted to squash it

The long range enemy: "eyes"

• when in the darkness the red glob looks weird. not a huge fan of that.
• it's fairly well balanced in its attack however
• maybe try to make the darkness sprite look more like an eye - "thought it was a mushroom" (??)

The large enemy: "scorpion sorta dudes"

• Favourite
• the two red eyes in the darkness were effective and made sense
• "coolest character design"
• really liked that they weren't just a one hit enemy, made it a more enjoyable experience to kill them
• didn't notice the injured sprite till it was mentioned to them

The tough range enemy: "3 spear dudes - spidery"

• They found the leg colour was too similar to the sidewalk in the first level but it still obvious what it is
• A cool way to incorporate a new ranged enemy with it having 3 shots instead of the same as the long range (variety is good)
• Liked the pointy spears as a projectile

The Spawner: "eggos"

• the damaged sprite is cool
• A bit different cause it's a passive enemy but you still want to destroy it
• it's often surrounded by other enemines, not just the grubs, which forces you to have to balance dealing with other enemies, the grubs and killing the spawner - which is really good gameplay experience
• it bled purple, which is cool.

other enemy notes

• Really enjoyed the variation between the ranged and melee enemies
• the knockback is useful
• although sometimes it looks like you can hit things from further away because of the animation
• didn't take a huge notice of the corpses but had no issue with them - it was a nice way to know you've done good
• "I like the concept of being able to see the eyes in the darkness"

other notes

• overall, it's good and quite fun. Didn't look too difficult but not so simple that it was boring
• "I really like picking things up. I want all the things. It's my motivation in games. I like it when they drop things" - enemy drops are good
• maybe having the harder enemies/bosses drop something "cool" to give players incentive to really want to kill them cause they can avoid the enemies if they really wanted
• The conserving of the torch fuel is a very effective aspect of the game (a bit different) - changes how you would've played the game
• it would be cool to have an enemy that can absorb the torch light for a bit, as another element to make it more difficult
• liked the firefly addition in the 2nd level - a nice safe spot for when your torch is running low
• the squidgy sounds were the favourite - "they're funny" (they weren't able to hear everything due to the testing situation)
• the game is short but it's not bad that it's short

Participant #5: Significant observations and feedback

The overall world was dark and scary, music does not match. Except for round 2 and further on.

There are a couple of enemy types that shoot things and others that hit close up. The spawner enemy clearly looks like that it spawns an enemy.

The slug was gross, didn’t want to touch it. The eye balls and large antlion thing that are ranged were more enemies to avoid the bullets. The large enemy was an enemy that they also wanted to avoid at all costs, as it looks like it does a lot of damage. The large enemy looks like the predator in alien vs predator.

Liked the large enemy the best as it looked cool.

The bullets feel satisfying when it hits an enemy. The animations felt like that it made an impact.

The bat feels like there is a delay in attacking, even though it actually instantly hits. The animation felt that there is nothing really happening until the last moment. Also they felt that the torch didn’t run out quickly enough.

The feeling when killing a slug was Yes it’s dead! But also gross.

Felt that they can’t really see the enemies around and that they should stay in the light. But it’s not the “you’re screwed” sort of moment that the dialogue suggested.

Run past everything if it gets too busy, but attack enemies in isolation.

The enemy sounds sounded like they blended into the soundtrack too much, it’s sounds like it’s more of an identifier that an enemy is nearby.

Liked the large enemy sound the best. Did not like any sound that blended into soundtrack too much

Survey results

The average survey results for the testers (rounded to the nearest option):

• I found the slug creature interesting or scary: Agree
• I found the eyball creature interesting or scary: Agree
• I found the large shooting creature interesting or scary: Agree
• I found the large melee creature interesting or scary: Agree
• I found the spawner and the spawner mechanics interesting or scary: Agree
• I found the enemy corspes interesting or scary: Neither Agree or Disagree
• I understood that the spawner was creating slugs: Strongly Agree
• The enemies had a consistent and coherent theme to them: Agree
• The enemies were consitent compared to how Shadow Crawlers were defined at the start of the game: Neither Agree or Disagree
• I knew when the tougher enemies were almost dead: Disagree
• I knew when I had successfully hit an enemy: Strongly Agree
• I felt like it was an accomplishment when I defeated a hard enemy: Neither Agree or Disagree
• I found the unlit areas of the game and the enemies glowing eyes scary: Neither Agree or Disagree
• I tried my hardest to stay in the lit up areas: Neither Agree or Disagree
• I felt that the loot the enemies dropped was rewarding: Neither Agree or Disagree
• I felt that the enemy dropped enough loot for me: Agree
• I felt that the game was too easy: Neither Agree or Disagree
• I found the slug creature easy: Agree
• I found the eyeball creature easy: Strongly Agree
• I found the large shooting creature easy: Agree
• I found the large melee creature easy: Agree
• I found the spawner and the spawner mechanics easy: Agree
• I found the enemy sound effects were scary: Disagree
• I understood what most of the sound effects meant: Agree
• I liked the game: Agree
• I felt that the game was too short: Agree
• I found the game interesting/fun: Neither Agree or Disagree

Evaluation of Results

Common Issues & General Options for improvement:

• some users found the game too easy
• there was an attack delay
• many users found that the torch running out didn't have enough consequences in terms of making the game harder
• some sound effects blended into the soundtrack too much and were hard to hear
• the enemies are more monster-like than 'shadow-crawler' and they don't take the light away
• there needs to be some work done on sound effect volume balancing

Currently Actionable Improvements:

• increase detection rate of enemies: making it more difficult
• increase enemy speed in darkness: adding more consequence to the torch running out
• increase attack/shooting speed: again, making the game more difficult
• adjust sound effect volume: pretty self-explanatory
• adjust lines in the prologue: to mention "creepy crawlers" to alleviate the issue of the "not a shadow" issue
• some final adjustments of enemy assets < what they look like in the darkness
• add more details to the long ranged glowing eye sprite as one tester thought it looked like a mushroom
• reduce the opacity of the small enemy glowing sprites as one tester thought they were too bright (looked unrealistic)