User Evaluation Plan

• Session Run by: Caleb Ang, Nisha Vashist
• Studio and game name: Studio 2, Atlantis Sinks
• Feature and team name: Combat Items – Weaponizers
• Location of Testing: In person think aloud
• Number of Participants: 5 individuals

Background Research

Throughout the previous three sprints, we have been testing and building upon certain elements of the game. Now that the other features, including ours, are playable within the game, the balance of our features within the whole game needs to validate. In line with juicy game design, we will also test if our feature fits with the player's positive gaming experience. Which works well as juiciness, such as sounds, were added this sprint. It will further include any items that weren’t explicitly tested or weren’t deemed as integral to the game(mentioned in the last sprint's future directions). It is hoped that this would allow us to further the refinement highlighting key functionalities and features of our branch to users and really go deeper into the features that we already have.

To facilitate this, we have decided to create a mock game whereby all weapons would be spawned at the start of the game as well as spawning the different buffs in front of the players. This would ensure that user testing time is kept to a minimal timing so that players are not distracted by other components of the games such as extended walking period etc.

It was also found from previous interviews that think aloud was a very successful method to attain information on how well the prototype was doing. As such, we have also decided to utilise the think aloud method for user testing in hopes of attaining information regarding the below mentioned questions and tasks.

Individuals being tested should have had experience with playing the game before. This would allow for a better understanding and concept of the basis/ foundation of the game. If users being tested do not know the concept or background origin of this game, they should be inducted or introduced to the theme of Atlantis sinking in order to be ‘caught up’ on the features being tested within this user testing.

Aim & Objectives

1. To attain better understanding into overall game aesthetic of weapons within the game
2. Discover any inconsistencies or any design that require refinement on when buff animations are activated
3. Understand and attain informant regarding weapon attack damage and weapon cooldown properties within the game
   • Does the attack damage need refinement?
4. To see if players register that a weapon needs to be used to attack
5. To see if the players has a positive experience with the addition of sounds on use of the items.
6. To see if the player will recognise buffs and understand whether it can be used to increase the weapon stats of an item.

Overall, each question gives the player the exploration needed to answer existing aims and goals but goes further as it confirms that the goals set out within the previous sprint and testing plans have been met.

Prototype

• Showcase gameplay and weapons, including the attack sounds. This can be seen in the following branch of #lachlans-playground

The following image is part of the prototype created for the user tests:

Testing Approach

Testing will take the form of informal interviews through think aloud, where individuals will be tasked to say their thoughts out aloud when they complete. The method will end with an exiting interview to get a more directed response on the user's overall experience.

Think Aloud Questions

1. Attack an enemy
2. Find a way to increase your weapon stats and attack with it.
3. Attack with the increased stats of your weapon.

Post interview questions to be Asked:

1. What do you think of the colours of the weapon swing?
2. What did you think of the weapon swing speed of these animations?
3. What did you expect the attack damage to be of these weapons to be when hitting the enemies?
   • Iterate through the weapons and demonstrate how to kill an entity with the weapon.
4. Did the sound engage you in the game?
5. If you were not satisfied with the attack interface of the weapon, how many hits do you think this weapon should've taken to kill this enemy?
6. How about for this boss, how many hits do you think it should've taken to kill the boss?

Post Round Interviews:

Question 1:

• I think they look really good, they fit the buffs. Most games the poisons would be green, the fire buff thing would be red so it’s similar to other games making it recognisable.
• The buffs look pretty on point, the colours would represent something that I would normally see. The damage buff seems a bit weird? But I guess that’s because they’re blue but like I mean they could also be slightly purple in some sense because I usually associate a slight hint of purple with an increase amount of damage. Blue works perfect fine though.
• The colour of the buffs looks sick! I really like the lightning buff looking like thing it gives off electricity or fast vibes which is what the buff is all about. Making the weapon swing or shoot faster! The fire buff looks on point and the poison and attack damage buff also look really nice.
• The swing colours look epic. I kind of feel icky when I equipped the green buff, along with the firey with the orange one, electric with the yellow one and powerful with the blue one.
• It is what I would expect the buff to look like when I picked it up. Even better, to be honest.

Question 2:

• I think that the weapon swing speed is fine. It looks really nice when it’s slower rather than faster because if you swing way too fast, then you may not to get to indulge in every frame of the animation which brings in the attention to detail.
• I feel like the weapon speed is fine for the majority of the weapons. Some weapons looks really in synced and nice but there are like one or two weapons where I feel like the attack speed animation/ swing could be made slightly faster. The impact doesn’t seem like very “oomph” but once again this is something I feel like might be a bit too much considering that my background is more of a fps game.
• I think that the weapon attack swing looks fine. It’s not too fast nor too slow for the way it is being used.
• Again, I think its epic. The weapon swing animation match the speed. Any faster, or slower may make it harder to actually the animation.
• The speed of the swing is good. But perhaps for the buff swing you could speed up the animations to really reflect the buffiness of the weapon, particularly the speed buff.

Question 3:

• Level one weapon damage could be increased slightly to reduce the amount of time spent trying to kill an enemy. (Range from 15-20)
  • The daggers damage should also be increased considering it’s a level two item and it’s dealing almost the same amount of hits as the level 1 items. (Range from 20-25)
• Level 2 weapons should range from 25-30.
  • I think that the lower level one items such as the plunger and the pipe deal too little damage for the game. Perhaps a slight increase in damage may allow these weapons to feel more realistic as currently I feel like they have to hit the enemies too many times.
• I think that currently the only weapons that need changing is the plunger as well as the pipe as those items take long for me to kill an enemy. You don't want to keep a player in suspense as that could lead them to leave the game.
• It is a bit slower for my liking, but I think that's what would encourage me to go for the buff and talk to the NPC to craft other weapons.
• I like the current damages for the weapon, but the lower weapons require excessive hits, which is annoyingg.

Question 3:

• I definitely think the sounds made me feel more mesmerized, however the sounds were a bit too harsh on the ears.
• It engaged me in the game because I was reassured the attack was happening.
• It would be nice if the sound matched the flow of the weapon swing. I liked how you guys used the same sounds for the same materials.
• The sound needs to be cleaner and clearer. When I say cleaner, I mean focused on that one sound.
• The sounds are good, but you could possibly look at the different variations of the sound for example, instead of a ting for the Melee weapons, have a more a long zing.

Question 5:

	Level 1 (Attacks)	Level 2 (Attacks)	Level 3 (Attacks)
User 1	7	5	2-3
User 2	5-6	4-5	2-3
User 3	8	6	3
User 4	7	6	3
User 5	6	5	2

Question 6:

• The bosses should be scaled according to the amount of health they have. E.g. if the boss has 10x more health, the amount of times it takes to kill them should also be 10x the amount. Suitable amount might be 3 times the amount.
• Hmm.. it does seem a little bit hard to kill the bosses within those levels but I guess that because they are boss, they should naturally be harder to kill than the enemies. Perhaps something that you might consider is lowering the bosses HP instead of increasing the weapon damage because that might ruin the gameplay from the earlier stages of the game.
• I think that the number of attack it takes to kill the current bosses is fine. Though it takes sometime, they’re called bosses for a reason. If they are killed too easily, the game wouldn’t be really challenging at all defeating the purposes. Furthermore, there are other items you can collect like buffs to help you kill the boss.
• The number of hits it took me to kill the boss was good. Like I said in my previous responses it encouraged me to go to get a buff.
• It was challenging to kill the boss, but it is the boss it shouldn't be easy. I just like easy wins.

User Testing Results

User 1

Task Number	User Comments
Task 1	I would try find an enemy and then try to hit them with something, maybe a weapon of some sort or use my hands.
Task 2	I don’t know what this means but I think there were floating things and I can use that because they seem like the buffs that I could pick up to increase the weapons stats of the weapon.
Task 3	I would collect one of those buffs that I mentioned earlier and then use them to attack an enemy.

User 2

Task Number	User Comments
Task 1	Judging by the way the game was made, I would ideally need a weapon to hit an enemy unless I can hit them with my hands? Though I feel like it makes more sense for me to equip the weapon and then attack it.
Task 2	Hmm.. to increase the weapon statistics… Maybe there’s something I can drink like a potion or a buff or something. Let me check whether this potion gives me more stats. Hmm, I let’s see oh, this buff, let’s try this.
Task 3	Where’s the monster, I would now need to just go find an enemy to kill.

User 3

Task Number	User Comments
Task 1	I would collect one of those buffs that I mentioned earlier and then use them to attack an enemy.
Task 2	I think in terms of this, I would just need to go get a buff that I went past previously on the map.
Task 3	Oh wow, the increased attack damage is really a lot of additional damage.

User 4

Task Number	User Comments
Task 1	I don't have a weapon in the inventory, I need to explore the map to find weapons. There is a crafting table, so I guess I need to find the resources to build my weapon to be able to attack the enemy.
Task 2	The chili pepper icon is speaking to me.
Task 3	Oh wow! That was like 3 hits, and it was gone. Sick animations on the swing.

User 5

Task Number	User Comments
Task 1	I can attack with my hands, but I need to find a weapon to deal more damage.
Task 2	I could craft another weapon like I did before or collect those icons.
Task 3	Nice! The sword requires so much fewer hits than the pipe.

User Feedback

There were several items that we learnt from the user testing completed here. The following is just a summary of the items that were significant enough to change.

Weapon Damage Overhaul

• It was found throughout the user testing that users found the level one items to be very underwhelming and as such, impacted the gameplay as it took too long for them to kill an enemy
• Changes that could be made for this may include increasing the damage of level one weapons (plunger and pipe) so that they reflect a reduction in the number of hits it takes to kill a basic enemy

Weapon Animation

• Weapon attack animation speed (with and without buff) seemed to have resulted in an overall positive feedback with majority of users agreeing that the animation and timings were appropriate
• Exception of some comments that have suggested the animation of the plunger seems a bit slow
  • Accepted this as an outlier as well as not accepting comment since plunger animation was originally supposed to go through two phases of animation. Initial Phase was for plunger to be stabbed and the second phase is where the plunger would 'pop' out poop

Weapon Implementation Against Bosses

• Users had common agreement that attack damage for bosses were alright too as they were already expecting to have to give the boss multiple hits before seeing it die
• User commented on how instead of increasing damage of weapon (after this user testing), would be to reduce the health of the bow slightly so that the game is more 'playable' and not over challenging

Weapon Sound

• Users highly value the quality of the sound.
• The sound is expected by the user, it reassures the player that an attack has occurred. Interestingly the comment on how the same sound was used for the same materials was memorable in recognising weapons of similar nature.

Balance Changes

There were several changes made in order to balance the game functionality as damages seemed to have been slightly inaccurate according to users when they experienced the game first hand. These changes are reflected under Balance Changes Ticket #284. The general guideline of balance changes throughout this sprint patch are as follows:

Athena Dagger

• Damage increase from 12.5 to 20.6 damage per second (dps)
• Cooldown increase from 500ms to 700ms

Hera Dagger

• Damage increased from 16 to 20.5 dps

Sword Level 2

• Damage nerf from 30 to 25 dps

Trident Level 2

• Damage nerf from 35 to 27 dps
• Cooldown buffed from 1500ms to 1200ms
• Reduced weight from 3 to 2

Hera & Athena Dagger

• Damage nerfed from 40 to 32 dps
• Buffed cooldown from 1000ms to 650ms

Plunger

• Damage buffed from 10 to 17 dps
• Reduced cooldown duration from 1000ms to 650ms

Pipe

• Buffed damage from 15 to 19.5 dps
• Buffed cooldown from 1000ms to 500ms

Regular Bow Plunger

• Nerfed cooldown from 500ms to 1500ms

Future testing plan

Although we have reached the end of the scope for our assignment, the direction that we need to take encompasses an iterative design approach. This would mean that the above user testing plan should be followed for the next sprints to ensure that the elements of the game balanced and are positively impact players that interact with this game. However, new forms of testing methods can be implemented to find other user pain points. One such example can be seen through control groups/ user control environments to truly test the variety of players that play the game. For example, we could do a split testing between experienced versus unexperienced users with background knowledge of these kind of games. This would assist in allowing us to know whether the game is at a challenging level that spurs players to try over and over again rather than wanting to quit the game.

Testing Methods and Procedures

Some questions that we asked may take the form of open ended questions that allow users to interpret the gameplay freely without misguidance/ being led on.

1. How difficult did you think it was to kill the enemies and bosses implemented within the game using a melee weapon compared to using the bow/ golden bow?
2. How many rounds do you think you'd need to play the game in order to fully explore the game and become an expert player?
3. Would you opt for attaining the buff to enhance your weapon or would you rather opt to collect and attain the golden plunger?

• This question would allow us to find whether experienced/ unexperienced users would go for buff/weapon. From here, we would then be able to understand user needs better and implement items that suit their requirements.

4. What was your overall experience with the game?

Author

• Name: Caleb Ang, Nisha Vashist
• GitHub: @PurrplePandaa, @NishaVashist0
• Slack: Caleb Ang, Nisha Vashist
• Discord: 🅰nnyeong Haseyo™#8083, nishavashist#4270