User Testing - Jantoom/DECO3801-Team-Mono-2022 GitHub Wiki

User testing templates and results

Activity 1 - Time on Task

Description: Simulate chair controls as inputs via keyboard to test usability by instructing users to complete tasks within the game and timing them. Results will be compared to benchmark expert user times which will be evaluated alongside post-test interviews.

Aim: To determine the validity of the chair controls as inputs by understanding the user’s mental model of the system.

Instructions:

Set benchmark time for the tasks below for 6 expert users
Calculate average completion time
Instruct non-expert users to complete the below tasks while timing and record
Take note of any verbal or physical frustrations when users are completing tasks
Record the successful or unsuccessful completion of tasks
Conduct post-testing interview
Analyse results

Task	Success/Fail	Benchmark Time (average)	User Time
Move the character to the other corner of the map		27
Kill the other player by placing a bomb		7

Post-test interview question guide:

How did you find the activities?
What did you think about the chair controls?
Did you have any difficulties completing the tasks?
If yes, what were they?

Analysis:

Did the users successfully complete the tasks? If no, what failure point did they encounter and when - what difference in the mental model of the user can be concluded and what are the implications for the design.
Were the user times within the threshold (how much??like 1-2 standard deviations or something) of the benchmark time?
Evaluate interview answers to better understand pain points and the user’s mental model

Results:

Task	Success/Fail	Benchmark Time (average)	User Time (seconds)
Move the character to the other corner of the map	Success	27	Pair #1: 29, Pair #2: 31, Pair #3: 36, Pair #4: 30
Kill the other player by placing a bomb	Success	7	Pair #1: 8, Pair #2: 9, Pair #3: 11, Pair #4: 9

Post-test interview:

User pair #1:

How did you find the activities? Yeah it was fine, pretty fun actually. Yeah it was good.
What did you think about the chair controls? Pretty good idea I think, felt a little awkward at first but I’m sure if there was a full set up it would be much more natural. _Yeah, like I said it was pretty fun. _
Did you have any difficulties completing the tasks? Not really, maybe just the initial stage of getting used to the movement. Yeah, we definitely got better with time when we decided who would be responsible for which movement and stuff.
Was it an enjoyable experience? Yeah even just this was pretty fun, I can imagine it being really fun to play once the game is actually completed. Yeah, same. * What did you like most about it? I think definitely having to jump on the chairs - Yep, and I think the fact that it's pretty simple and straightforward to play.

Analysis:

Inputs took a little time to get used to but overall worked as intended for the user’s
The chairs as input methods was the main driver of the experience being fun in conjunction with the simple nature of the game (tasks). * Insight: the game should be easy to learn and interact with, shouldn’t be too complex

User pair #2:

How did you find the activities? In terms of difficulty? It was pretty easy I’d say. Yeah I mean it was interesting…the whole thing with using the chairs.
What did you think about the chair controls? It was cool, makes it a lot more interesting and dynamic than just using the keys. Yeah, definitely made it more fun for sure.
Did you have any difficulties completing the tasks? No, not really. I mean, like we said the tasks were pretty simple and the controls were pretty straight forward. Outside of just the initial part of coordinating the movement with the chairs, we got the hang of it really quickly and it was fine from there. Yeah, I agree as well, nothing particularly difficult about it.
Was it an enjoyable experience? Yeah, I mean considering that there wasn’t much we could do, even just moving around with the chair controls was really fun. _I think especially the fact that we had to do it together, it made it more interesting having to coordinate and that. _

Analysis:

Chair input was the key point of interest and made the game more enjoyable * Made the game more “dynamic”, improving the enjoyment the users felt from the game.
The dynamic of coordinating with a partner added to the enjoyment of the game * Affirms the requirement to have a pair based interaction requirements for the game

User pair #3:

How did you find the activities? It was a bit confusing at first because we didn’t know much about the game. Like I didn’t know we could break the wall just by walking into it twice. We thought we had to use a bomb to break the walls? But we got the hang of it after a bit. Yeah.
**What did you think about the chair controls? **The chair idea was cool, I’ve never played a game with chairs as the main control before. I agree but the chairs moved quite a bit which made it slower to control at times. I feel like if the chairs were more stable it would be a lot easier to control the players. Yep.
Did you have any difficulties completing the tasks? If yes, what were they? So like we said the chairs moved a bit too much each time we changed positions. Also it was a bit difficult to focus on the small screen while also changing positions. Oh yeah I couldn’t really see the screen that well either. Maybe if it was bigger it would be easier to play? Cause’ I couldn’t run to another chair without having to, sort of process where I was on the game screen.
Was it an enjoyable experience? What did you like most about it? I think we enjoyed playing it together, well I definitely did. No yeah, me too. The overall chair thing was really interesting and playing together was a lot more fun than what I’d imagine playing alone to be like. I’d want to see how the game turns out when it is actually complete. Same.

Analysis:

Although a bit confusing at first, the chair control mechanic was interesting and the user testers got the hang of it quickly
The collaborative aspect of the game was a big influencer on the positive opinions of the user testers
Physical chairs need improvement, such as the stability and comfort of them whilst being sat/jumped on
The size of the game screen should be bigger (i.e. incorporating a monitor) to accommodate the nature of the gameplay such as rapid physical movement

User pair #4:

How did you find the activities? They were pretty straight-forward to do. Yeah nothing too difficult.
What did you think about the chair controls? I think the chairs were pretty innovative. I could see myself playing this with a bunch of other friends. I haven’t played an arcade game in a long time but I’d imagine this to be one of those games near the dancing arrow game area. Like lots of physical movement and chaos.
Did you have any difficulties completing the tasks? If yes, what were they? Not really. Honestly it felt like a bit of a work out._ Yeah it was a bit tiring by the end of it._
Was it an enjoyable experience? What did you like most about it? Yeah it was fun. It felt chaotic but fun at the same time. It was enjoyable to play alongside someone else.

Analysis:

The collaborative aspect of the game is of high priority for the game * Chair mechanic is enjoyable to use in collaboration with another person