Unit Selection Sprint2: User Testing - UQdeco2800/2022-studio-3 GitHub Wiki

Propose for Testing

Why we start user testing

In the last user testing, we got the test results for the "interface, perform and design concept" of the infobox through interview and built up confidence in the project itself. At the same time, the result ananlysis also reveals what players are more after, which is usually a more detailed function.

For Sprint 2, we've put a hold on infobox development and shifted more attention to the effects of selecting units. Therefore, this user test will analyze the feel and visual experience of the selecting operation. Additionally, the newly added dynamic status feature in the infobox will be tested.

To the homepage of our Sprint 2's work, click here.

Test Tasks

What we want through the test

This test will start from the functions of different blocks, divided into two parts: Unit Selection and information box, pursue relatively independent problems.

For the development of the Unit Selection section, we need to know:

Whether the Semi-selection function can effectively let them know their actions (whether users will realize that their mouse is hovering over a certain unit)
When multiple units are selected, whether users can accurately select the target group they expect

For the functional extension of the information box, we expect:

Know if the disappearance and appearance of the infobox can increase the user's attention
Try the representation of infobox in different resolutions
Test the performance results of dynamic values
Discover the user's operating habits for different types of units (such as between buildings and soldiers)

Key Questions for Testing

How we did in this stage. Test methods gonna be rotated to obtain the most objective and comprehensive results.

Test Method: User-Testing

Giving Tasks, let user do themselves, therefore questions.

Time Limited Testing Tasks:

Try to let the infobox appear
List a unit's status
Try to select over 3 boxboys once
Try to split 6 boxboys into 3 groups
Select all the unit those shown on the screen and get the amount
Make themself aware of the selected building

Questions After the tasks:

Which task let you confuse a lot? Why?
Which task is the easiest for you to achieve, why do you think that's easy?
Do you think you will also trigger these functions when actually playing the game?
Between each of the two tasks, is there any difficulty that keeps stopping you from completing the task?
Any additional advices will recorded in this index.

Testing result

User#1:

Task finishing time:

Let the infobox appear: 5s
List Status: 5s
Select 3 boys: 7s
Split boys into 3 groups: 20s
Select All: 3s
Aware the selected unit: 15s

The tester used the selection box to frame the plural units very skillfully. During testing, he behaved as if he had really gotten used to the game. However, he got stuck on the last mission because its perception of the game's mechanics was a little different from what we expected.

Answers after tasks:

I'm quite not sure what's the awareness of selected units. Isn't that always the units under my mouse?
Select all - that's actually the same as task 3.
Yes, if I wanna win the game.
The selection feel might different into other resolutions.

User#2:

Task finishing time:

Let the infobox appear: 3s
List Status: 5s
Select 3 boys: 12s
Split boys into 3 groups: 30s
Select All: 6s
Aware the selected unit: 3s

This tester was very generous with the selection box - he pulled out huge rectangles. This triggers the view and camera move functions, causing it to waste some time. At the same time, when selecting some units, he seems to be a little powerless.

Answers after tasks:

While I tried to select all the units, the camera moved and reset the selection box. That's annoying.
It's quite easy to appear the infobox, as I clicked the base.
I don't think I'll select all the unit into the screen.
When I tried to select 3 boys and split groups, I always select 2 boys unexcepted.
Why the boxboy and forager are using different selecting method? That should be same.

User#3:

Task finishing time:

Let the infobox appear: 4s
List Status: 7s
Select 3 boys: 12s
Split boys into 3 groups: 11s
Select All: 5s
Aware the selected unit: 4s

He is used to clicking each target one by one to complete the functions of multiple selection and grouping. When I prompted him, he tried plural selection box and got used to it.

Answers after tasks:

I always choose 1 more than 3 boxboys while testing.
There's no hard work I think, but actually task 1 is the easiest.
Like Red Alert, I'll need to Split my amry into different enemies' basement. If there's no mutiple enemies, then task 4 won't work for me.
Same as Q1. I got unexcpted selection on other units when selection 1 unit.

User#5:

Task finishing time:

Let the infobox appear: 2s
List Status: 4s
Select 3 boys: 9s
Split boys into 3 groups: 18s
Select All: 5s
Aware the selected unit: 4s

The tester seems to have an easy time selecting units - both through single click and click & drag functionality. The tester seems to have a difficult time adjusting to the camera movement as it triggers without warning.

Answers after tasks:

The camera moves without warning. Accidental touches on the touchpad can lead the camera to zoom in and out explosively.
Selecting units and making the info box appear, its literally a single click job.
I'm not sure if I'll use the multi-select function a lot.
When I tried to select 1 boy but ended up selecting 3 and the building as well since they overlap.

Result Ananlysis & Conclusion

What did we learned/recognized? What will we do after this...

We found that most testers are not used to the selected mode of RTS. This could be due to unpopular gameplay, or inexperience of gaming.

At the same time, we found some minor flaws in the selection of units. When the units are too close, even if the user does not use the selection box, there will still be a problem that more than one unit is selected. This may be due to the duplication of the determinant of the unit.

When the infobox is spawned or destroyed, the tester's eyes do move to the lower left corner - where the infobox is located. Users do perceive changes to information based on dynamic vision.