Unit Selection & Spell System: User Testing - UQdeco2800/2022-studio-3 GitHub Wiki

Propose for Testing

Why we start user testing

The development of Sprint 3 is a relatively independent and different chapter, but there is also parts of refinement in terms of our previous work which needed to be linked through the achieved conclusions from user testing done last time.

The purpose of this user testing is to confirm the completeness and user satisfaction of information-box and unit selection development, so that the development team can confirm the completion of this part.

This was followed by the first test of the new functionality, which is the spell system, which required a fresh interview on the design concept, presentation and functionality to get feedback that could be used in the next development phase.

Click here to the page for Unit Selection's Work in sprint 3, and click here to find our Spell system's design.

Test Tasks

What we want through the test

The main body of this test will be divided into two parts: Unit Selection and Spell System, which use different evaluation methods.

For the Unit Selection test, we were eager to know:

User satisfaction with redesign of images, sizes and UI.
User's operating habits and feedback after unit selection is compatible with worker units.
If users think this system has any flaws/bugs or areas of improvement that we can fix.

As the Spell system is a newly developed system, we need to recognize:

Are there any flaws, gap or salient points in the design concept of this system, and what are the elements that users can identify with?
Does the performance of this design implemented into the game meet the expectations of users (after knowing this system)?
How is the functionality of the in-game design, and how does the operation feel, and does it meet the user's operating habits?
Any inspiration, feedback, suggestions for this system will be added as new design concepts worth considering.

Key Questions for Testing

How we did in this stage. Test methods gonna be rotated to obtain the most objective and comprehensive results.

Unit Selection Testing Method: User-Testing

Giving Tasks, let user do themselves, therefore questions.

Users will be given a time-limited test to try to understand how well the new features of the system perform. The tasks are as follows:

Try to manipulate workers to collect resources.
Try to select multiple different types of workers to collect two resources at once.
Discover their other art designs (avatars)
Divide all the different types of workers in the map into different types by selecting and arranging

For each task completed, the tester was recorded the time it took to complete the task. After all is complete, they will be asked the following questions:

Have you experienced functional iteration and progress during the operation?
What elements greatly hindered your process of completing the task?
Do you think the development of this system is complete? If not, what else could be worth improving?

Spell System Testing Method: Interview

The interviewees will be consistent with the user testing part of the previous part, which is to satisfy the interviewees' basic understanding of the game settings and gameplay. The questions for the Spell System are as follows:

What do you think about adding this kind of combat system to RTS games?

It is used to obtain the user's mental expectation of spell system.

How do you think the system will perform in the current development period? How is it different from the spell you know from other games?

Used to know how satisfied users are with our development results, and how we fit into mainstream games.

While watching spell in action, what do you think of its interaction mode and flow?

It is used to obtain the user's satisfaction and psychological expectations in the operation and interaction part.

Do you think the system has higher room for improvement? In terms of design and functionality, what else do you think is missing?

Get feedback.

Testing result

User 1

Testing result for Unit Selection:

35s
1m4s
5s
25s

This user behaved like a duck to water during the operation, but hesitated for a while when trying to select a unit for the first time. He was very irritable when the worker unit was affected by the weather system.

Answer 1: Yes, I do find that all friendly units choose the same kind of selection logic...but I don't find that they have a highlight effect to represent pre-selection or something which indicated which unit I want to select, which is slightly different from the selection effect of buildings.
Answer 2: I don't know who designed the effect of the weather system... I don't mind the screen flickering experience, its deceleration effect even exists in most weather. This makes the unit move much slower than expected.
Answer 3: I would say that it basically meets the requirements in its due functions, but still needs to be polished in the details (such as the highlight effects of workers). In short, acceptable.

Answers for Spell System's Interview:

Well, like the super weapon system in Red Alert, I don't think he shouldn't be in this game. In other words, well-designed. It's boring only when the soldiers fight each other.
It's okay, it's justified. Among them, I really like the design of the buttons, the feeling of charging the battery is very cool. It would be better if some special effects could be added to the GUI itself.
My expectation is a full-screen attack, or an attack similar to some of the specified ranges in LOL. In fact, its performance is not bad, but the compatibility of special effects and expressions is a little worse.
If Spell's cooldown interval was only 10 seconds, the game would seem to have no difficulty. I would recommend redesigning its release conditions.

User 2

Testing result for Unit Selection:

30s
1m28s
1s
3m2s

The tester would subconsciously check the pop-up information=box when selecting a target, so he quickly found the unit's picture. But this test triggered a bug where the worker was stuck inside the building, so he restarted the game and finished the test a second time.

Answer1: Nothing special because they still operate by right-clicking on the designated destination. But their support for plural selection is really an upgrade.
Answer2: As you have seen, there are bugs that prevent units from successfully completing their run even if they are selected. Considering the unit selection itself, maybe the selection method can be diversified (remove the basic rectangular selection, you can also add circular selection, polygons, etc.).
Answer3: Maybe adding sound effects will make it more clear to me that I'm being chosen.

Answers for Spell System's Interview:

I play relatively few games, and it is difficult to understand the difference between it and conventional attack methods. But for me, having an extra special effect showcase is enough.
So, spell behaves more like a passive defense than an active attack, right? Because it can only resist enemies close to the base, but cannot kill enemies farther away. Either way, I think its performance is still obvious.
As mentioned in the previous question, I may have expected more scoped attacks. While soldiers might do it too, the presence of spell is the icing on the cake.
At present, I may be more looking forward to the improvement of the animation. In addition, maybe there will be voice or more frames, special effects and particles to enhance the visual perception?

User 3

Testing result for Unit Selection:

40s
1m20s
25s
1m21s

The tester completed the task with all her attention on the moving object and most other interactions ignored. This caused her to completely lose sight of the starting point when completing the third mission. Maybe it's a player who behaves as an example.

*Answer1: Only check is the new feature that is really added. There is no obvious iteration in terms of highlights and information display.

*Answer2: I think everything about the game is in order, it's not chaotic so that my gaming experience is quite normal.

Answer3: The unit selection function itself does not carry a large amount, it is more similar to an interface. So I would say that this function is enough here.

Answers for Spell System's Interview:

This is probably the screen I look forward to the most when I play the game. In my impression, spell always represents the climax and highlight of the game.
I have to say, you guys are really crazy about spell feature design. It is obviously too strong, and even eclipses other attack methods. However, the lack of relatively simple special effects and sound effects does not match its strength, and I hope it can be changed.
If it's just a self-defense device, then the intensity is barely acceptable. But its animation playback is a bit dull and I would suggest adding some shaker effects.
Adjust the value. Or restrict its release conditions. If the game is expected to have difficulty and gameplay, you better redesign it.

Result Analysis & Conclusion

Unit Selection Conclusion:

Aside from players with special behaviors (of course, they will continue to pay attention if time permits), most testers are still not used to the game operation mode of RTS. This may be due to its relatively unpopular gameplay genre. We missed the thinking about how our features are designed to accommodate players in other modes from this social perspective.

Also, this feature isn't really the point of user testing that gets their attention. That is, the functionality we develop is an interface to serve other objects that are more "visual experience and interaction", like buildings, units, resources.

The user's evaluation of this interface is usually only on the details, and will not give more suggestions for breaking the normal function. In other words, this function has already completed the task of satisfying the interaction of other objects. What's more important to pay attention to is the details of its interaction, such as highlights and sound effects.

Spell System Conclusion:

The first question shows the mental expectations and acceptance of the spell system among the respondents. In the user's perception, spell should be a low-frequency, high-power, designed with all special effects, and the climax of the game process.

We then obtained an evaluation of the results we made. We got low marks for "Low Frequency" and "Special Effects Design". This may be due to time and resource constraints, which caused us to not develop the "resource accumulation power bar" function as planned.

Alternatively, it's relatively easy to control frequency and power by limiting its cooldown. And the special effects design is as we were exposed during the development process, the relative lack of frame rate, response time and details make it more similar to the usual attack effect.

The final question point is whether we have more room for improvement. The previous questions have verified that our finished product satisfies the theory in the design concept and most of the user's mental expectations. What needs to be improved is the numerical control of the system and the processing of details. Perhaps, we can change more spell styles (consider more power of the Greek gods), redesign the animation frames, insert sound effects and vibration effects, and even break through its attack range and let the user decide where to shoot.

Therefore, I will propose a hypothetical scheme for this result analysis, that is, to design two sets of spell casting modes. It might be named "Attack Mode" and "Defensive Mode". Currently appearing in the game, the spell that performs range spikes around the base is defensive, while the attack mode can specify targets within the map range for fixed-point strikes. This is undoubtedly a design to expand the diversity and interactive experience of the system, and it is worth discussing in the later Sprint 4 development process.