Design User Testing Overview

Throughout the course of the project, user testing was performed consistently to ensure that our design aligned with the needs of our users. User tests were performed on students aged 18-23 with a range of gaming experience who are not apart of the DECO2800 course to avoid any biases.

Sprint 1: Interface User Design Testing

During Sprint 1, user testing was conducted to gather feedback on the design & functionality of the shop interface. To do this, users were given a set of tasks and questions to complete that mimicked the tasks that a player would undertake during gameplay. Following these tasks, general interviews were conducted to gather feedback on the overall design and look of the interface. For the purpose of this testing, it was assumed that the target audience was students in at least their second year of studying who were undertaking a degree in the technology field. Users were asked to do the following:

• Purchase an item. Outcomes: From the main category selection page, it was noted that there are no clear instructions for users to choose a category, therefore some users took a moment to familiarise themselves with the interface before clicking a category button. Once users progressed to the item selection page, there were no issues with navigation to make a purchase.
• Abandon your current purchase, navigate back, choose a different category & purchase another item. Outcomes: Since dummy icons were used in this sprint, the 'back' icon did not have any visual indicator of its purpose other than the placement - users noted this, which will be addressed in future sprints.
• Return to the main game. Outcomes: Similar to above, the top right corner placement of the back button signals to users the purpose of the button. However, a visual indication of this will be added in future sprints.

Users also undertook general interviews to give feedback on the overall design and look of the interface, and made the following observations which will be acted on in future sprints:

• More contrast (either colour or design) between icons and background would be beneficial in matching art style with the rest of the game
• In terms of player orientation, it would be helpful to include the category that the player is in once they have progressed to the item page
• There were also concerns around the amount of artefacts that would be up for sale, and if there were a lot that the carousel design would not be great to have to click.

Sprint 2: Interface refinements and UX Testing

Having implemented changes based on feedback from Sprint 1 such as the implementation of item navigation ('item 1/6'), user testing in Sprint 2 focused more on the user's experience when purchasing items and how these would fit into gameplay. Users were asked to perform tasks relating to the shop feature (navigational testing), with success measured by the ease of task completion, and time taken to complete the task.

Testing Plan

Whilst navigational testing takes place without directions or prompts, it was necessary in this scenario that users were given context about the game. Users were asked to complete the following tasks to provide insight to the usability of our interface:

1. Purchase an item that upgrades the player's attack.
2. Purchase an item that would produce the stone resource.
3. Your player's health is low. Purchase an item that would rectify this.

Observations

User 1: User 1 executed the first task with no obstacles. They navigated immediately to the 'equipment' icon, symbolised by a sword, and purchased the sword, for which the description indicates that the item increases the attack value. The user noted that the sword symbol made this easy to identify. For task 2, they hesitated when determining which category the item would be in, as they were not aware that the 'stone producing' resource would take the form of a building/quarry. After first navigating to 'equipment' & browsing the available items, they then navigated to 'buildings' where they were able to scroll and identify the correct item based on the description. The case was similar with task 3 - the user navigated first to 'equipment' but observed the items in this category were all weapons or similar. They then navigated to the 'artefacts' tab and were able to purchase the correct item.

User 2: User 2 commenced task 1 by browsing through each of the 3 categories & the items available. They noted that they were curious to see what items were available since it was their first time interacting with the shop feature, and wanted to investigate the feature before completing the required tasks. They also noted that the secondary navigation was helpful when browsing in this manner, as they were able to keep track of which category they were in when looking through the three categories in relatively quick succession. Having browsed the available items, the user completed tasks 1 & 3 seamlessly as they were already aware of the placement & categories of these types of objects. When completing task 2, the user noted that they were only aware that the item that produced stone was a quarry since they had seen it on the map when entering the shop feature, and otherwise would have assumed it was something you would purchase from the shop directly. As such, they observed that they would not immediately assume this type of resource would be under the 'building' category.

User 3: The third user firstly noted that they were confused by the different methods of currency (gold, wood & stone) that were displayed on the screen and were unsure which currency they were dealing with until they navigated to an item where the price was displayed in gold. They mentioned that the presence of the other two currencies was confusing if they had no purpose (note: no further context or clarification surrounding these was given to the user in order to observe how it affected their experience). User 3 had no difficulty in completing the first task, but also expressed similar opinions regarding navigation to the second and third task, in that their association between the item and the category was not instantaneous.

Sprint 3: Experience enhancements, interface refinements and UX Testing

Having implemented changes based on feedback from Sprint 1 and 2 and completing the finishing touches on shop functionality, Users were asked to perform tasks relating to the shop experience (User experience testing), with success measured by the ease of comprehension, subconscious connections and intuitive behaviour.

Multiple wireframes were created from the base design we finished within Sprint 2, however, they were iterated based on previous feedback provided and what the designer thought was in alignment with the purpose. It wasn't until there were hi-fidelity wireframes that true feedback and user testing could be completed.

Testing Plan

The ultimate goal of this testing was to see what the user automatically connected with the aesthetics of the hi-fidelity wireframes. Without using the platform, how did they feel, what was the overall experience and what were the subconscious triggers.
Users were asked to answer the following connections to allow insight to the experience of our interface:

1. What is the first thing you notice? (Aesthetics focused)
2. What is the emotional experience of the design?
3. what do you believe is the purpose of what is in front of you?

Based on the three images below:

Key Findings

User 1: Immediately liked wireframe 3 the best. Begin to be excited with this design, wanting to actually use it. The buttons popped enough that there was no hesitation in knowing what they were for, and the physical shop design drew in the player. It made them feel like they could have walked into the shop, the flowerpots on either side seem to direct the user on where to go. Wireframe 2 lost them with the blue and didn't make the emotional connection of a shop. While they liked wireframe 1, it felt like there was too much going on. Suggested a combination of Wireframe 1 and 3.

User 2: The shop title is very prominent within wireframe 1, due to the red roof being the eye-catching feature, this draws attention straight away. The emotional experience is that it looks like a shop front, with "cute" flowers at the front. However, the flowers do crowd the bottom of the screen just a bit. Almost too much is going on here, as they do clash with the buttons. The user had to take a moment to assess and think about where everything was and understand what the text/button's purposes were. The shop functionality was very quickly understood, just distracting.

User 3: Connected with the flowerpots straight away. Made a mention of how the features were "typical basic store features". Didn't like the red roof on either wireframe 1 or 3. Thought the blue in Wireframe 2 suits the Atlantis theme, however, unsure of the functionality/purpose of what they were meant to do. They liked the blue and thought it felt very aesthetic and connected to the game. The flowers suited the screen, however, there needed to be more of a contrast for the middle test bubble, as it fades into the flowers a bit. They did figure out the purpose, but it did take longer to make the connection.

Final Design decisions

The final design decisions were:

• Two flower pots on either side, with contrast for the buttons so they don't fade into the background.
• A red roof, as this is the most typical shop design, is easily recognizable, and draws attention to the shop title.
• There has to be some form of blue within the design to connect slightly to the theme, as the design in very simple the flowers look to be the pop of colour needed to make that connection.
• The roof will not be across the full screen, as this clutters the space.
• The coding team were spoken to in regard to what design for the roof out of Wireframe 1 and 3 would be the most realistic to create. It was settled on Wireframe 1.

Final Shop Design

Based on the feedback provided from the user testing, and the internal conversations among the team to deem what was feasible, the above design was created as the shop aesthetic design. The roof was kept from Wireframe 1, however, the shop sign was removed to allow for if the game was played on lots of different screen types there would be no distorting of the design. The two flower pots were incorporated, with black bases to blend into the border just a bit, but have the pop of blue to tie into the theme. The background colour was darkened to create more of a contrast between all the gold on the screen. This allows for those buttons' purpose to be immediately taken in without confusion or delay. The detail in the roof were taken away to limit any potential distraction, and to stick with the overall design of the game.

Sprint 4: Shop & Inventory Functionality Refinements

As part of the polishing process in sprint 4, we wanted to ensure that the process of purchasing & utilising game items was seamless and logical for users, and aligned with any changes made by other teams to other features over the course of the project. As such, similar user tests were performed to those in sprint 2, whereby users were asked to complete some specific tasks accompanied by open-ended interview questions. A mix of the following questions and prompts were used for each test to guide natural conversation:

• How would you go about purchasing an item?
• Where would you expect to access the items you have purchased?
• What visual cues indicate a shop and/or inventory?
• Which parts of the shop/inventory interface stand out to you the most?
• How does the process of buying & placing a building differ from other games you have played?
• How would you go about getting another tower & placing it on the map?
• If you had to change one aspect of the shop & inventory functionality/design, what would it be and why?

Observations

User 1 User 1 was prompted specifically on the visuals of the shop and inventory. They indicated that the design of the shop icon wasn't clear and looked too similar to a 'suitcase' which would imply an inventory feature. They liked that the two popups had a similar design and noted that this indicated to them that items in the shop would be available in the inventory.

User 2 User 2 was prompted on the interaction between the shop & inventory. They were asked to buy items from each of the categories & use these items in gameplay whilst speaking through the process as they went through. This user noted that there was a logical link between the equipment and artefact categories in the shop & the usage of these in the inventory, however they were confused by the placement of the buildings as the purchasing of buildings was the same as other items but the usage or placement of buildings differed, which they found confusing. They mentioned that they expected a 'build mode' or similar, where all aspects of building acquisition & usage, including purchasing, would be located in a central popup. When asked why they thought this, they indicated that their previous gaming experience (although limited) had this type of setup for a similar function.

User 3 User 3 was asked to purchase a building and place it on the map. This user located & purchased the building successfully, but showed some delay when deciding how they would go about placing it. They clicked back through the shop & tried various keyboard commands before entering the inventory, which indicates that the location of buildings in the inventory is not intuitive.

Actions

Based on the feedback from these users, the most notable change to make was the location of where buildings were stored in the interface. As per the suggestion from user 2, all building-related actions will take place in a new 'building' popup, symbolized by a hammer icon, and will include both the purchase and placement of buildings. This approach allows users to navigate to a central place when trying to achieve a specific goal ie. if they intend to add a building to their map, all related actions can occur within one popup. This also reduces mental load for users & allows them to devote more time & thought into their game strategy. The icon to access the shop was also changed from a 'shopping bag' to a 'dollar sign' as users indicated that the original shopping bag icon looked too similar to an inventory icon. The dollar sign is far more distinctive whilst keeping in line with the theme & functionality.

Sprint 4: Shop Redesign User Testing

Due to feedback that was provided by the UX team, in order to make the shop more aligned to the design of the game some changes had to be implemented. However, to ensure that the feedback was also reflected by the user’s experience, we completed some testing to confirm this. The Ux team said that the background needed to not be a solid colour and not red. The shop title needed to be moved into the middle and it needed to feel more like a shop.

A B Testing was followed (altered to include a third option), to test specific design features of the shop.

Step 1: User were shown two images, one of the original design and one in the middle and were told to pick the design they thought was a shop.

• 12 out of 15 chose the red design in the middle.
• 3 out of 15 chose the red design on the side.
• 7 said there should be a sign out the front (in the flower pots)
• 10 said the design in the middle felt like they were walking through the flowers into the shop

Actions: The actions taken from this were to update the design to have the red shade of the shop front moved into the middle and all future design changes to have that in mind. This allows the users to have a central focus on the popup and for it to mimic a 2D shop front. The feedback about including a sign in the flower pot was something that hadn't been considered, but due to the nature of the comments, as they were unprompted, supports the idea that something should be included. As it is only a small feature, more user testing won't be necessary, as long as it is something that directly supports shopping and buying items from the store.

Step 2: Users were presented 3 different designs in terms of encouraging shading and depth into the design. They were encouraged to select one that presented the best idea of a shop shade. This was based on the feedback that the shop needed more artistic depth in the design.

• 14 out of 15 preferred design 3.
• 1 out of 15 preferred design 2.
• 15 out of 15 said that the third design was a typical game shop design
• 1 out of 15 said that design 3 was overcomplicated
• 13 out of 15 thought design 1 was too simplisitc
• 12 out of 15 "liked" design 2 but didn't prefer it

Design 1:

Design 2:

Design 3:

Actions: From the user testing in this section, there were very conclusive results. Design 3 would be the chosen design for the shop shade. There was an outlier in the data, but due to the polarising numbers in support of the design, their data can be ignored. Design 3 was the third iteration in attempting to come up with a different design, and was used to highlight what a hyper-designed shop shade would look like, to see if perhaps that was what the overall design was missing. This was shown to be correct.

Step 3: The users were provided the favoured design from the last user testing in different colours to allow the user to determine which one they intuitively decide and connect with as a shop. They were offered a red, green and blue option.

Out of 15 the stats returned:

• 9 out of 15 preferred red
• 1 out of 15 preferred green
• 5 out of 15 preferred blue
• 12 out of 15 disliked the green the most
• 11 out 15 said red was the most obvious shop colour
• 4 out of 15 said blue was the best shop colour because it connected to the theme

Design 1:

Design 2:

Design 3:

Actions: There was an overwhelming favourite with the red, which confirms what was found in previous user testing earlier in the design phases. In the previous user testing, it showed that red was the unconscious choice for people to connect buildings to stores. The design choice for red will, therefore, stay the same with no impact on the current design. While this clashes with the UX committee's feedback, their feedback back does include bias, whereas user testing does not. Especially considering the medium-sized sample pool that was used to participate in the design testing.

Final design for the Shop (end of sprint 4)

Sprint 4: Placing buildings User Testing

As part of the user testing process for the functionality of placing a building with the new hammer icon and pop up, it was decided to allow the users unmoderated testing to see whether it was intuitive to use and complete. It was determines that the building pop was needed based on previous user testing completed. The minimal involvement was to allow the user to make the decisions they would normally if they had just started the game of their own accord, the only information provided being:

• Please buy a building
• Please place a building on the map

Observations from three testers:

• 3 out 3 could easily find the shop and buy a building o The new logo for the shop was easily recognised without hesitation
• 2 out of three when straight to the hammer icon o Figured it out relatively quickly on how to use o Placing the building had no complications
• 1 out 3 went to the inventory first o Checked if they had the building and what else they had o Quickly saw they had to go somewhere else to place the building o Tried the hammer icon next, and was easily able to place the building down on the map
• The icons were specific but allowed the players to interact with the game without being obvious with words.
• The functionality aspect of placing the building was successfully demonstrated

Actions: As this user testing was based on making sure that the correct functionality and actions were taken by the user, this can be called a success. The user’s experience is enhanced by not only supporting their natural tendencies but things they instinctively look for. Examples are the dollar sign and the hammer, which are the icons for the different functions.

Sprint 4: Artefact Redesign User Testing

Because the shop artefacts were already created, this time around we used user testing to help redesign and improve the items’ overall aesthetics and to better fit into the theme. Users were asked to first identify which designs they’d like to be improved. Most users reported the following 6 items:

• Axe
• Trident
• Sword
• Bow
• Potion
• Bed

Old designs

Key findings:

Axe

• Design too simplistic
• Looks like every basic axe
• Looks worse than main weapon – why get this?
• Very small and short – want an axe with more presence

Trident

• Most users thought it was a candle, not a trident
• End of blade not sharp
• Handle too basic
• Overall want more interesting design

Sword

• Did not look like pixel art
• Way too short & small
• More like a miniature knife
• No incentive to buy other than stats

Bow

• Size appears decent
• Expect a more complex design
• Boring – bring more colour into it

Bed

• Too simplistic
• Inconsistent design – doesn’t look like pixel art
• Perspective is not ideal – would prefer isometric

Health Potion

• Does look like a potion, but not a health one
• The flask looks jagged and rough – aim for more consistent and reconsider a rounder shape
• Users were given outline of potion and asked to colour it in – 5/5 people chose some shade of red

New Designs

This feedback gave us insight into what changes need to be made to each design. The user testing gave us the direction needed to begin re-working the old designs. The design process based on user feedback for each item can be found in the individual links below:

• Shop - New Axe Design Process
• Shop - New Trident Design Process
• Shop - New Sword Design Process
• Shop - New Bow Design Process
• Shop - New Health Potion Design Process
• Shop - New Bed Design Process

Potential Future Iterations

Sprint 1: Potential Future Iterations

As the shop grows in use, there is a larger opportunity to grow how player interacts with this feature and its ability to support the game play.

This could include:

• A bigger focus on integrated theme design
• Adding a page into the categories internal displays where there is a gallery of all the potential items, not just the carousel view which could be used to highlight specific items.
• Having things disappear from the shop based on how much money the user has
• Having different versions of the same thing e.g. 1 x wood, 5 x wood, 100 x wood
• Having more clear titles on the different pages and descriptions of items
• Having a total of how much money the user will have after they buy the item
• A way to upgrade the different items

It is currently a very basic shop, which means there are a lot of opportunities to help support the user and have an impact on the game, and help change the ‘unfinished’ people that we received from user testing. The idea for comes improvement for the shop is focused on what potential iterations will help benefit the player not just give functionality to the store. This will contribute to the store being used often, and with intent, supporting the user through every stage.

However, before we can think about potential benefits, we need to condense the different design styles and make sure they are the same as the games and are cohesive with the design style chosen. This was very evident in the user testing, which could mean that the store changes colour to represent a more consistent theme across the game, or more ‘store’ typical features such as the titles not just appearing flat on the store page but hanging in a wooden board. Ways that could make the store look more ‘physical’ for the player and integrated into the user experience.

In later sprints, based on time and ability we could also integrate the player’s experienced threat of water into the store experience. This could mean having what looks like water slowly filling up the store, until they either can’t buy from the store, or it is just a background effect, based on how well they are going in the game.

Sprint 2: Potential Future Iterations

It was apparent during user testing that the categories were not a sufficiently accurate representation of the items they contained, particularly to players that had no experience with the game. The following features are suggestions for what should be implemented in Sprint 3 to rectify this:

• Introduction of a user guide/instruction manual that is accessible during gameplay that describes the contents of each category. OR
• Adjustment of category names & images to better represent the items contained in that category. In doing this, it should also be considered whether the items are grouped appropriately. OR
• A text description on the category page with a succinct description of the category and its contents.

Sprint 3: Potential Future Iterations

The design aspect of the Shop has been finalised so there will be no more future iterations for this task.

If there was more time, the shop aesthetic should be more interactive, so there is a focus on functionality and purpose but also making the store fun to use for the user. Making the experience a primary focus as well.

The functionality is down pat, the users like the process and the feedback is positive.