Methodology

Overview

In designing the shopkeeper both character variations as well as two design directions will be tested to ensure the final design of the shopkeeper is appropriate and effective. Overall this will be done through a combination of broad user feedback from questionnaires on designs as well as A/B testing. Outside of implementation these are the two major methods that designs can be tested and therefore both will be utilised to maximise feedback.

Justification

Before the final designs for the shopkeeper are created testing will be done on the initial variants of the design both to help determine which character type best fits the role as well as to gauge the effectiveness for the character designs in getting across the intended character traits of the shopkeeper. This is essential as all players will be required to interact with the shop keeper as well as having to see them every time they pass between levels, therefore getting an effective and appropriate design is very important in order to not break the players immersion and bring them out of the world of the game.

Testing Process

User Brief

Before testing the user will be told about how they will be giving feedback on the designs for a game that is pixel art. They will also be told about aspects of the game world and specific design purposes but that will be in later testing.

User Tasks

The Users tasks will involve showing the player designs and requesting their feedback on them (as well as asking their preferences between alternatives). In order to ensure actionable/useful answers are given due to the very freeform nature of such testing, the tester will ask specific questions. These questions will be asked in stages to get user assumptions based on the designs both in and out of context. Though the in context information will be more actionable, the general information will help in developing the final design in an appropriate way.

Before explaining the world of the game the user will be asked the following question on each image set:

• What is your opinion on the character based on the design? Are they strong/weak? Are they frightened/calm? what kind of person do you think they are? (why do you think this)
• What kind of world do they come from?
• If you look at the alternative clothing what changes about your perception?
• Do the smaller versions and detailed versions match each other?

Finally the tester will explain the character is supposed to be a shopkeeper in the world an the scenario of the game world itself and will be asked the following final questions:

• Do you think this character fits into this world?
• Do you think either clothing version fits into the game better?
• Do you prefer either design between the two options? why?
• Given the current player character designs (detailed & undetailed) do you think any of these designs feel different / out of place?

Illustrations Set #1: Veteran Shopkeeper:

Illustration Set #2: Mystery Shopkeeper

Testing

Participant #1: Significant feedback

Veteran:

• Strong, loud, humorous, approachable (intimidating to shy people)
• A working person -> medieval era
• Feels like the low pixel design looks like the same character -> similar thoughts
• Shirt only: Less friendly -> not happy, Looks skinner / less strong, creepier, not friendly (prefers with apron)
• Shirt only: Pixel art didn't fit as it was more friendly than the detailed version
• Too medieval to fit in the apocalyptic scenario (for both)

Mystery:

• Calm, but could be scary if you annoyed them
• More mysterious / shady vibe (just knows things and doesn't explain how)
• Looks like he comes from a world with magic
• Seems other worldly
• Noticed different perspective but thought it worked from game scenario perspective
• Modern coat: doesn't seem to match him as much -> no different assumptions
• Fits okay into a world were light is disappearing
• Cloak version fits into the setting better
• Preferred this design for the shopkeeper overall (with cloak) -> better in world, less generic, more interesting

Comparing to the already existing game

• Both work together
• Veteran version fits more objectively as they are both people but still looks too non-modern
• Mystery guy fits less but they felt like that works as he would be supposed to feel more out of place and mysterious
• Felt the detailed design needed more shading to better fit the player character

Participant #2: Significant feedback

Veteran

• Farmer person -> hard worker, manual labour
• Completely alone or has a full family
• More of an old time-y character
• For small one it is a little weird to go between a real face and a pixelated face
• Prefers wearing an apron by a lot -> too much of one colour also he looks to pale
• Works as a kind of shop keeper

Mystery

• Detective, ghost-y vibes, kind of creepy
• Don't know what they are thinking but also knows everything
• Wouldn't explain themselves
• Less modern, old time detective -> live in the night
• Mini version makes sense -> thinks the hat looks a little different (more broad)
• Likes the collar/buttons/pocket on the modern coat, still likes outline and crease though
• Thinks the coat needs to be more even
• Hat for the small one doesn't look round enough
• Likes the buttons on the little one, gives the character more shape
• Would work as shop keeper in the games scenario -> likes that he is shadowy with an apocalyptic no light

Comparing to the already existing game

• Shape build and outline of veteran guy matches the player character
• Mystery guy would fit in the same game -> think he fits more (post apocalypse)
• Preferred the mystery guy overall -> liked the coat version but didn't like the flat colour

Participant #3: Significant feedback

Veteran

• Could see in a few roles
• The little one and the larger pixel one feel different
• Low pixel version feels more medieval
• Shopkeeper in the medieval times or modern labourer for the more detailed one
• Without the apron he feels too bland -> takes away the character
• The uniqueness mostly comes from the beard & apron
• Works as a shopkeeper -> Would sell swords and weapons
• An occasional person you run into making trades, more respectable

Mystery

• Shady guy in an alley -> reminds me of the guy from thief
• Physically weak but probably has some magic (likely a mage) -> strong overall
• More of a fantasy character -> high pixel one could be a dated-semi modern
• Call of Cthulhu style detective
• Thinks the small one works for the same character
• The coat version gives off the same energy but more detailed
• Prefers the button one cause they like the button detail (especially on the smaller one)
• Works as a shopkeeper -> would likely give you items and abilities
• Likely steals from corpses to get the inventory -> like patches the hyena from dark souls

Comparison to player character

• The low pixel versions all fit together
• Art style is generally consistent - just more
• Would be okay to use either, both are fun characters
• Would use button up coat or apron

Evaluation of Comments

Common Feedback that is significant for changing/improving to make the final design

Some didn't have a strong preference but most users preferred the shadowy design, either feeling it worked well with the setting (of a world being absorbed by darkness) or had more 'fun' theorising on the type of character they would be and got excited about the character type (some also mentioned it makes more sense to sell abilities and more varied types of items if the character was the more mysterious version).

• Will pursue the mysterious shopkeeper version of the design

All users had an issue with the veteran design without the apron, felt he was a lot less friendly or lacked character.

• When further working on the design need to ensure there is a significant variation in colour / shading to ensure there is more interesting / eye-catching elements
• Need to avoid 'generic' design elements as people tended to not identify or show interest in such characters (since it is an interactable character and not a background character)

When asked on a preference for the mystery shopkeeper design users tended to like the version using creases in the coat but multiple users mentioned that the button part of the design was nice

• Could incorporate both the crease elements and the buttons into a single design

When comparing to the already existing player character a users mentioned that the player character seemed more detailed (with more shading)

• In the final design using more than two colours for the shading and block colour of elements
• Could add more elements to the design, specifically in the coat

Additionally, a user mentioned on the small design that the hat of the mysterious designed seemed to not fit the hat on the detailed version, looked different.

• Could bring the brim of the hat out more and make it more round
• Could add more shades to the hats shading to make it more rounded

Some peoples initial assumptions though the mystery shopkeeper was some kind of detective / investigator. Not a direct problem but could work on the design to make this less of an assumption so that it is more clear.

• The coat seemed to be more reminiscent of a detectives coat so could do some modifications to bring it further away (likely related to the collar)

Responses to Feedback

• Will pursue the mysterious shopkeeper version of the design
• Need to make sure the design has some form of eye-catching elements that are more distinct and signify the character as important (currently keeping the shadowed face and large hat would likely achieve this)
• Need to avoid 'generic' design elements as people tended to not identify or show interest in such characters (since it is an
• Trying to incorporate both the crease elements and the buttons into a single design
• Using more than two colours for the shading and block colour of elements
• For the low pixel design, bring the brim of the hat out more to make it more round and adding a third shading colour
• Could add more shades to the hats shading to make it more rounded
• Testing different versions of the coat collar that are less stereotypically detective

Evaluation of Methods

Overall the testing was quite effective in giving guidance on what to do in developing the design as well as picking the version of the shopkeeper character to go with. It was overall pretty effective to use multiple versions of the design in A/B testing to get more feedback rather than only showing a single version. However, in order to improve the testing it could have been good to show the designs in the context of the shop screen or in the game world itself to give better insight as to whether the character fit and how they were perceived in the game. Additionally, in testing the design it would have been helpful to show different colour schemes for the characters to get an idea on the appropriateness of the colours as well as if they fit into the game overall.