Main Index

• Introduction
• Workflows
• Milestone Delivery Protocol
  • External Milestone
  • Internal Milestone
• Issues
  • Properties
  • Labels
  • Others
• Process for Quality Testing
  • Internal Quality Assurance
  • External Quality Assurance
  • External Testers
  • Issues Solving Procedure
• Tools and Software Used

Introduction

In this document we will gather all the information, protocols and procedures involving Quality Assurance for the project Order 227: Not a Step Back.

Workflows

The next are the workflows to follow within Quality Assurance sessions. In the first image, we can see how, each thursday meeting, if we are in the week previous to the deliver week, we will perform both internal and external QA Sessions. Otherwise, each time that a major design change is implemented (expected to be every week following how the calendar is organized and how the sprints are thought to have a build each week), we will do an internal QA Session (if we need a strong feedback we will prepare an extra External QA Session).

In the next workflow is shown the protocol to follow after each QA Session for each Bug Found at that session. As can be seen, after the report of the bug, the Q&A Lead will check it and, if confirmed, the solving stage will begin (it can be deferred if it’s not very important or critic, this is a decision of the Q&A Lead). During this solving phase, if, again, the bug lasts to be solved and is not very critic or important, it will be deferred. Otherwise, if solved, Q&A Lead will check it again and, if so, will be considered as fixed (if not, solving phase will keep going).

Milestone Delivery Protocol

In this section we will explain the procedures to follow as a team to deliver a milestone, both internal and external. As explained in the Production Plan Document, in the section of General Calendar , we plan to have a build to deliver two days before the official deadline to perform Q&A Sessions the week before, which means that we have to compile a test build a week before the official delivery.

The delivery for the official deadline will be considered as an External Milestone and the ones thought to deliver two days before, will be considered as Internal Milestone. The other builds won’t be considered as milestones, just as simple builds and sprints objectives. We will explain the delivery protocol according to whether the delivery is external or internal.

External Milestone

The external milestones will be our red bold lines, they will be after a week of solving problems after QA sessions plus two more days of error margin and extra details addition (this is, two days after each Internal Milestone). The protocol to follow for External Milestones will be the next:

1. Just after the Internal Milestone Delivery, the Code Lead will be responsible to commit a last, definitive and working code version in GitHub repository at main/master branch.

2. Now it begins safe time zone in which keep going with Q&A looking for major issues of the game while the other part of the team makes additions, fixes other bugs or, in general, completes tasks still not completed (if any).

3. The same day of the official deadline, at 16:00h, the Code Lead will be responsible to commit a definitive and working code version in GitHub repository at main/master branch with all the changes done during this period.

4. Q&A will pull the changes and will perform a last general test in which to see if everything works well and as expected.

   1. If it finds any fast-solvable problem, this one will be fastly fixed and the process will be restarted from point 3.
   2. Else, Q&A Lead will compile a release in order to publish it in Releases section of the project’s GitHub repository following the versioning parameters detailed in the Technical Specification Document.

5. In case that the game still has major problems, the procedure will begin again with a previous working version of the game still later to the Internal Milestone that had to be delivered two days before. If there’s still no working version, the Internal Milestone will be used as External too and this procedure will be repeated with that version (which should have no problems since it should have been already tested in the Internal Delivery Protocol, so if there is not much time, this one can be delivered without passing this process).

6. If necessary, will also publish a link in the releases folder of the university's campus. Also, aside the release, it will publish any other document required.

Internal Milestone

We are also going to delimitate internal milestones to have some reaction time in case we need more time to fix problems related with the game design or code and, in case there are no major problems, to have a secure time zone in which the deadline is already accomplished but in which to keep having time to add extra details or polishing. We must not forget that the main objective of these Internal Milestones is to ensure a working build two days before the official deadline to be covered.

As told above, our Internal MIlestone will be always two days before the official due date, and the procedure to follow for delivering will be similar than the one for External Milestones:

1. The day after finishing the last Q&A Session of the Q&A week, the Code Lead will be responsible to commit a definitive and working code version in GitHub repository at main/master branch.

2. Q&A will pull the changes and will perform a last general test in which to see if everything works well and as expected.

   1. If it finds any fast-solvable problem, this one will be fastly fixed and the process will be restarted (don’t overwhelm, there are still two more days).
   2. Else, Q&A Lead will compile a release in order to publish it in Releases section of the project’s GitHub repository following the versioning parameters detailed in the Technical Specification Document and aside will publish any other document required (if necessary, will also publish a link in the releases folder of the university's campus).

3. In case that the game still has major problems, the procedure will begin again with a previous working version of the game. Since during the Q&A week (before which we should have had a working testing build, so we have a “safe build”) is expected to have many builds from problem solving, there is a high probability on finding some previous working version of the game.

Issues

The issues that the testers find in the game (such as a bug) in any QA Session, will be reported using Github issues, just by clicking in “New Issue” and completing the BUG Issue template. Then the QA Lead will check the bug and decide if it is kept as an issue or rejected & deleted (it depends on if the bug can be reproduced, if it is very minimal or specific and not experience-breaking…).

Properties

A bug has several catalogable properties that can be classified in different ways following different parameters and variables. We have decided to differentiate them like this, as it’s a common way to do it:

• Title and description of the bug.
• Type of Bug: To which area of the videogame affects the bug (art, animation, audio…).
• Severity: Intensity of affection to the game system and functionality (crash, game stopper/slower, cosmetic).
• Steps to Reproduce the Bug: Step by step, how to reproduce the bug-
• Frequency: Which is the frequency with which the bug appears (often, sometimes, rarely, under specific circumstances).
• Conduct: What’s the expected and normal conduct of the game and which is the real one.
• Build: The build (and version) to which affects the bug.

In GitHub Issues, to report a bug, there’s a template with these parameters to fill and send (and therefore, to let us know that there is a bug). That template will look like this:

Labels

For each bug report, the QA Lead will assign one of the next labels to its report:

• Fixed: When an issue is marked as fixed by a fixer, QA Lead will check that issue again and test it. If the test is passed, the issue can be closed.
• Bug: Assigned to a report pointing out something that does not work in the game and needs an urgent fix (a bug).
• High Priority: When an issue becomes as much as important that needs more attention that the others. QA Lead will mark as it and immediately will assign a responsible to solve the issue.
• Extra Help: When a fixer cannot solve the issue and needs help of other fixer.
• Duplicated: The report already exists (reported more than once). The QA can decide, if it has nothing different from the other duplicates, to reject and delete the issue.
• No Valid: For any reason, the issue is marked as no valid.
• Won’t Fix: For any reason, the issue won’t be fixed.
• Need + Info: The fixer needs more information to solve the issue.
• Feature/Request: The issue is an enhancement proposal (to add a new feature or request to the game).

At each Thursday meeting, the QA Lead will make a review of the issues and its labels. The team will decide if issues marked as Fixed, Duplicated, No Valid, Won’t Fix or Need + Info are closed or kept there in case there’s any solving probability. For the cases of issues marked as Bug, High Priority, Extra Help and Feature/Request, in the same meeting the team will decide what to do (if assign someone, if reject them…).

Others

GitHub Issues allows to assign other standard labels or characteristics to each issue, like assign the issue to a milestone, to a project, to a member of the repository or to a label already existing. This is very automatic, meaning that, for example, users that can be assigned are the repository collaborators, the milestones that can be assigned are the ones already created in the Milestones section of GitHub… They will be used by the QA Lead to classify better the issues but are optional (if a tester that reports a bug assigns this, the QA Lead will review this markup to decide what to do).

Process for Quality Testing

In this section will be explained the organization and processes to perform Quality Testings within Quality Assurance Sessions.

Internal Quality Assurance

As told above, a week before each Internal Milestone, we will have to compile a testing build to properly begin the Q&A Week. That’s one of the internal builds that are thought. The fact is that we will ensure to have a build each week with the sprint’s working changes in order to correctly close that sprint. At each Thursday, the Q&A Lead, following the team’s updates on the project, will decide if the weekly build is subject of internal testing. If so, the next Saturday, the Code Lead will be responsible to commit a definitive and working code version in GitHub repository at main/master branch and Q&A Lead will compile a build that will share among teammates. On Sunday, an Internal QA Session will be performed 3 times during the day with a duration of 2 hours each. If the Q&A Lead sees that there is a major change, the whole team will be testing the game to report issues, but for minor changes, a minimum of two members will test the build. The objective of these sessions is to check if the features implemented work as expected and then try to find as many bugs as possible. When the session is finished, the QA Lead will be checking the issues reported and begin the Issues Solving Procedure. Finally, he will also decide if the build tested is pushed to have an External Quality Assurance session if necessary to ensure the external feedback.

External Quality Assurance

Regarding the external QA session, after the testers are chosen, they will be presented a brief description of the game with some basic information about the mechanics, then, they will fill a questionnaire to gather some information about them and its gamer profile before they dive into the QA Session, to classify better the testers’ output, and therefore, classify better the feedback with which we will be working with.

The External QA Sessions can be remote, by publishing an announce on social media asking for testers or presencial, performed in University with students and other interested testers there.

During the session itself, if the QA Lead is present, it will be taking notes about the players’ actions inside the game, and then asking some specific tasks to be performed in order to see how they are developed. This can help us on having knowledge on how the player feels and plays to improve its experience.

After the game sesion, the testers will be asked to answer a post-game questionnaire, to get feedback about the features of the game itself, its experience and opinions/feedback. After that, if they have any suggestion more, they can fill a Github issues template with the Feature/Request label, which will be discussed by the game designer later on. It’s important to incise, in each QA External Session, in the importance of reporting any request of modification or addition of any element of the game so we can ensure the fun factor of the game, which is very important.

On monday reunions, if the last week we did an external QA session, we will discuss about the feedback received and how are we going to act the next week.

Note: Questionnaires can change according to what we want to test at each session.

External Testers

The optimal profile of a tester of our game is a person between 16 and 40 years’ old that have played some RTS before and wants to feel challenged and overwhelmed to feel the calm after it. It can seek distraction, abnegation, intellectual challenge or just the feeling of disconnection.

They will perform the Pre-QA questionnaire for us to evaluate each profile and to see which are more profitable or subject to provide a feedback and to purpose improvements more adapted to the objectives of our game. Of course, no tester will be rejected from testing our game since we want to many people to test it, to find problems and to give us their personal opinion about how to make a better videogame, but the “optimal profile” will ensure better the accomplishment of our objectives and therefore, the game’s fit into its pillars.

Issues Solving Procedure

After each QA Session (both internal and external), the QA Lead will review the open issues in GitHub. He will classify them and assign them labels and move them to close issues if necessary. He will be in charge of keeping trace of the issues and communicate the team when an important issue needs immediate solution or if it can wait to other priorities to finish. At each meeting, he will be able to go over all issues and communicate the team its status in order to decide the procedure for each.

Tools and Software Used

The software we are going to be using is Github Issues to report the bug with all its properties, as it’s an easy way to utilize the software we already have. If some reported and important issue labeled as a bug is still there during 2 weeks, we will do a high priority task in HacknPlan assigned to code lead so we can solve it. For others, Q&A Lead will decide what to do with them.

---

Go Up or return Home