7. Testing Process

7.1 Testing Process Details

One of the first testing practices that was completed was running the built-in test cases located in the test directory. This was completed in order to establish a base-level for comparing changes in the future. We can use this as a benchmark for regression testing changes to the code because we know that out-of-the-box, all of the built-in test cases passed.

---

According to Robert Cecil Martin (2002 Agile Software Development: Principles, Patterns and Practices. Pearson Education. ISBN 0-13-597444-5), the ideal packages are either completely abstract and stable (I=0, A=1) or completely concrete and unstable (I=1, A=0). Using JDepend, we are able to identify that most packages fall outside of the acceptable range. The chart displaying this metric can be found under TestingMetrics. In fact, only 7 packages are marked as ideal by JDepend.

Using JDepend we can identify the packages with the Abstractness/Instability ratios that are most deviant from ideal. The packages that are the most in need of review are:

• common.io
• common.util
• common.model.pathfinding
• common
• common.i18n
• common.option
• common.io.sza
• commoon.resources

Upon further analysis we can see that the Ca number (how responsible this package is for the functionality of other packages) is highest for common.model, common.util, and common.io. This is good reason to focus in on these three packages for optimization of class usage. The reasoning for considering these packages is derived from the Dependency Inversion Principle (Robert C. Martin, 2003), which states that "high level modules should not depend on low level modules".

---

Another approach we took to testing the complexity of methods was by using Codecity. Codecity is a code visualization tool that indicates which methods can be problematic. More specifically, it measures how many methods have been declared, how many declared fields are apparent, and the amount of problem markers. Minimizing the amount of source code and maximizing efficiency is an integral aspect of development. Conveniently, Codecity can expose duplicate lines of code.

After running Codecity on our code, a Jacoco Test Coverage XML report was generated which measures the numbers of lines in the source code, number of branches, and how many lines/branches were not covered by tests.

---

Another further test metric was SourceForge Metrics. We focused in on the metric titled "Method Lines of Code" which identifies methods with large lines of code. The packages/classes with the worst ratios can be seen below:

Furthermore, the classes that need to be looked at are below with the package name above it's list of problem classes:

server.model

• ServerPlayer
• ServerColony
• ServerRegion

server.generator

• SimpleMapGenerator
• TerrainGenerator
• FreeColMapLoader

tools

• TranslationReport
• ForestMaker
• GenerateDocumentation
• RiverMaker

---

Using FindBugs, we were able to identify several issues rated as Scariest and Scary. These issues were given precedence over the Toubling of Of Concern bugs. These bugs can be seen below:

---

Using SonarLint, we were able to determine the following bugs:

---

Using JArchitect, we were able to view various code metrics for FreeCol, some of which uncovered potential quality issues affecting how the game runs.

Although this tool mostly reported common code violations such as "Types too big", "Methods too complex", and "Methods with too many parameters", hundreds of these errors in our code can cause problems and depending on how these methods and types are implemented, they could be fairly easy to resolve.

---

Using JaCoCo tests, we were able to run automated tests in the ANT build file. These tests didn't help much with detecting problems in the code because a lot of the tests failed, and the ones that did run passed.

---

Using Checkstyle, we were able to identify the following deviations from standard coding practices. As seen here, the main deviations were indentation issues.

The more detailed breakdown of the code style issues can be seen below:

These details show that while indentation issues were the prevalent, there were other common issues. Other common issues included the usage of unnecessary white space, too many capitalized letters, and empty lines being followed by a paragraph tag

---

Using PMD, we were able to identify more common programming flaws. Most notably, we were able to identify some Cyclomatic Complexity issues. Of these cyclomatic complexity issues, the highest was from the FreeCol.java class's handleArgs method as seen below:

A cyclomatic complexity of 60 for this method was very high, likely due to the amount of if statements inside of it. An issue was created to lower the complexity of this method and ultimately the complexity was lowered to 22.

7.2 Test Deliverables

Testing metrics have been placed in the TestingMetrics on the main repository branch.

7.3 Testing Tasks

In order to prepare for the testing activities, the specific tools were needed be installed via the eclipse marketplace, or via other means. Some tools that were not available on the eclipse marketplace needed to be installed by downloading the plug in and adding them to the plugin directory of the eclipse project. This was done on Mac by finding Eclipse in the finder, right clicking and clicking "Show package Contents." Once inside of this directory, the plugin folder is available for the plugins that were not available in the eclipse marketplace

7.4 Responsibilities

Each member of the group was responsible for running 2-3 tools to identify which areas of the code would receive attention. These results have been committed to the Testing Metrics folder in the main repository. Each team member then created issues resolved issues through commit messages.

7.5 Resources

Each group member identified their tools in their commits to the Testing Metrics directory.

7.6 Schedule

Initially, we wanted to establish a benchmark for future tests so we ran the built-in test cases included with the application. This will allow us to keep an eye to the future and use these results for regression testing.