Tutorial 6.1.1: Code Coverage - McGill-ECSE429-Winter2022/tutorials GitHub Wiki

1. Introduction

Code coverage is a metric that can help you understand how much of your source is tested. It's a very useful metric that can help you assess the quality of your test suite, and we will see here how you can get started with your projects.
Code coverage tools will use one or more criteria to determine how your code was exercised or not during the execution of your test suite. The common metrics that you might see mentioned in your coverage reports include:

  • Function coverage: how many of the functions defined have been called.
  • Statement coverage: how many of the statements in the program have been executed.
  • Branches coverage: how many of the branches of the control structures (if statements for instance) have been executed.
  • Condition coverage: how many of the boolean sub-expressions have been tested for a true and a false value.
  • Line coverage: how many of lines of source code have been tested.

2. Code Coverage Tool: Jacoco

Jacoco is an open-source project, which can be used to check production code for test code coverage. It creates reports and integrates well with IDEs like the Eclipse IDE. Integration is also available for other IDEs and continuous integration environments. So there are also Gradle, SonarQube and Jenkins plugins to make these code coverage checks outside the IDE and therefore globally available to the development team.

Installation for Eclipse IDE

Please follow the steps mentioned here.

3. Running Code Coverage

Before we start looking at JaCoCo's code coverage capabilities, we need to have a code sample. Here's a simple Java function that checks whether a string reads the same backwards and forward: 

public boolean isPalindrome(String inputString) {
  if (inputString.length() == 0) {
      return true;
  } else {
      char firstChar = inputString.charAt(0);
      char lastChar = inputString.charAt(inputString.length() - 1);
      String mid = inputString.substring(1, inputString.length() - 1);
      return (firstChar == lastChar) && isPalindrome(mid);
  } 
}

Now all we need is a simple JUnit test:

@Test
public void whenEmptyString_thenAccept() {
    Palindrome palindromeTester = new Palindrome();
    assertTrue(palindromeTester.isPalindrome(""));
}

Now we can use the jacoco:report in order to generate readable code coverage reports in several formats, like HTML, CSV, and XML.

4. Code Coverage Report Analysis

Following insights in the report, we can drill through a more detailed view for each Java class:
Our report shows 21% instructions coverage, 17% branches coverage, 3/5 for cyclomatic complexity, and so on. The 38 instructions shown by JaCoCo in the report refer to the byte code instructions, as opposed to ordinary Java code instructions. JaCoCo reports help us visually analyze code coverage by using diamonds with colors for branches, and background colors for lines


  • Red diamond means that no branches have been exercised during the test phase.
  • Yellow diamond shows that the code is partially covered – some branches have not been exercised.
  • Green diamond means that all branches have been exercised during the test.


JaCoCo mainly provides three important metrics:
  • Lines coverage reflects the amount of code that has been exercised based on the number of Java byte code instructions called by the tests.
  • Branches coverage shows the percent of exercised branches in the code, typically related to if/else and switch statements.
  • Cyclomatic complexity reflects the complexity of code by giving the number of paths needed to cover all the possible paths in a code through linear combination.

To take a trivial example, if there are no if or switch statements in the code, the cyclomatic complexity will be 1, as we only need one execution path to cover the entire code.

5. Code Coverage Score

Now that we know a bit about how JaCoCo works, let's improve our code coverage score.
In order to achieve 100% code coverage, we need to introduce tests that cover the missing parts shown in the initial report:

@Test
public void whenPalindrom_thenAccept() {
    Palindrome palindromeTester = new Palindrome();
    assertTrue(palindromeTester.isPalindrome("noon"));
}

@Test
public void whenNearPalindrom_thanReject(){
    Palindrome palindromeTester = new Palindrome();
    assertFalse(palindromeTester.isPalindrome("neon"));
}

Now we have enough tests to cover our the entire code, but to make sure of that, let's run the Maven command mvn jacoco:report to publish the coverage report:

In a real-world project, as developments go further, we need to keep track of the code coverage score.
JaCoCo offers a simple way of declaring minimum requirements that should be met, otherwise the build will fail. We can do that by adding the following check goal in our pom.xml file: 
<execution>
    <id>jacoco-check</id>
    <goals>
      <goal>check</goal>
    </goals>
    <configuration>
        <rules>
            <rule>
                <element>PACKAGE</element>
                <limits>
                    <limit>
                        <counter>LINE</counter>
                        <value>COVEREDRATIO</value>
                        <minimum>0.50</minimum>
                    </limit>
                </limits>
            </rule>
        </rules>
    </configuration>
</execution>

As we can see, we're limiting the minimum score for lines coverage to 50%.
Keep in mind though, 100% code coverage doesn't necessarily reflect effective testing, as it only reflects the amount of code exercised during tests.
⚠️ **GitHub.com Fallback** ⚠️