Tutorial 3 Static Analyzers, SonarQube and Infer - McGill-ECSE429-Winter2022/tutorials GitHub Wiki

1. For this tutorial, we will use two popular Internet of Things (IoT) projects, to run the static analysis process on them. These projects are licensed under Eclipse; however, public community may contribute to these projects, i.e. open source.

2. Go to the main page of Eclipse IoT platform (https://iot.eclipse.org) and select the "Projects" tab. You will find a list of different IoT projects, with various goals and application domains.

3. For the first part of the tutorial, we would use the project "Ditto" (https://www.eclipse.org/ditto/). go to the GitHub repository of the project (https://github.com/eclipse/ditto) and clone the project in a dedicated directory on your own machine.

4. For the second part of the tutorial, we would use the project "Leshan" (https://www.eclipse.org/leshan/). go to the GitHub repository of the project (https://github.com/eclipse/leshan) and clone the project in a dedicated directory on your own machine.

Attention Both of these projects are Java based projects, using Maven build system.

This section looks at how to use SonarQube as a stand-alone setup. Please note that you may integrate the static analysis process with automatic build tools like TravisCI and Jenkins.

1. Go to the SonarQube main page (https://www.sonarqube.org/) and click on the blue Download button.

2. Choose the community edition and click on the associated Download For Free button.

3. Extract the sonarqube-9.0.1.4.zip to the location of your choice.

4. Start the SonarQube Server

   • Assuming you are in the root folder of the sonarqube directory, redirect yourself to the "bin" directory. From there, based on the OS, you need to change your path to the OS-based specific directory including the shell script/ batch file which is used to start the SonarQube server. By running the associated script the server would boot and be accessible (few seconds may be needed!)

   • In this example, we have a OSX machine, so we use ./sonar.sh [option] as a command. With option as start, the server should boot and SonarQube would create a default web server at localhost:9000. If you type (localhost:9000) at your browser’s address bar, you will be redirected to the SonarQube server.

Attention You can also setup the SonarQube server, through setting up a running Docker daemon from a non-root user. For this matter, you need to have "Docker", "Docker-compile", "Docker-machine" and if you are using an OSX-based machine, "Virtual Box" installed on your system. Check this link out if you want to know how to setup your server using a running daemon of SonarQube! (https://docs.sonarqube.org/latest/setup/get-started-2-minutes/)

1. Log in the server The default account created for your page has "admin/admin" as its credentials.

2. Setting up the project for analysis and running the analysis

I have already setup the Ditto project in my SonarQube server account. Now, it’s your turn to do so!

• For doing so, you have to click the + button to add the project manually. When doing so, you will be prompt to choose a "Project key" and a "Display name". By default, the "Display name" would follow your input for the "Project key". However, you can change the "Display name" to be different from the project key.

• As the next step, you have to provide a "Token" for your project. This token is used for identifying your project. You can always revoke an existing token of a project and assign a new token to that project. Furthermore, different projects may use same token! By choosing a name for your token and clicking on the Generate button, a new token assigned to your project would be generated. Click on the Continue button.

• Now you need to choose the main programming language your project is written in. Ditto is based on Java. So, we would choose the Java option. Furthermore, this project is built using Maven technology. So, we would choose the associated option for it.

• By doing this, a bash/terminal command would be created. Running this command in the root directory of Ditto project, where its pom.xml file is located, would start the sonar static analysis on the project.

Attention We might need to pass additional switch values to be able to run the sonar analysis as a Maven target. For instance, if there are no binaries associated with some Java classes, we need to pass a specific switch for this situation!

• After finishing the analysis, the "Issues" tab would be activated and the analysis report would become visible.

  1. Accessing projects' analysis report(s) It is always possible to access the analysis of your different projects from the "Projects" tab. A list of all analyzed projects and their analysis history is available under this tab. If a project has been analyzed multiple times, each analysis can be accessed individually and in comparison to other analysis at different time stamps. These reports include the number of "Bugs", "Vulnerabilities", "Security Hotspots", "Technical Dept" and "Code Smells" detected for the project. Based on the number of issues in each category and their severity, SonarQube would assign a mark to the project for its general reliability, security and also maintainability. Also, these marks are available for different modules of the project. It is also possible to investigate identified issues individually. Through clicking on the "Issues" tab for your desired project, you can filter the reported issues based on their category, i.e. bug, vulnerability and security hotspot. You can also choose individual issues and inspect their information including their severity, e.g. major. minor, etc., if they are confirmed or not and also the (time) effort needed to fix the issue and submit a new patch. SonarQube would also locate the exact placement of the issue within the project, i.e. the parent module and source code lines.

  2. Changing the project’s quality profile While adding your project to the SonarQube server for analysis, through declaring the main programming language of the project, SonarQube would use a default "Quality Profile" associated with that specific programming language to evaluate the project for regular system bugs, vulnerabilities, etc. It is possible to change this quality profile for the next analysis of the project. For doing so, from the "Project’s Settings" scroll down, choose the option "Quality Profiles". Within this tab, a list of default quality profiles for evaluating the project’s source codes in each specific programming language, is visible. For our project, we would want to change the profile for Java language. I have already setup a new profile for this language, so I have the option to choose between the default (Sonar Way) or my new profile (tutorial_3). But, I would like to add a new quality profile and use that one for my next analysis.

  3. Creating a new quality profile Through the main tool bar of the SonarQube server, navigate to the tab "Quality Profiles". Under this tab all of the profiles are accessible. You can search and choose the profiles based on the specific programming language. Click on the button Create. Follow the first two prompts for assigning a name and a language to the new profile. The last property would make the new profile to inherit all of its detection rules from its "parent" profile. After creating the quality profile, it is possible to activate desired remaining rules which were not originally present within the parent profile. After activating desired additional rules, if any, now the new quality profile is ready to be set as the default profile for the next analysis of our project.

Alongside of SonarQube, Infer (FaceBook Inc.) is another popular choice for running static analysis on projects. Similar to SonarQube, this analyzer covers multiple programming languages. Furthermore, it is possible to change the setting of analysis for Infer to add or remove rules that are used as the basis of the analysis. For instance, in the default mode of the analysis, the target project would be analyzed for detecting "Resource Leak" vulnerabilities. It is possible to analyze the code for "Buffer Over Run" vulnerabilities as well (through passing -- bufferoverrun to the main command.

1. Infer Workflow

Infer workflow includes two main phases: * Infer Capture: Capturing binaries, created after compiling the project’s source codes, into the specific intermediate language used by Infer for running the analysis. * Infer Analysis: Analyzing generated intermediate files, artifacts of the previous phase dumped into a newly created directory named "infer-out", for detecting potential vulnerabilities.

1. Running static analysis using Infer

It is possible to perform each phase of the analysis, i.e. capture and analysis, individually or as a global workflow. For running the analysis as a global workflow, simply run the following command in the root directory of your project including the specific build file:

infer run — [build tech.] [<target>] [switch]

For our tutorial, we would run the analysis for a project with Maven build technology. So, we can run: infer run — mvn <target> [swtich]

More specifically: infer run — mvn clean install -D.enforcer.skip=true

Attention 1 : In my command, I am passing -D.enforcer.skip=true ! This is because the JDK version used by my Maven is not compatible with the project’s prerequisites. The correct approach would be to change the default JDK which my Maven would use for building projects. However, based on my needs and other projects at hand, I decided to skip changing the Maven’s JDK version.

Attention 2: Based on your needs and setup, also considering the status the project, you may need to pass different options to your Infer analysis. For instance, you may need to ask Maven to perform its target phase without running the tests!

After finishing the analysis, its report would be (partially) available on the terminal window. similar to SonarQube’s reports, the exact placement of the issue and its detected type would be declared. The complete report is accessible in the "report.txt" file, located within "infer-out" directory.