linters formatters cicd cycle - juancamilocc/virtual_resources GitHub Wiki

Linters and Formatters in the CI/CD cycle

In this guide, you will learn how to integrate linters and formatters into a CI/CD cycle using Jenkins. This approach enhances code quality by promoting clean and consistent code through static analysis. We will explore linters such as Ruff and ESLint, and formatters like Black and Prettier.

The general workflow is as follows:

General workflow

First, we will understand how each formatter and linter works through examples.

Formatters

Black

It is an automatic code formatter for Python that follows a "code looks the same regardless of who writes it" philosophy. Its primary goal is to format code uniformly, minimizing the time developers spend discussing code style.

You can install it as follows.

pip install git+https://github.com/psf/black

You can use it this way.

black <path/file1> <path/file2>

Here is an example.

We will have the following basic code.

def my_function(a,b):
  if(a>0):
      print("Positive number")
  else:print("Negative number")

my_function(5)

We apply the formatter.

black path/file
# reformatted test.py

# All done! ✨ 🍰 ✨
# 1 file reformatted.

And the result is the following.

def my_function(a, b):
    if a > 0:
        print("Positive number")
    else:
        print("Negative number")

my_function(5)

Prettier

is a code formatter that supports a wide range of languages and web frameworks, including JavaScript, TypeScript, HTML, CSS, Markdown, and more. It is designed to enforce consistent formatting rules across frontend and backend projects.

You can installi it, as follows.

npm install -g prettier

Here is an example.

We will have the following basic code.

import { Component } from '@angular/core';

@Component({
selector: 'app-root',
templateUrl: './app.component.html',
styleUrls: ['./app.component.css']
})
export class AppComponent {
title = 'my-angular-app'; constructor() { }
sayHello(name:string) { console.log("Hello, " + name + "!");}
}

We apply the formatter.

prettier --write path/file 
# src/app/app.component.ts 92ms

And the result is the following.

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css'],
})
export class AppComponent {
  title = 'my-angular-app';
  constructor() {}
  sayHello(name: string) {
    console.log('Hello, ' + name + '!');
  }
}

Linters

Ruff

Ruff is a robust linter that allows us to perform static analysis on Python code. It can be configured using a pyproject.toml, ruff.toml, or .ruff.toml file, or you can simply run it with the default configuration.

You can install it as follows.

pip install ruff

The following is an example of a .ruff.toml file, which should be saved in the project's root directory.

# Exclude commonly ignored directories.
exclude = [
    ".bzr",
    ".git",
    ".hg",
    ".ipynb_checkpoints",
    ".mypy_cache",
    ".ruff_cache",
    "node_modules",
    "venv",
]

# Set the line length limit according to PEP 8.
line-length = 88

[lint]
# Enable rules related to style and errors.
select = [
    "E",  # All pycodestyle errors
    "F",  # All pyflakes errors
    "W",  # Warnings
    "C901" # McCabe complexity
]

# Allow unused variables when underscore-prefixed.
dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"

[format]
# Use double quotes for strings and respect magic trailing commas.
quote-style = "double"
skip-magic-trailing-comma = false

# Respect indentation and line ending conventions.
indent-style = "space"
line-ending = "auto"

Now, let's analyze our code.

ruff check . --show-fixes --unsafe-fixes
# test.py:22:89: E501 Line too long (131 > 88)
#    |
# 21 |     def get_todos(self) -> List[Dict[str, str]]:
# 22 |         return [{"id":todo.id, "item":todo.item} for todo in self.todos]  # Error 2 & 3: Missing space after colon and before value
#    |                                                                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ E501
# 23 | 
# 24 |     def remove_todo(self, todo_id: int) -> None:
#    |

# test.py:29:17: F841 [*] Local variable `found` is assigned to but never used
#    |
# 27 |             if self.todos[i].id==todo_id:  # Error 5: Missing space around operator
# 28 |                 self.todos.pop(i)
# 29 |                 found = True
#    |                 ^^^^^ F841
# 30 |                 break  # Error 6: Should use 'break' instead of 'return' here to continue execution
#    |
#    = help: Remove assignment to unused variable `found`

# test.py:30:89: E501 Line too long (99 > 88)
#    |
# 28 |                 self.todos.pop(i)
# 29 |                 found = True
# 30 |                 break  # Error 6: Should use 'break' instead of 'return' here to continue execution
#    |                                                                                         ^^^^^^^^^^^ E501
# 31 | 
# 32 |         print("Todo not found")  # Error 7: Should use logging instead of print statement
#    |

# test.py:32:89: E501 Line too long (89 > 88)
#    |
# 30 |                 break  # Error 6: Should use 'break' instead of 'return' here to continue execution
# 31 | 
# 32 |         print("Todo not found")  # Error 7: Should use logging instead of print statement
#    |                                                                                         ^ E501
# 33 | 
# 34 |     def clear_todos(self) -> None:
#    |

# test.py:48:89: E501 Line too long (101 > 88)
#    |
# 46 |     # Inconsistent quote styles (single vs. double quotes)
# 47 |     for todo in todo_list.get_todos():
# 48 |         print(f"Todo ID: {todo['id']} - Item: {todo['item']}")  # Error 9: Use consistent quote style
#    |                                                                                         ^^^^^^^^^^^^^ E501
# 49 | 
# 50 |     todo_list.remove_todo(3)
#    |

# test.py:53:5: F841 [*] Local variable `unused_var` is assigned to but never used
#    |
# 51 |     todo_list.remove_todo(1)
# 52 | 
# 53 |     unused_var = 42  # Error 10: Unused variable
#    |     ^^^^^^^^^^ F841
# 54 |     todo_list.clear_todos()
#    |
#    = help: Remove assignment to unused variable `unused_var`

# Found 6 errors.
# [*] 2 fixable with the --fix option.

We can see that Ruff suggests fixing two errors using the fix option. To automatically fix these errors, run the following command.

ruff check . --show-fixes --unsafe-fixes --fix
.
.
.
# Fixed 3 errors:
# - test.py:
#     3 × F841 (unused-variable)

# Found 7 errors (3 fixed, 4 remaining).

Furthermore, you can save the logs to a file using the -o <filename.txt> option, as shown below.

ruff check --show-fixes --unsafe-fixes -o logs-linter.txt

This can be important for generating reports and sending them to the developers in charge.

ESLint

It is a robust static code analysis linter for JavaScript environments. Its configuration rules are included in a .eslintrc file. ESLint can be adapted for use with frameworks such as React and Angular.

Next, we will show you examples of React and Angular configurations in an .eslintrc.json file.

React

{
    "root": true, // Indicates that this is the root ESLint configuration file
    "ignorePatterns": [
      "build/**/*", // Ignore all files in the build directory
      "node_modules/**/*" // Ignore all files in the node_modules directory
    ],
    "extends": [
      "eslint:recommended", // Use the recommended ESLint rules
      "plugin:@typescript-eslint/recommended", // Use the recommended TypeScript rules
      "plugin:react/recommended", // Use the recommended rules from the React plugin
      "airbnb", // Extend Airbnb's base configuration for best practices
      "airbnb/hooks" // Extend Airbnb's configuration for React hooks
    ],
    "parser": "@typescript-eslint/parser", // Use the TypeScript parser for ESLint
    "parserOptions": {
      "ecmaFeatures": {
        "jsx": true // Enable JSX support for React
      },
      "ecmaVersion": 2024, // Specify the latest ECMAScript version
      "sourceType": "module" // Enable the use of ES module syntax (import/export)
    },
    "plugins": [
      "react", // Enable the React plugin
      "@typescript-eslint" // Enable the TypeScript plugin
    ],
    "rules": {
      "react/react-in-jsx-scope": "off", // Disable the rule that requires React to be in scope
      "import/prefer-default-export": "off", // Allow named exports without a default export
      "@typescript-eslint/explicit-module-boundary-types": "off" // Do not require explicit return types on functions and class methods
    },
    "settings": {
      "react": {
        "version": "detect" // Automatically detect the React version from package.json
      }
    }
}

Now, to facilitate its execution, we will add some lines to the package.json file, as follows.

"scripts": {
    .
    .
    .
    "eject": "react-scripts eject",   
    "lint": "eslint \"src/**/*.{ts,tsx}\"",
    "lint:fix": "eslint \"src/**/*.{ts,tsx}\" --fix"
  },

We must install the following dependencies.

npm install --save-dev eslint
npm install --save-dev eslint-plugin-import eslint-plugin-jsx-a11y eslint-plugin-react @typescript-eslint/eslint-plugin @typescript-eslint/parser 
install-peerdeps --dev eslint-config-airbnb --legacy-peer-deps

Run linter, as follows.

npm run lint
# > [email protected] lint
# > eslint "src/**/*.{ts,tsx}"


# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/App.test.tsx
#   3:17  error  Unable to resolve path to module './App'        import/no-unresolved
#   3:17  error  Missing file extension for "./App"              import/extensions
#   5:1   error  'test' is not defined                           no-undef
#   6:10  error  JSX not allowed in files with extension '.tsx'  react/jsx-filename-extension
#   8:3   error  'expect' is not defined                         no-undef

# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/App.tsx
#   3:28  error  Unable to resolve path to module './ErrorComponent'                                                    import/no-unresolved
#   3:28  error  Missing file extension for "./ErrorComponent"                                                          import/extensions
#   5:13  error  Function component is not a function declaration                                                       react/function-component-definition
#   5:19  error  Unexpected block statement surrounding arrow body; move the returned value immediately after the `=>`  arrow-body-style
#   7:5   error  JSX not allowed in files with extension '.tsx'                                                         react/jsx-filename-extension

# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/ErrorComponent.tsx
#    4:24  error    Function component is not a function declaration  react/function-component-definition
#    5:19  error    Strings must use singlequote                      quotes
#    9:5   warning  Unexpected console statement                      no-console
#    9:17  error    Unexpected string concatenation                   prefer-template
#    9:17  error    Strings must use singlequote                      quotes
#   15:5   error    JSX not allowed in files with extension '.tsx'    react/jsx-filename-extension

# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/index.tsx
#    4:17  error  Unable to resolve path to module './App'              import/no-unresolved
#    4:17  error  Missing file extension for "./App"                    import/extensions
#    5:29  error  Unable to resolve path to module './reportWebVitals'  import/no-unresolved
#    5:29  error  Missing file extension for "./reportWebVitals"        import/extensions
#    8:3   error  'document' is not defined                             no-undef
#    8:38  error  'HTMLElement' is not defined                          no-undef
#    8:49  error  Missing trailing comma                                comma-dangle
#   11:3   error  JSX not allowed in files with extension '.tsx'        react/jsx-filename-extension
#   13:22  error  Missing trailing comma                                comma-dangle

# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/reportWebVitals.ts
#   5:32  error  Expected a line break after this opening brace   object-curly-newline
#   5:74  error  Expected a line break before this closing brace  object-curly-newline

# ✖ 27 problems (26 errors, 1 warning)
#   10 errors and 0 warnings potentially fixable with the `--fix` option.

In the previous result, the linter shows us that it can auto-fix some error, if you want to use the auto-fix do this.

mpm run lint:fix
.
.
.
# > [email protected] lint:fix
# > eslint "src/**/*.{ts,tsx}" --fix


# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/App.test.tsx
#   3:17  error  Unable to resolve path to module './App'        import/no-unresolved
#   3:17  error  Missing file extension for "./App"              import/extensions
#   5:1   error  'test' is not defined                           no-undef
#   6:10  error  JSX not allowed in files with extension '.tsx'  react/jsx-filename-extension
#   8:3   error  'expect' is not defined                         no-undef

# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/App.tsx
#   3:28  error  Unable to resolve path to module './ErrorComponent'  import/no-unresolved
#   3:28  error  Missing file extension for "./ErrorComponent"        import/extensions
#   7:5   error  JSX not allowed in files with extension '.tsx'       react/jsx-filename-extension

# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/ErrorComponent.tsx
#    9:5  warning  Unexpected console statement                    no-console
#   15:5  error    JSX not allowed in files with extension '.tsx'  react/jsx-filename-extension

# /home/juan-camilo/Camilo/linters-guide/testejs/react/myreactapp/src/index.tsx
#    4:17  error  Unable to resolve path to module './App'              import/no-unresolved
#    4:17  error  Missing file extension for "./App"                    import/extensions
#    5:29  error  Unable to resolve path to module './reportWebVitals'  import/no-unresolved
#    5:29  error  Missing file extension for "./reportWebVitals"        import/extensions
#    8:3   error  'document' is not defined                             no-undef
#    8:38  error  'HTMLElement' is not defined                          no-undef
#   11:3   error  JSX not allowed in files with extension '.tsx'        react/jsx-filename-extension

# ✖ 17 problems (16 errors, 1 warning)

Other errors must be fixed manually.

To select specific files and redirect the output to a file, you must run linter as follows.

# Run for a specific file
npm run lint path/file path/file2 path/file3 ...

# Redirect output to file
npm run lint path/file > linter-logs.txt

Angular

{
  "root": true,  // Indicates that this is the root ESLint configuration file
  "ignorePatterns": [
    "projects/**/*"  // Ignore all files in the projects directory
  ],
  "overrides": [  // Overrides for specific file types
    {
      "files": [
        "*.ts"  // Apply these rules to TypeScript files
      ],
      "extends": [
        "eslint:recommended",  // Use the recommended ESLint rules
        "plugin:@typescript-eslint/recommended",  // Use the recommended TypeScript rules
        "plugin:@angular-eslint/recommended",  // Use the recommended Angular ESLint rules
        "plugin:@angular-eslint/template/process-inline-templates"  // Process inline templates in Angular
      ],
      "rules": {
        "@angular-eslint/directive-selector": [
          "error",  // Error level for directive selector rule
          {
            "type": "attribute",  // Selector type for directives
            "prefix": "app",  // Prefix for directive selectors
            "style": "camelCase"  // Style for directive selectors
          }
        ],
        "@angular-eslint/component-selector": [
          "error",  // Error level for component selector rule
          {
            "type": "element",  // Selector type for components
            "prefix": "app",  // Prefix for component selectors
            "style": "kebab-case"  // Style for component selectors
          }
        ]
      }
    },
    {
      "files": [
        "*.html"  // Apply these rules to HTML files
      ],
      "extends": [
        "plugin:@angular-eslint/template/recommended",  // Use recommended rules for Angular templates
        "plugin:@angular-eslint/template/accessibility"  // Use accessibility rules for Angular templates
      ],
      "rules": {}  // No specific rules for HTML files
    }
  ]
}

We must install the following dependencies.

npm install --save-dev eslint
ng add @angular-eslint/schematics
npm install eslint-config-airbnb-base eslint-config-airbnb-typescript eslint-plugin-simple-import-sort eslint-plugin-unused-imports --save-dev

Run the linter, you should get something similar like this.

ng lint
# Linting "my-angular-app"...

# /home/juan-camilo/Camilo/linters-guide/testejs/angular/my-angular-app/src/app/app.component.ts
#   15:17  error  Unexpected empty method 'emptyMethod'   @typescript-eslint/no-empty-function
#   18:18  error  Unexpected empty method 'Invalid_Name'  @typescript-eslint/no-empty-function

# ✖ 2 problems (2 errors, 0 warnings)

# Lint errors found in the listed files.

If you want to save the linter results to a file, you can do the following.

# Linter result of a specific file and save them in a file 
ng lint --lint-file-patterns="src/app/app.component.ts" &> linter-logs.txt

Integration with the CI/CD Cycle Using Jenkins

We can integrate these static code analysis tools into our pipelines.

Next, we will integrate all the linters and formatters mentioned in the previous steps. The following pipeline encompasses a monorepository; for that reason, the frontend and backend are in the same place.

The functionality of the pipeline is represented in the following stage.

pipeline {
    agent {
        kubernetes {
            cloud 'kubernetes-v3'
            defaultContainer 'jnlp'
            yaml """
            <Definition pod in yaml format>
            """
        }
    }
    environment {
        FRONT_PROJECT = "front-project/"
        BACK_PROJECT = "back-project/"
    }
    stages {
        stage("Linter and Formatter") {
            steps {
                container('container') {
                    script { 

                        sh "git config --global --add safe.directory ${WORKSPACE}"

                        def check_back = sh(script: "git diff --name-only HEAD~1 ${BACK_PROJECT}", returnStdout: true).trim()
                        def check_front = sh(script: "git diff --name-only HEAD~1 ${FRONT_PROJECT}", returnStdout: true).trim()

                        if (check_back) {
                            def modifiedFiles = sh(script: "git diff --name-only HEAD^ HEAD | grep -E '\\.py\$'", returnStdout: true).trim().split('\n')
                            sh """
                                pip install ruff
                                pip install git+https://github.com/psf/black
                                black ${modifiedFiles.join(' ')}
                                ruff check --show-fixes --unsafe-fixes -o linter-logs.txt ${modifiedFiles.join(' ')}
                            """

                            def lintBack = readFile('linter-logs.txt').trim()

                            if (!lintBack.contains("All checks passed!")) {
                                echo "There was an error, check code"
                                error("Backend linting failed") 
                            }

                        } else if (check_front) {
                            def modifiedFiles = sh(script: "git diff --name-only HEAD^ HEAD | grep -E '\\.ts\$'", returnStdout: true).trim().split('\n')
                            
                            // Example for Angular project
                            sh """
                                npm install --save-dev eslint
                                ng add @angular-eslint/schematics
                                npm install eslint-config-airbnb-base eslint-config-airbnb-typescript eslint-plugin-simple-import-sort eslint-plugin-unused-imports --save-dev
                                npm install -g prettier
                                prettier ${modifiedFiles.join(' ')}
                                ng lint --lint-file-patterns="${modifiedFiles.join(' ')}" &> linter-logs.txt
                            """

                            // Example for React project
                            sh """
                                npm install --save-dev eslint
                                npm install --save-dev eslint-plugin-import eslint-plugin-jsx-a11y eslint-plugin-react @typescript-eslint/eslint-plugin @typescript-eslint/parser
                                install-peerdeps --dev eslint-config-airbnb --legacy-peer-deps
                                npm install -g prettier
                                prettier ${modifiedFiles.join(' ')}
                                npm run lint ${modifiedFiles.join(' ')} > linter-logs.txt
                            """

                            def lintFront = readFile('linter-logs.txt').trim()

                            if (!lintFront.contains("All files pass linting.")) {
                                echo "There was an error, check code"
                                error("Frontend linting failed")
                            }

                        } else {
                            echo "No changes in Backend or Frontend"
                        }
                    }
                }
            }
        }
        stage ('Another stage') {
            steps {
                // Here you can include more stages
            }
        }
    }
    post {
        success {
            echo "SUCCESS"
        }
        failure {
            echo "FAILURE"
        }
    }
}

Conclusion

The integration of linters and formatters in software development processes is crucial for maintaining high code quality and consistency. Linters analyze code for potential errors, enforce coding standards, and ensure best practices are followed, while formatters automatically adjust code style according to defined conventions. Together, they help reduce bugs, enhance readability, and streamline collaboration among team members.

Incorporating linters and formatters into Continuous Integration and Continuous Deployment (CI/CD) pipelines ensures that code is consistently checked and formatted before it is merged into the main codebase. This practice minimizes technical debt and fosters a culture of quality, as developers receive immediate feedback on their code. By automating these processes, teams can focus on writing functionality while maintaining clean and maintainable codebases, ultimately leading to faster development cycles and improved project outcomes. In the next guide Kaniko Builder, an alternative to Docker, we will discuss how to incorporate the Kaniko Builder using caching to improve the performance of our images.

⚠️ **GitHub.com Fallback** ⚠️