[[TOC]]

Karate is a powerful tool that can be used to test the APIs we develop here at Nextens. Due to it's flexibility there are several ways that APIs can be tested. Despite this, team test automation have come together to determine the best way to test APIs. The intention for this wiki page is to keep growing and expanding with time.

As of this moment the best practices mentioned below are what we consider the best practices when it comes to API testing. If you have any suggestions, feel free to reach out to team test automation on our teams channel.

For good examples and test cases to use when testing APIs check the following wiki page:

Karate API Testcases and examples

The best practices consists on tips for structuring karate tests, test cases and tips and tricks. Before starting, we recommend to always use Karate NPM with the help of Jbang when working with karate. As karate.jar can introduce security issues we have chosen to work with Karate NPM.

In order to learn more about karate JBang please follow the following wiki links:

Jbang Wiki page

• Set karate JBang to use the latest version of karate under test.js (As of writing this 1.4.1).

• If you want to use the newest version without needing to update it every time you can do the following in the "package.json" file:

  

It's important to add comments to your tests. Describe your actions, what you did, how you did it etc. Not only does it help you remember what you've done but it also makes it easy for others to follow your tests and reuse your tests or update them for maintenance purposes.

When using karate use # to add comments for that line. If using JS (JavaScript) then use //

#This is a karate comment
//This is a JS comment

Follow the following link for a guide on best practices for writing comments by stackoverflow:

Stack overflow Guide

It's important to follow a consistent format when writing tests. This helps to keep things clean and legible. Sometimes feature files can get long and complex. That's why it's important to keep things clean and clutter free. See the following code block below for an example of clean formatting

Feature: Features are always at the start of the line - Always leave a free line underneath every Feature, Scenario and background

Background: Background is always at the start of the line too 

    # Anything underneath a background should be spaced with a tab (4 spaces)

Scenario: Scenarios are always at the start of a feature

    # Anything underneath a tab should be spaced with a tab (4 spaces)
    # Any objects in a scenario or background should always have another tab space see below
    {
        #I am part of an array or JSON object so I am also indented
        {
             #The same goes for every other object inside an object
        }
    }

Scenario: A new Scenario so back to the start of the file

    # An empty line is added underneath the scenario and intended again with a tab

DRY is a common programming best practice but it holds up for when working with Karate too. Essentially it is what it says, Don't Repeat Yourself. If you have a feature, scenario or even an script you need to use across multiple tests instead of reusing it make it a shared file. When we talk about a shared file, we mean having a feature or script that can be called from any test.

There are two ways to do this: shared feature or script within a shared folder or NPM packages.

1. In the root folder of your karate tests, create a new folder called SharedFeatures. Place any shared features in this folder for example a feature that generates BSN. See the image below:

In the shared file, make sure to have a variable that you want to be extracted saved at the bottom of the file. For example see the generateBSN.feature below:

In order to call a shared file, use the following code block as an example:

* def BSN = karate.call("file:path to shared file")
* def bsn = BSN.BSN

The way the code above works is by setting BSN as an variable that calls the shared file. Then on the second line we create a new variable then make it equal BSN (the variable we called) and select BSN from the variable listed in the image above. This allows us to call this feature from any test and create a BSN from one location instead of repeating the same code.

2. NPM packages: It's always good practice to check the existing karate NPM packages to find if there may be a package out there that already does exactly what you're looking for. If an NPM package doesn't exist, and you think it could be useful for your colleagues to use think about creating a new NPM package.

More information on NPM packages can be found in the links below:

NPM package guide

Scenario outlines are a very useful tool, one that can support the DRY principle. Scenario outlines allow you to define a series of steps within a scenario that can be run multiple times with different data. This allows for a more dynamic scenario and can be used when the same steps must be run with different data. For example: you have an endpoint that can get aangiftes for a certain year. The endpoint expects a parameter for year, and you want to check the years 2019-2023.

Usually, you would create a scenario for each year and then have it check the year. However, with a scenario outline you would just create a scenario that calls the API, runs the desired assertions/tests and then using scenario outline you could run the scenario several times with different year data. See below:

# <year> is the way to determine the data is dynamic

Scenario Outline: Check the aangiftes created for the year of <year>

Given url https://www.exampleAPI.com/aangifte/<year>
When method GET
Then status 200

# Examples is the way to pass the chosen data for the outline, year is the name of the matching <year> and then the years are the given value.

Examples:
|year |
|2019 |
|2020 |
|2021 |
|2022 |
|2023 |

# Another way to do this is by passing a JSON file then the JSON data will be used as the outline value. See below:
# | read('../callarray/kittens.json') |

There are several ways to setup a karate-config.js file, many of them are correct. Your way of setting up karate-config.js is more of a personal style. As long as your system works for both local testing and via a pipeline then it's sufficient. However, the preferred way is to have one karate-config.js with separate config files for each environment.

In the root folder of the karate tests, place the karate-config.js file. The recommended way is to set it up as below:

function fn() {
    var env = karate.env || 'dev' // Set env to karate.env if it is set, otherwise default to 'dev'

    if (env.contains("__")) {
  
       // Below we tell the config to take the configuration file set to the chosen environment
       var definitions = karate.read("file:Configuration/" + env.toLowerCase() + ".json");  
      
       // Set up all required variables to run in a pipeline below. They should start and end with __ as that how the variables will be named in the pipeline. See below for an example:
        var yourVariable =  "__variable__"
        var userSecrets = "__userSecrets__"
        
        var config = {
          //Fill in all pipeline variables here
          definitions: definitions,
          yourVariable:  yourVariable,
          userSecrets: userSecrets
        };

     }  else {
      
       // Below we tell the config to take the configuration file set to the chosen environment
       var definitions = karate.read("file:Configuration/" + env.toLowerCase() + ".json");
      
       // Below we tell the config to take the secrets file set to the chosen environment
       var secrets = karate.read("file:Secrets/" + env.toLowerCase() + ".json");
       
       var config = {
        // These two variables are setup to have all variables accessible
        definitions: definitions,
        secrets: secrets
      };
  }
  return config;
}
  
  

In the root folder of the karate tests, there should be a folder called "Configuration", in that folder you have files set up for each environment. This is to make sure each environment will have the data specific to it's own environment. For example you would have "dev.json" and then in that file you would have the data setup in a json format. This would be the same for each environment used like "acc.json", "delivery.json" etc. The same should be done for secrets, except this time the folder should be called "Secrets".

Git Ignore

In the git ignore file (.gitignore) you can set up specific folders we don't want commited. Either because they're not needed or they contain sensitive data. See the example below for the recommended values to be in there:

# C#
bin/
obj/

# Karate
Secrets/
target/
node_modules/

test.js

Last but not least, we want to have the environments set up in the test.js. Generally, we want it to be generic so that the value can be passed by the pipeline. However, when testing locally we may want to test different environments. See below on how to set it up (Code below is written in JS):

#! /usr/bin/env node
const karate = require('@karatelabs/karate');

// Get the environment from command line arguments or default it to 'dev'
var argv = require('minimist')(process.argv.slice(2));
// Below the environment will be set to dev by default. 
// However when testing locally and you want to test another environment. Change dev to the environment you want to test for example acc.
const env = argv['env'] ? "" : "--env=dev";
const tests = argv._[0] ? "" : "tests/**"

karate.exec(`-f cucumber:json,junit:xml ${tests} ${env}`);

Sometimes we want to verify that a test fails on a specific condition. For example, we expect one of the tests to fail. If we want to be clear what went wrong and where we can use karate.fail. See below for an example:

# The failed message can be set to whatever information will be useful to display
* if (condition) karate.fail('Test failed because the condition did not match our expectations')

Under this heading, you can find a list of general tips and tricks that could prove useful when creating API tests.

• Instead of creating all test cases under one scenario, create a new scenario for each test case. However, if you can make multiple test cases out of one endpoint call, for example more than one assertion on the response then DO use one scenario to avoid repetition.

• Define your URL in the background, then use path to set the specific path of the URL. See below for an example:

Feature: Example feature

Background:

* url exampleUrl

Scenario:
   
    * path "/test/api"
    When method GET
    Then status 200

• Access any secrets stored in the secrets folder in any feature. If using the recommended karate-config.js then the secrets file that is used is dependent on the environment being run. In other words, if using dev as an environment it will use the dev secrets file. Thanks to this we can call the secrets in the background of every feature. See an example below:

Feature: Showing how to access secrets

Background:

   # Lets pretend that in your secret files the following variables exist: userSecret, userName, userPassword
   * def userSecret = secrets.userSecret
   * def userName = secrets.userName
   * def userPassword = secrets.userPassword

• Use Background to define anything you can use across multiple scenarios. For example shared URLs, shared variables etc. As seen in the examples above.

• Determine which version of karate you wish to use. At the moment Karate will get the latest version as supported by the * under karate version in packages.json. However, if you wish to use a specific version of Karate that is also possible. In the test.js file you can set the karate version. (Code below is in JS) See below:

#! /usr/bin/env node
const karate = require('@karatelabs/karate');

// Get the environment from command line arguments or default it to 'dev'
var argv = require('minimist')(process.argv.slice(2));
// Below the environment will be set to dev by default. 
// However when testing locally and you want to test another environment. Change dev to the environment you want to test for example acc.
const env = argv['env'] ? "" : "--env=dev";
const tests = argv._[0] ? "" : "tests/**"
// The following line determines that Karate will use version 1.4.1. Any available version is possible. 
// However if you just want the newest version set the version in packages.json to "*"
karate.version = '1.4.1';

karate.exec(`-f cucumber:json,junit:xml ${tests} ${env}`);

Ben explained about the structure of the API's and how they interact. This can help us finding out where to put our test effords. The diagram about the fiscal API landscape can be found here.

• On Percipio you can find the basics of API testing.
• This diagram shows the 9 different types of API testing.