Testing Cucumber - AutolabJS/autolabcli GitHub Wiki

Cucumber is a testing framework for behavior driven development. It works by allowing you to define your tests in Gherkin form, and makes these gherkins executable by tying them to code.

Gherkin is the Domain Specific Language (DSL) that is used for writing Cucumber tests. It allows for test scripts to be written in a human readable format, which can then be shared between all of the stakeholders in the product development.

Cucumber.js is available as an npm module.

$ npm install cucumber --save-dev

In a Gherkin defined test, you have the concept of features and scenarios. These are analogous to test suites and test cases in other testing frameworks, allowing for a clean way to structure your tests.

A scenario is literally just a single test. It should test exactly one thing in your application. A feature is a group of related scenarios. As such, it will test many related things in your application. Ideally, the features in the Gherkin files will closely map on to the Features in the application — hence the name.

Every Gherkin file contains exactly one feature, and every feature contains one or more scenarios. Scenarios are then comprised of steps, which are ordered in a specific manner:

• Given – These steps are used to set up the initial state before you do your test
• When – These steps are the actual test that is to be executed
• Then – These steps are used to assert the outcome of the test

Gherkin also gives more keywords like And, But which are used when we several Given's, When's or Then's. These should be used wherever possible to increase the readability of the scenarios, making sure each step does exactly one thing.

It also provides Background steps. These are useful when we find ourselves using the same given steps over all the scenarios of the feature. A Background is run before each scenario, but after any Before hooks. They provide a context to each Scenario. The key idea is, you can only have one set of Background steps per feature. If you need different Background steps for different scenarios, you’ll need to split them into different feature files.

Gherkin files are designed to be human readable and to give benefit to anyone involved in the product development. This includes non-technical people, so the Gherkin files should always be written in business language and not technical language.

The following is an example Gherkin for searching Google for Cucumber.js:

Feature: Google search
  Backround:
    Given I am connected to the internet

  Senario: I search for cucmber.js
    Given I have loaded Google
    When I search for "cucumber.js"
    Then the first result is "GitHub - cucumber/cucumber-js: Cucumber for JavaScript"

Once you have written your test cases in Gherkin form, you need some way to execute them. In the JavaScript world, there is a module called Cucumber.js that allows you to do this. It works by allowing you to define JavaScript code that it can connect to the various steps defined inside of your Gherkin files. It then runs the tests by loading the Gherkin files and executing the JavaScript code associated with each step of a given senario in the correct order. ( i.e Before Hooks --> Background Steps --> Given --> When --> Then )

For example, in the above example you would have the following steps:

Given('I am connected to the internet', function() {});
Given('I have loaded Google', function() {});
When('I search for {stringInDoubleQuotes}', function() {});
Then('the first result is {stringInDoubleQuotes}', function() {});

Here is a simple example to test addition in math using cucumber.js.

Set up the project -

$ npm init
$ npm install --save-dev cucumber
$ mkdir features steps

The Gherkins code goes into the features directory and the corresponding JavaScript code goes into step directory.

Create a new file in features directory called addition.feature and add this -

Feature: Addition
  Scenario: 1 + 0
    Given I start with 1
    When I add 0
    Then I end up with 1

  Scenario: 1 + 1
    Given I start with 1
    When I add 1
    Then I end up with 2

Create a new file in steps directory called addition.js and add this -

const { Given, Then, When } = require('cucumber');
const assert = require('assert');

Given('I start with {int}', function (input) {
  this.answer = input;
});
When('I add {int}', function (input) {
  this.answer += input;
});
Then('I end up with {int}', function (input) {
  assert.equal(this.answer, input);
});

Each of the scenarios are executed in an isolated context called World. The this passed to each of the steps in the scenario helps to maintain the state. Further, the scope of the this is that of the Scenario itself.

Placeholders are placed into the step string, and these placeholders will be extracted out from the Gherkins files and made available as parameters to your function.

Execute the test like this -

$ ./node_modules/.bin/cucumber.js features/ -r steps/

The -r flag is a directory containing JavaScript files to automatically require for the tests.

And the output will be as follows, an indication that all the cases have passed.

Feature: Addition

  Scenario: 1 + 0
   Given I start with 1
   When I add 0
   Then I end up with 1

  Scenario: 1 + 1
   Given I start with 1
   When I add 1
   Then I end up with 2

2 scenarios (2 passed)
6 steps (6 passed)
0m00.001s

Cucumber-js also provides various hooks. They are typically used for setup and teardown of the environment before and after each scenario. The main ones under consideration in this docs is the Before and After hook.

• Before - This hook runs before each of scenario, of all the features in the suite. (i.e. it is independent of the Feature being tested)
• After - Similarly, this runs after each scenario, of all the features in the suite.

Also, the hooks receive the this context of the scenario under execution. In this project, we use the Before and After hooks to set up stubs/mocks needed by the scenarios to assert on.

The following example explains it:

// hooks.js

// setup
Before(function () {
  this.logStub = sinon.stub(console, 'log');
});
// teardown
After(function () {
  this.logStub.restore();
});

All the hooks are to be written in a different file, steps/hooks.js in this project. These hooks are automatically used when we run the test suite using cucumber.js features/ -r steps/.

We use the Before and the After hooks to mainly setup the test doubles, as we do in any other testing framework. These setups and teardowns are not required to be visible to a person reading/setting up the Feature tests(Ghekrin docs), rather are necessary for the testing and programming purposes.

The order of execution is: {Before hook, Background steps, Scenario steps, After hook} for each of the scenarios. The background steps and scenario steps are written in the steps file of a feature. The Before and After hooks are in steps/hooks.js.

• Step definitions aren’t linked to a particular feature file or scenario. The file, class or package name of a step definition does not affect what Gherkin steps it will match. The only thing that matters is the step definition’s expression.
• Whatever happens in a Before hook is invisible to people who only read the features. Prefer Background steps wherever such visibility is necessary.
• Please note that if you use arrow functions, you won’t be able to share state between steps!

  Given('I have {int} user in my chat', users => {
    // Don't do this. The value of "this" is the "global" object
    this.users = users;
  })

  Hence, we use the good old JavaScript functions throughout the Feature tests.

Similar to Mocha, we can pass a callback function to the test case to handle asynchronous code. For example -

When('I make an API call using callbacks', function(done) {
  request('http://localhost:3000/api/endpoint', (err, response, body) => {
    if (err) {
      done(err);
    } else {
      doSomethingWithResponse(body);
      done();
    }
  });
});

Another way is to return a Promise from your step then the step will only be considered to have finished when the Promise is settled. If the Promise is rejected then the step will have failed, and if the Promise is fulfilled then the step will have succeeded.

But a better way is to use async/await for better readability.

The previous example can be rewritten as -

When('I make an API call using promises', async () => {
   const res = await fetch('http://localhost:3000/api/endpoint');
   const body = await res.json();
   doSomethingWithResponse(body);
});

Scenario outlines are a way of generating scenarios from a table of test data. This allows for parameterized testing in an even more efficient way than before, since we can have the exact same test script repeated many times with different values inserted.

Here is an example -

Feature: Addition

  Scenario Outline: <a> + <b>
    Given I start with <a>
    When I add by <b>
    Then I end up with <answer>

  Examples:
    | a | b | answer |
    | 1 | 0 | 1      |
    | 1 | 1 | 2      |
    | 2 | 2 | 4      |

When we run it with the existing steps file, the result is exactly the same as before but it has significantly less repetition. You’ll actually see if you run this that it generates the exact same Scenarios as before in the output:

$ ./node_modules/.bin/cucumber.js features/ -r steps/
Feature: Addition

  Scenario: 1 + 0
   Given I start with 1
   When I add 0
   Then I end up with 1

  Scenario: 1 + 1
   Given I start with 1
   When I add 1
   Then I end up with 2

  Scenario: 2 + 2
   Given I start with 2
   When I add 2
   Then I end up with 4

3 scenarios (3 passed)
9 steps (9 passed)
0m00.001s

World is an isolated context for each scenario, exposed to the hooks and steps as this.

One can set their own World constructor, for example (consider a web page testing using seleinum) :

// world.js
var { setWorldConstructor } = require('cucumber');

function CustomWorld() {
  this.driver = new seleniumWebdriver.Builder()
    .forBrowser('firefox')
    .build();

  // Returns a promise that resolves to the element
  this.waitForElement = function(locator) {
    var condition = seleniumWebdriver.until.elementLocated(locator);
    return this.driver.wait(condition)
  }
}

setWorldConstructor(CustomWorld)

• Cucumber.js
• Getting Started with Cucumber
• Cucumber Docs
• Todd Anderson's blog on World and Background
• cucumber java tutorial - explains background and hooks