Testing for Quality - Telecominfraproject/oopt-gnpy GitHub Wiki

gnpy is a work product of the PSE-Team in TIP. The PSE team strives to meet the high quality standard of industrial code and the group is encouraged to perform quality checks at any time. As the gnpy master branch represents the gnpy status made available to an external audience, quality rules are set out:

  1. The gnpy master branch shall always provide a working code and associated examples to let non-expert users try out gnpy without going into code details.
  2. The information contained shall be correct and examples are easily accessible.
  3. It shall be avoided to expose code, examples or procedures that fail to start, produce incorrect results or run into error conditions.
  4. Code compilation and executing examples should not raise warnings

Note: The develop branch of gnpy is under development and it may temporarily contain code and documentation that does not match the standards for the master branch outlined above.

Before committing the development branch into master, it's content shall be tested by code contributors and potential users. The target is to ensure that the information provided in the documentation together with the master branch is accurate and operational, meeting the quality criteria expected by the PSE-Team. A non expert user shall be able to follow the documentation step by step and execute examples and tests without difficulty. To keep the description simple, we assume a UNIX operation System and bash as a shell. We also assume that python is installed on the PC.

The following tests need to be performed before committing into Master:

  1. create an empty folder on your PC and follow the installation information provided.
  2. follow the chain of commands described in the documentation for every example provided
  3. check if there are any errors, warnings or failure to start/stop the examples
  4. check if the results match the explanation provided in the documentation
  5. the results produced by gnpy shall be extensively tested and documented.
  6. Results are considered "correct" when they match the description provided in the documentation.

During the installation and testing of the master branch code, issues may be detected. In this case:

  1. in case of failures: immediately file an issue on the repository issue list with a copy of the cli
  2. try to find the root cause or hints to help the team to address it in time
  3. in case of improvements or clarification required: file an issue with the problem and a suggestion (if possible)

The maintainer of the gnpy code will not merge any code from the develop branch, unless above conditions are met. The decision to merge into master will be taken by the PSE-Coders team after reviewing the test results. In case issues are detected in the content of the master branch, a fix shall be made available with high priority to ensure the master branch maintains the Quality standards of the PSE team. The target is to address the issue in less than 3 working days after root cause analysis by the PSE coders.

In an effort to maintain high quality standards, PSE-Team chairs are monitoring the compliance to the rules and report the status to the PSE-team every 3 month.