Coverage - overturetool/vdm-vscode GitHub Wiki

Generation

When a VDM model is being interpreted, the server will automatically collect execution coverage information. Test coverage measurements help you to see how well a given test suite exercises your VDM model.

To save coverage information, right-click in the editor and select VDM > Generate Coverage. This generates new files called <filename>.covtbl in the project subfolder named .generated/coverage/coverage <date>,<time>.

Some care is required to make sure you collect and display coverage for the evaluations you require:

  • Coverage information is reset whenever the Language Server initializes the VDM specification (i.e. the start of a new execution session, the start of a one-shot execution, or when you run the init command in a session).
  • Coverage information is recorded cumulatively whenever expressions are evaluate with the print command.
  • Coverage is also recorded cumulatively as combinatorial tests are executed.
  • Coverage .covtbl "snapshots" are written whenever you select the VDM > Generate Coverage menu item, and they contain the coverage recorded so far.
  • The visualization of coverage uses the snapshot .covtbl files.
  • You can create covtbl files for individual VDM source files, individual folders, or the whole project.

Visualisation

Coverage information can be visualised as an overlay on VDM project files. The overlay can be enabled and disabled by clicking the Coverage Overlay button available at the top of the screen:

Coverage Button

The overlay is enabled and disabled across all open editors. If there is no coverage available for a given project with a VDM file visible in an editor when enabling the overlay a prompt to generate coverage will appear. Alternatively, new coverage information can be generated as described above.

Coverage Overlay

By default, the overlay uses the latest coverage information. However, by changing the relevant setting (detailed below) the user can instead be prompted to choose which coverage information to use.

There are some options available for the coverage display, that can be found at Settings > Extensions > VDM VSCode > Other, they are:

Setting Description Default Value
Overlay Latest Coverage Enabled: The latest generated coverage information is utilised for overlaying. Disabled: Choose which coverage information should be used for overlaying Enabled (true)
Use Heatmap Colouring Enabled: The number of hits (larger than zero) for a section corresponds to a brighter green. Disabled: All sections with any number of hits (larger than zero) are coloured the same green Enabled (true)
⚠️ **GitHub.com Fallback** ⚠️