Test plan - add-ons/plugin.video.vrt.nu GitHub Wiki

Before release we want to be sure everything is working as expected.

Code testing

Test matrix

  • Kodi versions
    • Kodi 17
    • Kodi 18
  • Python versions
    • Python 2.7
    • Python 3.7
  • Languages
    • Dutch
    • English
  • Platforms
    • Android
    • LibreELEC
    • Windows
  • Remotes
    • Kore
    • Yatse
  • Playback
    • Kodi player
    • inputstream.adaptive
    • inputstream.adaptive + Widevine (DRM)

Manual testing

Playback

  • Test A-Z listing and playback
  • Test Categories and playback
  • Test Live streams

Credentials

  • Use add-on with and without credentials
  • Use add-on after removing all tokens

My programs

  • Follow and unfollow a program
  • Mark and unmark an episode to watch later
  • Seeking to the start and stopping should remove "ongoing" overlay and reset resumetime
  • Seeking to the end should automatically stop the player (and mark as watched)
  • Seeking and stopping, should have "ongoing" overlay, propose start from (correct) resumetime
  • Refreshing any listing after playing should not show any differences

Up Next

  • Playing the next episode (without action) should advance without delay
  • Playing the next episode (by clicking) should advance without delay

Unit tests

Almost everything can be tested automatically. And Travis CI currently already tests a lot.

With credentials

Some tests requires credentials. Two solutions:

  • Copy over the add-on userdata directory content to test/userdata/
  • Add a file test/userdata/credentials.json with your username and password

Example layout credentials.json

{
    "username": "your-user-name",
    "password": "your-password"
}