Running the Tests - Azure/azure-webjobs-sdk-dashboard-tests GitHub Wiki

These tests use the TestEasy UI framework for doing automated UI testing on the WebJobs SDK Dashboard. The tests are designed to run a test app using the last released WebJobs SDK to generate output, and verify that the Dashboard produces the expected output. The tests will open the browser and click around in the Dashboard UI and perform verifications. The goal of the tests is to verify that the Dashboard package under test doesn't have any regressions.

Setup

To run the tests locally, you must perform the following setup:

  • Set the following app settings or environment variables
    • DashboardSiteExtensionLocation - point this to to the full file path to the unzipped Dashboard site extension. You can get this from the WebJobs SDK build.
    • StorageAccount - set this to the Azure Storage account you want to use for the tests
    • ServiceBusAccount - set this to the ServiceBus account you want to use for the tests
    • TestBrowser - set this to IE. It is recommended that you use IE for the tests. Also, make sure the zoom level on IE is set to the default 100%, otherwise the tests will fail.
    • TestEasySupportPath - set this to the full path to the TestEasy support files. Details on how to get these support files can be found in the TestEasy documentation. At a minimum, you'll need the default.config and `IEDriverServer.exe' files.
  • Make sure your AzureWebJobsStorage and AzureWebJobsDashboard settings are empty

Once the above is set up, you can run the tests in VS using xUnit. You must be running VS in Administrator mode - otherwise TestEasy won't be able to create test server instances in IIS Express. Being UI tests, they can sometimes be flaky. From time to time you'll get a few failures, that will succeed on rerun.

Permutations

The default mode of the tests run the latest Dashboard against the last released SDK (the latest version on Nuget). This setup allows you to verify that any Dashboard changes you've made continue to work for deployed WebJobs.

To test other permutations, you can modify the above settings to achieve the desired behavior. E.g., to test the last released Dashboard + latest SDK you can set DashboardSiteExtensionLocation to point to the current production site extension, and upgrade the test project to the version of the SDK you want to test with (e.g. from a local Nuget feed, or by updating the project temporarily to point to your locally build assemblies). You might want to do this if you're making runtime changes to the SDK but no Dashboard changes, and you want to make sure your changes don't regress the last released Dashboard.