WebDriver Based Web Test Automation - PeterWangIntel/crosswalk-website GitHub Wiki

#Introduction

This session introduces a solution to extend WebDriver based test automation from browsers to web applications. This solution implements BDD (Behavior-driven Development)principles to allow tests developer to use tests written in a ubiquitous language style to operate UI of web applications, provides a unified interfaces for test automation of both web and native applications, and it also brings a new approach to solve automation of existing traditional manual tests by WebDriver.

#Background

WebDriver

WebDriver as a new W3C standard defined by a set of interfaces to discover and manipulate DOM elements on a page, and to control the behaviour of the containing browser, is widely applied in popular web browsers.

Selenium2.0 is the most popular instance of W3C WebDriver specification, it supports almost all popular browsers and easy to combine with popular program language.

##Web Tests Generally, web tests mainly written by JS/CSS/HTML and related media resources. In web community, a large number of web tests are waiting for a effective and convenient way run them on target browsers or web runtime. Normally, web industry divide those existing web tests into the following categories:

• "js" tests: those tests are all using w3c javascript test harness to implement the test assertion, show test result and error message after execution automatically, they are all traditional "auto" test cases.
• "ref" tests: the test case not only has a test file, but also has a refer file, the test result should be pass if the rendering of the test file equals reference test rendering. Most of them are traditional "manual" test cases.
• Others: named "General Web GUI Tests" which no unified test harness or test method, some of them need some UI interactions, and some of them need test platform native operations ......

Normally, web tests have two deployment ways:

• Online Service: The tests host on a HTTP server in network, the test device access the service and select the web tests to run. This way is mainly used by browsers testing, Web Testing Service project covers all details
• Packaged Web Tests Application: Mainly used by web runtime test scope, the web Tests be packaged into special application packages, and be installed on the test target device for testing.

#Challenges

• Hard to apply image recognition or platform native UI automation solution to web space
• Multi-formity of web UI layout and elements requires a more convenient test design and development method
• Existing browser test automation techniques and tools have gap on support on web applications or web runtime, need an overall test framework which should easily to use for not only browsers but also web runtime and web applications.
• Need a common way to implement both web and native UI automation for the "General Web GUI Tests" tests.
• Need a common framework to run the large number of existing web tests automatically, e.g. existing automated "js" tests and "ref" tests.

#Solution In this new solution, WebDriver works as another web test engine, unlike the general web test engine which uses HTTP requests for communications between web applications and browsers or web runtime, WebDriver controls the inside of web Applications at the outside of browsers or web runtime.

On the one hand, the solution should focus on "General Web GUI Tests" tests which need behavioral steps and interactions on both web and native applications. On the other hand, WebDriver helps testing tools to run the web tests(e.g. "js" and "ref" web tests) which has existing test harness automatically and conveniently.

This figure shows the typical implementation of WebDriver based web test automation.

This framework also involves testkit-lite as the tests runner, the tests be organized by XML file. This introduction of the solution mainly focus on the second deployment way as not all of web tests can be run on the service mode although WebDriver can be easily applied on first way too.

##General Web GUI Tests The "General Web GUI Tests" involves testkit-lite "script" test set type for test execution. Particularly, for those tests, the test framework involves BDD principles to organize and execute the tests effectively and conveniently. The overall BDD principles for WebDriver web tests automation host on WebDriver Based Behavior Driven Test.

##"js" Tests This figure shows the processes of handling the "js" tests by WebDriver:

.

##"ref" Tests This figure shows the processes of handling the "js" tests by WebDriver:

.

#Tests Development As stated previously, WebDriver web test engine supports all three types of web tests. This section will introduce the details about how to automate web tests by WebDriver web test automation. The section will not cover the general tests development and packing as the Crosswalk web tests project, testkit-lite project and Behavior Driven Test Automation have the detailed steps to show how to develop web tests for Crosswalk test framework.