Testing a mobile App - nsvir/logTest GitHub Wiki

Using SpiTest you can test a mobile application, using the logs retrieved from the beta-testers. To test any system we need two pieces of information, the log file and the test file that identify the log scenarios.

The Restaurant App overview

The Restaurant App is doing the following scenarios:

  1. Login the user.
  2. Show a list of restaurants.
  3. Select a restaurant.
  4. Show the restaurant informations.
  5. Show the proposed dishes.
  6. Select a dish.
  7. Order a dish.
  8. close the App.

This App contains many scenarios, the framework will assess them one by one, in each user's log.

This can be used to know the usability of your App, the most used buttons and where your users spend the most of the time (analytic metrics).

Once we have all of these scenarios defined as The login scenario bellow, the framework can draw a map of the scenarios run. For example:

Login the user => Show a restaurants list of restaurants => Select a restaurant => Show the proposed dishes => close the App.

If you have a bulk of log files then you can have a stats like:

Scenario Run Warning Fail
Login the user 15 2 0
Show a list of restaurants 45 7 3
Select a restaurant 30 1 3
Show the restaurant informations 24 9 3
Show the proposed dishes 22 4 8
Select a dish 18 2 3
Order the dish 5 2 1
close the App 45 4 0

Other stats can include, the time passed on each scenario, the performance, etc.

Then, you can have a graph of the transitions between scenarios:

** Graph Needed **

The login Scenario

The main content in the log file for the login scenario is as bellow:

09-04 18:28:02.165 27030-27030/fr.spirals.dishes D/LoginActivity: Login button pressed
09-04 18:28:02.165 27030-27030/fr.spirals.dishes D/LoginActivity: Retrieved, User:  username: John, password: 11111111
09-04 18:28:02.165 27030-27030/fr.spirals.dishes D/LoginActivity: Show Progress Bar
09-04 18:28:02.168 27030-27030/fr.spirals.dishes D/Auth: Authenticate, User: username: John, password: 11111111
09-04 18:28:02.168 27030-27030/fr.spirals.dishes D/Auth: Send credentials to the server, User: username: John, password: 11111111
09-04 18:28:05.458 27030-27030/fr.spirals.dishes D/Auth: Received, name : John Matts
09-04 18:28:05.568 27030-27030/fr.spirals.dishes D/Auth: User is Authenticated, name: John Matts
09-04 18:28:05.869 27030-27030/fr.spirals.dishes D/LoginActivity: Hide Progress bar
09-04 18:28:05.869 27030-27030/fr.spirals.dishes D/LoginActivity: Show Welcome back, name: John Matts

Testing the scenario

We will get you through this Restaurant App Test

  1. Testing a simple scenario :
def testLoginScenario():#This high level scenario has to be as general as possible, eazy to understand by the non-technical people.
    scenario("Testing simple login scenario")
    log("android-user123.log", "Android")

    user = login()
    p = expect("Show Progress Bar").ping()
    name = authenticate(user)
    expect("Hide Progress bar").pong(p ,2000 ,5000)
    expect("Show Welcome back, " + name)
  1. Defining the login method (semantic method):
def login():
    behavior("Login the user")#define a semantic to the added behavior
    expect("Login button pressed")
    user = extract("Retrieved,").data()
  1. Defining the authenticate method, this can unit test the method:
def authenticate(user):
    behavior("authenticate the user")#define a semantic to the added behavior
    expect("Authenticate, " + user)
    p = ping("Send credentials to the server, " + user)
    name = extract("Received, ").pong(p, 2000, 5000)#pong and extract
    expect("User is Authenticated, " + name)
    return name

To Do