GSOC 16 Report - smarshy/vms GitHub Wiki

Weekly Reports

Wiki Page for description of GSOC work for Systers Volunteer Management System

As a part of GSOC 2016, the following problems have been tackled for automated testing for VMS -

  • Reducing and managing dependency on UI elements - Earlier, the tests were extremely brittle. Any single change in UI required several changes in the tests themselves
  • Reducing time and duplication for tests - The tests reused very little parts of code and data. When they were run on the local machine, they also took up a lot of time (approx. 1.25 hrs)
  • Continuous monitoring for changes in VMS - Since the tests were in a different repository, they were no longer up to date with the changes in VMS over the last one year. Also, they weren't run against each proposed change to the project
  • No database checks were present
  • There were many situations and cases which were untested in both unit and functional tests

Links:

For submission purpose, systers automated testing repository has been used while all the actual changes have been made to the main VMS repository. Code has been merged to the automated testing repository only once it has been reviewed for VMS.

Completed tasks:

Tasks Related Issue/PR in VMS Status
Added automated tests to vms and synced them as per latest code PR-315 Merged
Integrated VMS with Travis Issue-140, PR-332, PR-334, 343 Merged
Reported new bugs in vms Issue-325, Issue-326, Issue-327, Issue-336, Issue-337, Issue-340, Issue-345, Issue-350 N/A
Increased test coverage via functional tests Issue-346, PR-347, PR-335, PR-357 Merged
Minor changes to configure coverage for vms PR-348, PR-353 Merged
Reduced time, duplication for unit tests Issue-344, Issue-369, PR-356, PR-339 Merged
Increased test coverage via unit tests Issue-346, PR-358, PR-360 Merged
Reduced time, duplication for selenium tests Issue-369, PR-370 Merged
Migrated entire test architecture to POM Issue-369, Issue-376, PR-375 Merged
Added Database checks for CRUD operations PR-380 Merged
Documentation PR-378 Merged

Results/ Summary

  • Migrated entire selenium test architecture of around 115 tests to follow page object model design, thus increasing scalibility and ease of maintenance for the system

  • Refactored existing 167 tests to increase re usability and observed following approximate reductions:

Test Type Time Duplication Database objects
Selenium 41% 30% N/A
Unit 50% 50% 74%
  • Increased test coverage to 83% by fixing errors in initial 76 tests, adding 45 new selenium tests, extending 22 unit tests and adding database checks to 64 tests
  • Setup Travis and local Browserstack integration for vms

Detailed documents recording reductions - unit and selenium

Guide to Work Continuation:

  • Local Browserstack integration has been setup for my fork of VMS using username and local access key for my account. For the main project, these need to be updated to use the Systers account. Also, the desired capabilities need to be updated as per requirements and the results of the tests need to be recorded. If these tests are to be run automatically, Browserstack needs to be integrated with travis.
  • Coverage report needs to be inspected to find out which lines are being missed and then tests should be expanded to address relevant lines.

Weekly Reports

Short Bio

I am a final year undergraduate pursuing Electronics And Communication Engineering from Visvesvaraya National Institute of Technology, Nagpur. I have a strong inclination towards programming and I continue to enjoy it more everyday. Apart from coding, I like travelling, sketching and experimenting with food.

Profile Links

Github
LinkedIn
Facebook

Automated Testing

I am working on automated testing for VMS (Volunteer Management System) this year. I would be building on top of the framework created previously, which is based on Python, Django and Selenium Webdriver. My main focus is on enhancing the maintainability, performance and increasing the coverage for the system. My mentors are Nidhi Makhijani and Tapasweni Pathak.

Github Repository
Summary of work done

Project details and Timeline

Proposal

Blog

Week 1
Week 2
Week 3
Week 4
Week 5
Week 6 and 7
Week 8,9,10 and 12
Week 11
Week 13

Related Documents

Drive Folder

AUT VMS - Weekly Status Report for Week 0

What have you accomplished this week (list specific items accomplished)?

  • Researched more into recommended tests for a web application
  • Completed setup and resolved selenium compatibility issues
  • Got in touch with the dev intern regarding changes planned for vms this summer
  • Updated timeline and proposal
  • Had a hangout with mentors to discuss project plan
  • Cleared up doubts regarding where I should write tests

What issues or roadblocks have you encountered this week?

  • None yet.

Have they been resolved, and if so, how?

  • NA

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • NA

Is further assistance necessary to resolve existing issues?

  • NA

What do you plan to accomplish next week?

  • Merging tests with vms repository
  • Resolving test errors which might occur due to changes in the code in the past one year

How does your progress compare to your project schedule?

  • On track.

AUT VMS- Weekly Status Report for Week 1

What have you accomplished this week (list specific items accomplished)?

  • Added existing automated tests to vms
  • Searched commit histories and files for tracking changes leading to breakage in numerous tests
  • Resolved errors occurring in 76 tests
  • Covered tests for a few new features, like dropdowns

What issues or roadblocks have you encountered this week?

  • Plenty of errors and failures (much more than I had anticipated). Earlier, all 101 tests were passing. However, after syncing and running tests for the latest code, 76 tests seemed to break.
  • Strange errors in accessing elements and in triggering actions, even though the page structure did not seem to change.

Have they been resolved, and if so, how?

  • Yes :)
  • Tackled errors for one django app at a time, thus breaking the work into smaller parts.
  • Searched google for selenium errors. Could not understand the reason for a few of them, but was able to resolve them by taking suggested steps.

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • Nope.

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Integration with Travis
  • Writing functional tests for new features

How does your progress compare to your project schedule?

  • On schedule.

AUT VMS- Weekly Status Report for Week 2

What have you accomplished this week (list specific items accomplished)?

  • Reported new bugs in vms
  • Configured travis file for continuous integration
  • Made a list of tests for missing scenarios for registration and home page

What issues or roadblocks have you encountered this week?

  • None.

Have they been resolved, and if so, how?

  • NA

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • NA

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Complete selenium tests for home, registration, event sign up and log hours functionality
  • Finalise tests required for event, shift and job creation by admin

How does your progress compare to your project schedule?

  • On Schedule.

AUT VMS- Weekly Status Report for Week 3

What have you accomplished this week (list specific items accomplished)?

  • Wrote tests for validation of location, phone number, email and organization fields for both admin/volunteer registration forms
  • Wrote tests to check for retention of form field values on reloading
  • Wrote tests for shift sign up when shift is outdated or has no slots
  • Wrote tests for Search event functionality
  • Made a list of missing test scenarios for admin functionality, including event, shift and job creation
  • Reported new bugs in vms

What issues or roadblocks have you encountered this week?

  • None.

Have they been resolved, and if so, how?

  • N/A

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • N/A

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Write tests for event, shift and job creation
  • Write any missing tests for admin/volunteer
  • Start refactoring unit tests to follow fixtures

How does your progress compare to your project schedule?

  • On Schedule.

AUT VMS- Weekly Status Report for Week 4

What have you accomplished this week (list specific items accomplished)?

  • Wrote tests for null values in create/edit event, job and shift forms
  • Wrote tests to check deletion of shift with volunteer
  • Wrote tests to check validity of dates for create/edit job
  • Wrote tests to check validity of dates for create/edit event
  • Wrote tests to check validity of date and time for create/edit shift
  • Wrote tests for retention of fields in create/edit event, shift and job forms
  • Wrote tests for validity of hours in log hours functionality
  • Wrote tests for view shift details and view job details
  • Wrote tests for validity of max_volunteers field
  • Wrote test for an event with elapsed start date
  • Read about test design patterns recommended for maintainability of code
  • Narrowed down on more tests that need to be added
  • Found new bugs in vms

What issues or roadblocks have you encountered this week?

  • None.

Have they been resolved, and if so, how?

  • N/A

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • N/A

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Write tests for simplify job/shift form
  • Write tests for checking event and job uniqueness
  • Work with dev team to solve build failures in the PRs
  • Refactor unit tests to follow fixtures in administrator, authentication, event, job, home, organization and volunteer apps

How does your progress compare to your project schedule?

  • On Schedule.

AUT VMS- Weekly Status Report for Week 5

What have you accomplished this week (list specific items accomplished)?

  • Modularized code thus, reducing duplication by over 40% for event, job, volunteer and organization unit tests
  • Wrote tests for checking event and job uniqueness
  • Wrote tests for simplify job/shift form
  • Submitted PRs for 30 new tests (including those added last week)

What issues or roadblocks have you encountered this week?

  • None.

Have they been resolved, and if so, how?

  • N/A

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • N/A

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Modularize unit test code in shift app
  • Add fixtures to all unit tests
  • Read more about POM and restructure automated tests to reuse code via functions/loops
  • Work with dev team to solve build failures in the PRs

How does your progress compare to your project schedule?

  • On Schedule.

AUT VMS- Weekly Status Report for Week 6

What have you accomplished this week (list specific items accomplished)?

  • Modularized code for shift unit tests
  • Added fixtures to unit tests
  • Read about database tests, went through POM examples
  • Analyzed tests to find code which is potentially reusable

What issues or roadblocks have you encountered this week?

  • Delay and loss of some work due to issues with my laptop

Have they been resolved, and if so, how?

  • No

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • Yes, I am doing the setup on another one, but the work hasn't been recovered yet.

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Refactor automated tests to follow POM

How does your progress compare to your project schedule?

  • Slightly behind, I had also planned to start working on selenium tests this week.

AUT VMS- Weekly Status Report for Week 7

What have you accomplished this week (list specific items accomplished)?

  • Resolved errors in refactored unit tests regarding creation of same objects with unique keys across different test files
  • Analyzed selenium code to find reusable parts and data
  • Modularized code for registration, event and shift

What issues or roadblocks have you encountered this week?

  • Work has been generally delayed as I have had to redo the setup. In particular, setup of selenium tests on virtualbox is still causing problems.

Have they been resolved, and if so, how?

  • No

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • Yes. I have followed solutions listed online relating to installation of xfvb/similar components and modifying display configurations. But nothing seems to work till now.

Is further assistance necessary to resolve existing issues?

  • No, I am still working on it.

What do you plan to accomplish next week?

  • Modularize code for volunteer, home and administrator
  • Refactor automated tests to share data and use POM
  • Write DB tests, if time permits

How does your progress compare to your project schedule?

  • Behind schedule.

AUT VMS- Weekly Status Report for Week 8

What have you accomplished this week (list specific items accomplished)?

  • Broke selenium code into modules across all 8 apps
  • Resolved errors on virtualbox
  • Reduced code duplication and time for all tests
  • Broke all UI elements as per their respective pages

What issues or roadblocks have you encountered this week?

  • None.

Have they been resolved, and if so, how?

  • NA

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • No.

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Finish migration of code pattern
  • Add Database tests
  • Extend Unit tests

How does your progress compare to your project schedule?

  • On schedule.

AUT VMS- Weekly Status Report for Week 9

What have you accomplished this week (list specific items accomplished)?

  • Cleaned unnecessary tests in the files
  • Extended unit tests for event, job, organization and volunteer
  • Setup page objects and linked them partially to the tests

What issues or roadblocks have you encountered this week?

  • Restructuring of all the selenium tests taking more time than anticipated.

Have they been resolved, and if so, how?

  • No.

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • Yes. I am completing it simultaneously with the work I had planned for the upcoming weeks so that there is minimum delay.

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Finish linking page objects to the tests
  • Add Database tests
  • Extend Unit tests for shift

How does your progress compare to your project schedule?

  • Slightly behind schedule.

AUT VMS- Weekly Status Report for Week 10

What have you accomplished this week (list specific items accomplished)?

  • Extended Unit tests for shift
  • Generalised utility functions to maximise reuse
  • Discussed and analysed errors in tests for development PRs

What issues or roadblocks have you encountered this week?

  • Restructuring of all the selenium tests taking more time than anticipated.

Have they been resolved, and if so, how?

  • No.

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • Yes. I am completing it simultaneously with the work I had planned for the upcoming weeks so that there is minimum delay.

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Finish linking page objects to the tests
  • Increase coverage by adding db tests/ tests for views
  • Integrate tests with browserstack

How does your progress compare to your project schedule?

  • Slightly behind schedule.

AUT VMS- Weekly Status Report for Week 11

What have you accomplished this week (list specific items accomplished)?

  • Added page locators for different pages in vms
  • Linked page objects to tests for registration, event, home, volunteer, job and shift
  • Recorded and documented test timings and improvements for all selenium tests
  • Started working on browserstack integration

What issues or roadblocks have you encountered this week?

  • Error while integrating with Browserstack.

Have they been resolved, and if so, how?

  • Yes, partially. I reached out to my mentor and she gave some suggestions.

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • No.

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Link administrator and authentication app page objects to tests
  • Documenting test cases and new design pattern
  • Finish Integration with browserstack
  • Add database tests/tests for views if time permits

How does your progress compare to your project schedule?

  • On schedule.

AUT VMS- Weekly Status Report for Week 12

What have you accomplished this week (list specific items accomplished)?

  • Linked administrator and authentication page objects to tests
  • Migrated entire test architecture to pom based design
  • Documented new design pattern for tests

What issues or roadblocks have you encountered this week?

  • None.

Have they been resolved, and if so, how?

  • N/A.

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • N/A

Is further assistance necessary to resolve existing issues?

  • No.

What do you plan to accomplish next week?

  • Documentation
  • Finish Integration with browserstack

How does your progress compare to your project schedule?

  • On schedule.

AUT VMS- Weekly Status Report for Week 13

What have you accomplished this week (list specific items accomplished)?

  • Added database checks for CRUD operations
  • Setup local integration of tests with browserstack
  • Documented new architecture design
  • Submitted code and final evaluation

What issues or roadblocks have you encountered this week?

  • None.

Have they been resolved, and if so, how?

  • N/A.

Do any of the issues or roadblocks still exist and what steps have been taken to resolve them?

  • N/A

Is further assistance necessary to resolve existing issues?

  • N/A

What do you plan to accomplish next week?

  • Resolve any possible conflicts/issues in PRs
  • Help new contributors to the project

How does your progress compare to your project schedule?

  • On schedule.