Performance Debugging - quality-manager/onboarding GitHub Wiki

Performance debugging based on BVT Reports

  • Analysis starts from using the generated BVT report. You should be provided a link like... /jazz/foundation/web/releng/clm_performance_results/rqm/qd-perf-rqm-6061-20190208-0721/report/report.html

Identify the last good Performance build and subsequent failing Performance build

  • Identify the failing test and in the report and click on its name under the Task column
  • On the Response Time Graph Analysis page, click on the historical radio button under plot
  • Remove or add builds on the graph to determine which was the last good build.

Identify the QM builds from the Performance builds above

  • Go to the BVT Performance report.html pages for the builds identified in the previous step.
  • Click on Build Information on each page and look for CLM application tested
  • Make note of the build ID

Get the Diff of all WI Changes between each build

  • From the Team Artifacts view in RTC Client, expand the Builds under Rational Quality Manager
  • Open the RQM Mainline Stable builds
  • Within the build view, find the build IDs from previous step above. Select both, then right click and select compare
  • From the Change Explorer click on the down triangle in upper right and select Save Change Log...
    • On the Save Change Log Dialog:
    • Check Change sets / Show creator
    • Check Advanced Options / Hide unchanged components

Understanding what is tested by the BVT Perf Test

There are 2 types of Performance BVTs. REST API tests and Browser tests

REST API Tests

  • Proxies are used by the performance test are defined in our repo at Test Automation / com.ibm.rqm.server.proxy.test.performance
  • Given a task name like ICompositeWebRestService_getOpenCreateData_BuildRecord_Task this would correspond to com.ibm.rqm.web.common.service.rest.API.ICompositeWebRestServiceAPI.getOpenCreateData()
  • Further information about the task can be gathered from the performances tests
    • From the Team Artifacts view...
    • The tasks are defined in Jazz Collaborative ALM / CLM Performance *version* / RPP rqm / com.ibm.team.performance.rqm.tasks/src/com/ibm/team/performance/rqm/tasks/sequences/BenchmarkSequence

Browser Tests

  • The report.html page will have an expandable test name that may reveal a URL hint about what is tested.
  • These tests are RPT tests and are defined in Jazz Collaborative ALM / CLM Performance *version* / CLM_4.0_Workloads / BVT /RQM
  • Without RPT, you cant see the definition of the test. You may need to contact the performance team for better understanding what is performed by the test.

Analyzing diff results

  • If the list is very long, add the list to the WI and ping each person who contributed a changeset and ask for assistance identifying any related changesets to the performance task that failed.
  • Go through each changeset and narrow down which changesets seem to affect related areas of code
  • Look for possible inefficiencies in newly delivered code. If changes are large, you may wish to ask for assistance from the original authors of the changesets to better understand the code.