Proposal ‐ QA Improvement - kent-cheung-usps/MyStory GitHub Wiki

Quick Draft on Development Gaps:

  1. Requirements Gap: Missing or misunderstood business requirements that reduce delivered business value.
    • Example: PFSC Splash Screen
  2. User Expectation Gap: Misalignment between what users expect and what is actually delivered.
    • Example: Missing 508 Compliance, lack of end-to-end system integration.
  3. Documentation: Insufficient documentation of component interactions, data flow, and system architecture, leading to confusion during development and inefficient problem-solving.
    • Example: Missing Software Architecture Document & Troubleshooting Guide
  4. Quality Gap: Insufficient test coverage resulting in defects leaking into production.
    • Example: Code coverage belows 80%

Based on the discussion in Mark's team meeting on June 20, quality improvement appeared to be an important topic. Therefore, the following discussion will focus on quality analysis.

Detail SQA Gaps

  1. Source Code coverage is below 80%.
  2. Integration tests with dependent services are missing.
  3. Continue testing pipeline (Jenkins) is broken.
  4. There is no troubleshooting guide available.

Proposed Solutions

  1. Ensure Consultant team follows USPS development and quality Standards.
  2. Require the Consultant team to create a troubleshooting guide for all stakeholders (Dev/QA/Support)
  3. Ensure the QA Team provides continuous validation, real-time reporting, and traceability from requirements to defects.
  4. Improve code coverage to at least 80%.

Propose Short Term Action Items (Role Alignment)

  1. Improve Overall Code Coverage to 80%.
    • Status: Work in progress.
    • Current Challenges:
      • Frequent context switching to other tasks
      • New code introducing regressions and increasing test maintenance overhead.
    • Recommendation: Allocate approximately 30% of development time to writing and maintaining unit tests to reduce technical debt and improve code coverage.
    • Measurement:
      • Increase in number of unit tests
      • Measurable increase in code coverage percentage.
  2. Resurrect the broken CI/CD automation pipeline.
    • Status: Broken/Unknown (Refer to the screenshot)
    • Recommendation:
      • Verify proper Jenkins pipeline setup by running a basic UI test.
      • The test should log in with an existing user and validate existing static data.
      • Ensure consistent generation and delivery of the last execution test report.
    • Measurement:
      • Successful daily execution of a basic UI test via CI/CD pipeline.
      • Documented setup requirements and troubleshooting steps.

Long Term Action Items

  1. Ensure Adherence to Standards:
    • Ensure the consulting team follows USPS development and quality standards.
  2. Maintain Comprehensive Documentation:
    • Require the consulting team to provide up-to-date Software Architecture Documentation and Troubleshooting Guides.
  3. Implement Advanced QA Strategies:
    • Require the consulting team to implement an integrated test automation and A/B testing strategy using CI/CD and ALM tools for continuous validation, real-time reporting, and traceability.
  4. Validate System Integration:
    • Validate end-to-end system integration across all dependent services, APIs, and cloud components.
  5. Incorporate Comprehensive Testing: -Incorporate performance, scalability, and security testing into the QA strategy.
  6. Implement Disaster Recovery and Data Backup:
    • Ensure disaster recovery (DR), data backup, and rollback mechanisms are in place and validated.

References

Broken UI Automation Infrastructure: image

SonarQube Report: image

SAD: https://github.com/kent-cheung-usps/MyDocuments/blob/main/VaccineFinder_Sample.pdf

Need: CIAM team authentication/authorization API call. Stable Jenkins for automation. Open POP3 or IMAP for Jenkins node.