1MEIC07T1: Story2Test - FEUP-MEIC-DS-2024-25/ai4sd GitHub Wiki

Story2Test automates the conversion of User Stories into acceptance tests, revolutionizing the Agile development process. By streamlining this crucial task, this helps development teams save time and reduce errors while ensuring tests align with user requirements. This assistant is a member of 2Test.

Vision

Story2Test's objective is to revolutionize the Agile development process by automating the conversion of User Stories into Acceptance Tests (Gherkin syntax). Creating Acceptance Tests is critical but time-consuming, and its main goal is to ensure that the development aligns with the user intent. However, manual testing can cause many errors and inconsistencies, especially for large projects.

Our assistant solves this problem by using natural language processing (NLP) techniques, maintaining the user intent while automatically performing the transformation. By streamlining this process, development teams will save much time while reducing the potential for human errors. This will ensure that the user needs are always the number one priority, improving the efficiency and software quality of many developers.

We want to create a simple tool, yet very powerful, allowing users to almost effortlessly generate the tests. To achieve this, we will combine React, the Gemini API, and RAG(Retrieval-Augmented Generation), ensuring our system is flexible enough to handle a complex variety of user stories, and generate more than one Acceptance Test for each User Story. We will guarantee that testing keeps up with the evolving Agile processes nowadays, making Teams much more productive and delivering high-quality software that meets user expectations.

Research

Below is a survey of similar projects and how they compare to our Story2Test project:

Cucumber: Cucumber is a widely used behaviour-driven development (BDD) tool that helps teams define tests in plain language and then automate them using Gherkin.

  • Pros: It has a very large community, lots of support, and complex and robust capabilities for writing/running tests across a variety programming languages.
  • Cons: It requires manual writing of acceptance tests. Cucumber doesn’t automatically convert user stories into test cases, which can lead to a very time-consuming task.

SpecFlow: Similar to Cucumber (also a BDD tool), but specific to .NET environments.

  • Pros: It integrates very well with the .NET framework, making it ideal for teams working in Microsoft technologies.
  • Cons: Like Cucumber, it requires manual effort to write the tests and doesn't automate the process of converting user stories.

Domain Analysis

Class Diagram

Activity Diagram

Sequence Diagram

Architecture and design

Our current system architecture focus on three key components: React, the Gemini API and Retrieval-Augmented Generation (RAG).

Current Architecture

  • React provides the user-friendly interface for our project. It will also be responsible for the handling of user inputs, requests and responses.
  • Gemini API is the brain of the system. It is responsible for parsing the User Stories and performing NLP tasks in order to convert the text in actionable items that will further be converted into tests.
  • RAG (Retrieval-Augmented Generation) will allow the system to generate more accurate and robust acceptance tests, as it brings external knowledge to enrich the test cases.

Future Architecture

As the project grows, we could improve it by using cloud-based services (e.g. Google Cloud) to handle scaling and enhance performance, especially if we need to process a larger number of activities, or if the system requires more sophisticated NLP capabilities.

Risks and Choices

  • Risks: The reliance on external APIs such as Gemini and RAG introduces some potential points of failure, especially if those services experience downtime or performance issues. There is also a risk associated with scalability if the system's usage grows rapidly as mentioned above.
  • Key Choices: We decided to use React by its ease of use and flexibility for web applications. Similarly, Gemini and RAG were chosen for their strengths in NLP and generating robust and high-quality outputs. These decisions aim to provide a solid foundation while keeping the architecture simple and maintainable.

Diagram

Architecture

Technologies

  • React for web development, chosen for its robust framework and easy handling.
  • Gemini API for interpreting User Stories, providing NLP processing for our project.
  • RAG to enhance tests with knowledge retrieval, and improve the accuracy of the acceptance tests generated, making sure that they align with the user requirements.

Prototype Implementation (Sprint 0): During this sprint, we built a simplified version in order to validate the core transformation process. We used React and Gemini to be able to send a request and receive the corresponding response, that majorly aligns with the requirements.

Development guide

Setup:

  1. Clone the repository and navigate to the avengers/frontend/webapp/src/app/assistants/story2test folder.

  2. Dependencies:

react==4.3.0
google-generativeai==0.8.3
  1. Run the project with: npm run dev

Gemini API

  • Obtain an API key and include it in all requests.

  • Refer to Gemini API for detailed instructions on authentication, endpoints, and request/response formats.

Coding Conventions and Guidelines

  • Code Style: Follow PEP8 guidelines for Python code to maintain consistency.

  • Version Control: Use Git for version control. Commit messages are but descriptive.

Security concerns

We will provide authentication in our system, so that all data can be saved and only accessible by each unique user.

Quality assurance

As the project is in the project, quality assurance has not yet been fully implemented. Below are the current quality assurance practices:

Testing: Even though there are no tests currently implemented, the system is being manually tested and verified for basic functionality

How to use

Below is a example of how to use the tool (simple prompt of a User Story and its respective response with the Acceptance Test and Scenarios)

https://github.com/user-attachments/assets/0fc14309-988d-4995-951b-fd58abc561b8

How to contribute

To contribute to Story2Test, developers should have a basic knowledge in React, Python and API handling.

Create a branch for version control and commit your changes with clear messages.

After that, create a Pull Request in order to merge your changes to the project and help us grow.

Sprint 1 - Retrospective

In this sprint we worked on some features such as conversation history, ability do delete and edit conversations, prompt enchacement, and providing a tutorial on how to use correctly our assistant. We did exactly what we planned for this sprint, so we can say that our objective was achieved. For sprint 2, we will integrate our assistant in the main, along with the other assistants from all the teams.

Contributions