1MEIC01T3: REQ2TEST - FEUP-MEIC-DS-2024-25/ai4sd GitHub Wiki
The goal of the product is to generate Acceptance tests from the project requirements.
Vision
Enjoying the power of AI, Req2Test envisions a world where converting software requirements to acceptance tests is efficient and accessible for all development teams. Our product aims to simplify the acceptance test generation, making it intuitive and helpful for technical or non-technical users to meet the non-functional and functional requirements of the clients.
Research
Cucumber is a tool that supports behavior-driven development using a language parser named Gherkin. It's used to develop test cases based on user stories to test the behavior of the software. It uses simple, but structured, English prompts in Gherkin to create the tests. However, it cannot generate tests from unformatted and ambiguous requirements, written in natural language like Req2Test. A tool that works similar to this one is SpecFlow, that also uses Gherkin syntax to help write tests and requires you to write structured prompts and to manually write the acceptance criteria.
ReQtest is a tool that helps you manage testing, requirements and track bugs. While it is very useful in dealing with requirements and the process of linking them to the tests, it does not automatically create tests based on requirements like our tool.
Domain Analysis
Activity Diagram
Sequence Diagram
Architecture and design
Technologies
The main technologies used in this project are Next.js for the frontend and Python for interacting with the Gemini API. Next.js was chosen primarily because the development team had prior experience with it, making implementation more efficient. Additionally, it provided a seamless way to connect with the Python backend. Python was used to interact with the Gemini API as it offered the most straightforward integration for our needs. Initially, we planned to use Google Cloud, but due to constraints, we opted for the Gemini API for our prototype.
Development Guide
We started using Docker for portability and consistency because it will help us build, share, and run containerized applications. Docker Official Website.
Step 1: Generate a Gemini API Key
Go to the following website to generate a Gemini API key: Gemini API Key Generator
This API key is required to access and use the Gemini AI.
Step 2: Update the .env file
- Rename .env-example to .env
- Inside the .env file, replace the MACRO GOOGLE_API_KEY with the API key generated in the previous step.
Step 3: Create Docker Containers
Now that the API key is set up, you're ready to work with Docker and create the necessary containers: one for frontend and another one API.
docker-compose up --build
In the first terminal run the following commands:
docker run -it -p 8000:8000 --name api-container t01_g3-api
make api
In the second terminal run the following commands:
docker -run -it -p 3000:3000 --name frontend-container t01_g3-frontend
make build
As soon as the containers are running, you can access the Req2Test by visiting the following URL in your browser: Req2Test
Security concerns
The usage of such a tool raises obvious security concerns, namely regarding the use of a network-based service in the context of a project in which confidentiality is paramount, though it also manifests through the sharing of personal or sensitive information with the service. In order to address these concerns, Req2Test will make use of user authentication to provide the necessary confidentiality to its communication channels, thus ensuring that eavesdroppers cannot get access to any valuable data.
Quality assurance
To ensure the quality of Req2Test, we will test the output of our assistant against real gherkin tests by experts in the field. We'll provide high quality training data, such as requirements from diverse domains and ensure that they cover a wide range of edge cases, both simple and complex scenarios. We also plan to do A/B testing: test different versions of the assistant with varying configurations, such as different prompts, to compare performance. Finally, since our focus is the final user of the assistant, we plan to involve human testers to validate the output, especially for edge cases and complex requirements, and to gather their feedback.
How to use
In the following video, you will learn how to use the chatbot and all the features implemented until 24th of October 2024.
https://github.com/user-attachments/assets/e45df8c3-df80-49a3-ac0c-02e7c38bfab2
How to contribute
At the moment we are closing sprint 1. As the team is still developing the project we are always open to ideas, so if you want to contribute to the project, you can always fork, improve, and then make a request. For further details on how to run the project please check the topic: Development Guide
Sprint 1 - Retrospective
what went well? We were able to make start working on integrating our tool in main and made advances on the database and on the implementation of the backend with the firestore. We were able to contribute to the collective effort with a PR regarding the Avengers Integration with a Guide for the Assistants' Integration into the AI4SD that was shared with everyone. We were able to implement 2 PBIs as planned in the Sprint 1 Planning: a user is now able to copy the Gherkin code provided by the tool directly to the clipboard with just one click and also the tool is now able to interpret requirements, even if they have gaps or ambiguities, and create acceptance tests from them anyway.
What didn't go well? In the sprint planning we had a few more user stories for this sprint, that we ended up not finishing because we spent a lot more time than expected working on the integration with the main infrastructure. For sprint 2 planning we took that into consideration when selecting which PBIs we will be working on.
Sprint 2 - Retrospective
what went well? In this sprint, we were able to integrate our tool and features in main, finish the database and implement the backend with the firestore locally, as planned. We were also able to merge our tool with another group's tool, because it made sense based on our assistants, creating 2Test. https://github.com/FEUP-MEIC-DS-2024-25/ai4sd/wiki/2Test
What didn't go well? We thought by now we would have everything working on the cloud, so that next sprint we could focus on more cool features, but we did not manage to do that. We should improve our sprint planning, as for this sprint we had many incomplete PBIs due to unrealistic expectations.
Sprint 3 - Retrospective
what went well? We were finally able to get everything working on the cloud, which was the main thing planned for this sprint. We also did something else for the collective effort: we were responsible for the deployment of the AI4SD Avengers, contributing for it to be now live and operational!
What didn't go well? Initially we thought we would be able to implement authentication related features in our tool, but unfortunately there was not enought time to do that, as getting everything working on the cloud took a lot longer than we expected.
Contributions
Link to the factsheets of each team and of each team member.