1MEIC02T03: ADR Generator - FEUP-MEIC-DS-2024-25/ai4sd GitHub Wiki

ADR Generator aims to assist software development teams in generating ADRs (Architectural Decision Records) for repositories with already identified architectural patterns.

More details about the product should be found here below, including the product vision, market research and domain analysis.

Vision

The ADR Generator extension for Visual Studio Code transforms how teams approach architectural documentation. Leveraging a Large Language Model (LLM), it streamlines the process of creating Architectural Decision Records (ADRs) for repositories with pre-recognized architectural patterns. This eliminates much of the manual effort traditionally required and ensures accurate, consistent documentation directly within the developer's workflow.

Our goal is to help software teams effortlessly document architectural decisions with accuracy and relevance. The ADR Generator automatically produces ADRs that reflect the context and reasoning behind design choices, promoting greater clarity, collaboration, and traceability across intricate projects. With context-aware insights from the LLM, architectural decisions are captured in a manner that highlights project-specific nuances, producing detailed, impactful records.

This extension is crafted to integrate seamlessly with the VSCode interface, providing teams with tools to review, adjust, and personalize ADRs through flexible templates and contextual suggestions. As projects grow and adapt, the ADR Generator ensures that crucial architectural knowledge stays relevant, actionable, and in sync with project objectives, supporting informed decision-making and preserving architectural coherence.

By incorporating LLM-driven functionality into the development environment, the ADR Generator strengthens software resilience, fosters shared architectural understanding, and drives innovation with efficient, intelligent documentation processes.

Research

Existing tools like Workik generate ADRs based on user-provided context and requirements through a web interface, offering tailored ADR creation but lacking codebase analysis. This reliance on manual input limits automation and precision compared to the ADR Generator, which analyzes repository code directly to generate ADRs based on identified architectural patterns.

LLM-based tools, like ChatGPT or Gemini, can create ADRs from user prompts, offering flexibility and adaptability. However, their effectiveness depends on user-crafted prompts, and they often lack integration with development environments, disrupting workflows. In contrast, the ADR Generator integrates into VS Code, automating ADR creation via real-time code analysis to streamline developer operations.

Domain Analysis

class_diagram

Architecture and design

diagram-architecture-design

Technologies

We chose Python as the main language for the backend, mostly due to its simplicity, flexibility and compatibility with most APIs. Gemini was chosen as our LLM to provide ADR Generator's AI capabilities. We will use the Visual Studio Code (VSCode) extension to serve as UI, allowing developers to access and adjust ADR records without leaving VSCode.

In Sprint 0, we developed a basic prototype of the extension for VSCode, capable of sending prompts to LLM (Gemini) through a Python integration. This initial implementation was essential to validate the integration with Gemini, confirming that the LLM API meets the needs for generating ADRs.

Development Guide

Building, Running, and Testing the System

Prerequisites

  • Python 3.8+: Ensure Python is installed for running the backend script.
  • Gemini API Access: You’ll need access credentials for the Gemini API, which should be stored securely.
  • VSCode: Required to test the extension integration.
  • Git: For version control and repository management.

Setup

  1. Clone the Repository: Clone the project from GitHub and navigate to the project directory.
  2. Environment Setup:
    • Set up a virtual environment: 
      python3 -m venv env
source env/bin/activate  # For Windows, use 'env\Scripts\activate'
    • Install dependencies: 
      pip install -r requirements.txt
  3. Configure API Key: Store the Gemini API key in a secure location, such as an environment variable: 
    export GEMINI_API_KEY="your_api_key_here"

Running the Script

  1. Activate the virtual environment:

    source env/bin/activate

  2. Run the script

    python script.py

  3. When prompted, provide the GitHub URL and architectural patterns to generate an ADR.

APIs, Formats, and Protocols

Gemini API

  • Endpoint: The Gemini API endpoint processes text-based requests to generate content. It is accessed using the google.generativeai package.

  • Configuration: The generation_config dictionary in the script customizes model behavior, including:

    • temperature: Controls the randomness of the output (0.0 to 1.0).
    • top_p: Controls the nucleus sampling (0.0 to 1.0).
    • top_k: Limits the number of highest probability tokens considered for each step.
    • max_output_tokens: Specifies the maximum number of tokens to generate in the output.
    • response_mime_type: Specifies the desired output format, set to text/plain in the script.

  • Safety Settings: The safety_settings list defines thresholds for different harmful content categories to ensure safe output from the API. Categories include:

    • HARM_CATEGORY_HARASSMENT
    • HARM_CATEGORY_HATE_SPEECH
    • HARM_CATEGORY_SEXUALLY_EXPLICIT
    • HARM_CATEGORY_DANGEROUS_CONTENT

    These safety settings help prevent the generation of harmful content.

  • Model: The script uses the gemini-1.5-pro model from the Gemini API, which is configured with the generation_config and safety_settings.

  • Request Format: The prompt, including the GitHub URL and architectural patterns, is dynamically generated and sent to the API. The template instructs the model to generate an ADR in markdown format.

  • Response Format: The API responds with the generated ADR in markdown format, which is then saved to a file named ADR.md in the project directory.

Coding Conventions and Guidelines

Code Structure

  • Modularity: The script is divided into distinct sections:

    • Configuration: The setup of the generation_config, safety_settings, and API model.
    • Input Handling: Code that prompts the user for input (GitHub URL and architectural patterns).
    • API Request: The creation of the prompt and sending it to the Gemini API.
    • File Handling: Saving the ADR to a file (ADR.md).

  • Error Handling: The script uses a try-except block around the API call to catch and log errors. This ensures the script doesn't crash unexpectedly and can provide useful error messages to the user.

  • Documentation: Each function or section of the code has comments to describe its purpose and behavior.

Naming Conventions

  • Variables and Functions: Use snake_case for variables and functions. Example: generation_config, safety_settings.

Security concerns

  • API Credentials: The API key is currently configured directly in the script via genai.configure(api_key="YOUR_API_KEY"). To enhance security, it is recommended to store API keys securely, such as in environment variables, to avoid exposing sensitive data in version control.

  • Rate Limiting: The script includes basic rate limiting to prevent exceeding the API’s rate limit, especially during bulk operations. The rate limiting logic ensures that requests are spaced out appropriately, with a maximum of 60 requests per minute (configurable).

  • Input Validation: The script validates user input for the GitHub URL and architectural patterns to ensure that the inputs are properly formatted. This helps to prevent errors from malformed input and mitigates potential security risks, such as malicious code injection.

Quality assurance

For the project, considering that Python is the script running on the backend, the application can be tested with Coverage.py, a useful library for Code Coverage Analysis. Additionally, we can use VS Code Testing Integration to integrate tests directly within the editor for extension-specific tests.

How to use

We start with the extension on the left-hand side as in the image extension-closed

When we click on it, the extension opens and looks like this extension-opened

Enter the url and the architectural patterns as in the following image. The result is what ADR shows on the right. extension-result

Sprint Retrospective

Sprint 1

  • What Went Well?

One teammate took the time to learn how to build VSCode extensions, which will be super important for the front end of our system. This sets us up nicely for future work.

  • What Went Poorly?

None of the sprint tasks got done because we had to focus on other projects.

It felt like we didn’t have enough room to balance this project with everything else going on.

  • What Can We Do to Improve?

Plan better during sprints so we can juggle priorities more effectively.

Build in some breathing room for unexpected tasks so we don’t lose all progress on the sprint.

Communicate earlier when priorities shift, so everyone’s on the same page.

  • What Ideas Do We Have?

Let’s have a quick session where the teammate who learned about VSCode extensions can share what they’ve picked up.

Try a staggered approach to sprint planning, so not everyone is tied up in the same sprint tasks at once.

Sprint 2

  • What Went Well?

The visual interface for the VS Code extension was completed by the two assigned team members.

The backend containerization was also successfully finished.

Tasks were clearly assigned at the start, which helped everyone know what they were responsible for.

  • What Went Poorly?

The interface and the backend are not communicating yet, leaving integration work for the next sprint.

Even though PRs were made before the end of the sprint, we couldn’t merge them into the main branch in time.

The main functionality improvement task is still in progress and wasn’t finished as planned.

  • What Can We Do to Improve?

Set earlier deadlines for PR submissions, leaving more time for reviews and fixes.

Add some buffer time in the sprint schedule to account for integration and merging delays.

  • What Ideas Do We Have?

Plan an integration-focused task for the next sprint to connect the interface and backend.

Consider having a mid-sprint checkpoint for critical tasks to identify blockers earlier.

Scrum Board

  • Start of Sprint 2

image

  • End of Sprint 2

image

How to contribute

To contribute to the project, you can fork our repository and then submit a pull request. You can also create an issue on GitHub (for potential errors or bugs) or request a feature that you think is necessary. For more information on how to start the application, as well as the required dependencies and libraries, refer to the setup topic on that GitHub. Additionally, we ask that you avoid leaving your credentials in the modified code and ensure proper documentation and comments.

Contributions

Link to the factsheets of each team and of each team-member.