1MEIC06T2: Diagram Detective - FEUP-MEIC-DS-2024-25/ai4sd GitHub Wiki

The goal of our AI assistant is to use large language models (LLMs) to analyze architectural diagrams and other visual information in software repositories, enabling the inference and identification of architectural patterns present in the system.

More details about the product should be found here below, including the product vision, market research and domain analysis.

Vision

Our product is primarily aimed at users in the software development industry, especially professionals working in software development. We believe that other professionals in the industry can also use it, even if out of curiosity. University students will also benefit from a tool to further their knowledge in this area. It can be more difficult for users outside of this domain to use the product, not because of its complexity, but for technical reasons related to the field.

The goal of our product is to analyze architecture diagrams in repositories like GitHub, with the goal of finding patterns that are current. To achieve this, we will use artificial intelligence through Google’s Gemini and other LLMs. For each diagram, we will also provide a description of the observed pattern. It is important to note that the description will be general and will not examine specific details of the diagram provided.

This will allow users to quickly identify architectural examples in their diagrams, facilitating understanding and communication of best practices. Students will be able to learn about software architectures and patterns through interaction with the plugin, providing a practical approach to theoretical knowledge. Finally, those without technical experience learn even with artificial intelligence users can quickly and easily access information about software architecture.

Research

Several tools assist in visualising and documenting software architecture. While none fully integrate AI or LLMs for automatic pattern inference, they offer valuable features. The third tool, Enterprise Architect, is the closest to aligning with our project’s goals.

Structure101

Pros: Focused on visualising code dependencies and architectural structure, allowing teams to have a clear view of dependencies and potential areas for refactoring.
Cons: Does not offer analysis or automatic inference of architectural patterns and does not integrate LLMs for detailed or interactive explanations. Lacks support for images or architectural diagrams.

PlantUML

Pros: Allows the creation of UML diagrams from text, assisting in the development of architectural and software structure diagrams.
Cons: Does not perform inferences or analyses on the created diagrams. It is purely a visualisation tool without AI integration for interpreting or explaining patterns.

Enterprise Architect

Pros:

Offers a wide range of features for UML modelling, including class diagrams, sequence diagrams, component diagrams, and other architectural diagrams.
Has good capabilities for generating automatic documentation from diagrams.
Allows dependency analysis and generates detailed reports on system design.
Supports the identification of architectural patterns such as MVC, service layer, and other well-known design patterns.

Cons:

Does not use AI or LLMs to generate detailed or interactive explanations of diagrams or architectural patterns.
Architectural pattern analysis is limited to those predefined in the system, with no robust capability to infer new patterns from unstructured data.
Lacks native support for analysing visual diagrams that are not UML-based, such as 3D architectural diagrams or unstructured images.

Domain Analysis

In the domain diagram below, we identify the main entities that are part of our system and their relationships.

Domain diagram

User: Represents the person who interacts with the tool, submitting images or repositories for analysis.
Repository: A software repository that contains images and other relevant visual information for analysis.
Chat: A communication channel between the user and the language model for conversations and inquiries about results.
Large Language Model: The language model that analyses repositories and images, inferring architectural patterns and answering questions.
Pattern: Represents architectural patterns inferred by the LLM, with associations to practical examples.
Image: Images that the user can submit directly or that are contained in a repository for analysis.

The sequence diagram of the main functionality of our system is presented below: detecting architectural patterns from diagram images. In this case, the user provides a repository for the system.

Sequence diagram

Architecture and design

DS_diagrama

Frontend: This container is responsible for providing a client interface, handling user interactions, communicating the backend API and is composed of 4 components:
- ChatPage.js: Chat page initializes and is composed by Chat.js and Input.js
- Chat.js: Chat component is responsible for rendering user and server responses
- Input.js: Input component is responsible for rendering input box and sending user inputs to the backend
- Home.js: Home page component contains informations regarding our AI assistant
Backend: This container is responsible for business logic, data management and handling integrations with third-party services like Gemini or other microservices. It is composed of 4 components:
- models.py: This component is responsible for defining the data models.
- model_setup.py: This component is reponsible for providing gemini with the necessary prompts to prepare it to provide the best possible responses.
- urls.py: This component is responsible for routing the urls to the correct views.
- views.py: This component contains the logic and handles requests and responses.
DB: This container is responsible for storing Chats.

Technologies

As of the conclusion of sprint 0, the following technologies have been integrated into our development process.

Python

The team chose Python as the main development language as it was the language most of the team was proficient in, allows easy integration and API setup, and its versatility allows for ease in adjustments in a constantly ever-growing and ever-changing project.

Django

The Django framework provides a fairly simple development environment that fits our full stack needs. Django provides the comprehensive range of features needed for both the backend and front-end development.

Gemini

Google's Gemini API was the recommended generative AI API. Gemini is used to analyse architectural relevant images and infer architectural design patterns of a software product. Our app provides Gemini with the relevant Architectural Design literature so that it can more accurately analyse, infer and retrieve Architectural Knowledge from the given images.

Docker

Docker was used to package entities of our project into containers to ensure consistency across development environments. Docker also provides a platform to develop and maintain the microservices architecture we aim for.

React

As we reached the end of sprint 0, we chose to re-structure the project to implement a Microservices driven approach. When dividing the project into microservices, we chose to implement a React application to replace the previously developed front-ent, which mainly used Django. This React application provides an interface for users to interact with Diagram Detective.

Development guide

This guide provides all the essential information needed for a new developer to contribute for the project, including setup instructions and an overview of the project structure.

Setting up the environmental variables

To be able to run the project, the developer needs to create a .env file with the following structure:

# This file contains the environment variables for the project
# Do not commit this file to the repository

# React settings
REACT_APP_API_URL=http://localhost:8000/

React and the Docker Compose file will search for this file to get the variable values needed to configure the application's environment.

Setting up the Frontend

The frontend is shared between all assistants. More information about this can be found here.

Setting up the Backend

Since we are using Django for the backend, the developer has to make sure to have Python installed.

Python has a virtual environment module that allows you to create isolated environments for Python projects. This is useful because it allows you to manage dependencies for different projects without interfering with each other.

To create a virtual environment, run the following command in the root directory (if you use VSCode, Ctrl + Shift + P -> Python: Create Virtual Environment):

    python -m venv venv

To activate the virtual environment, run the following command in the root directory (if you used VSCode, it will activate it automatically):

    venv\Scripts\activate # Windows
    source venv/bin/activate # Linux

The dependencies of the project will be listed in the requirements.txt file.

To install the dependencies, run the following command in the root directory of the project:

    pip install -r requirements.txt

(NOTE: before running the server, make sure to install the packages necessary to connect the backend to the database and create a database user, see section Database)

Since we created a Django project, to run the server, make sure to be in the backend directory and run the following command:

    python manage.py runserver

The server will be running at http://localhost:8000/.

Setting up the Database

For the database we are using Firestore along with other assistants that also benefits from a database. More information can be found here.

Project Structure

To separate the project into microservices, we will have the following components:

Frontend: The user interface of the application. This will be a React application that will communicate with the backend.
Backend: The backend of the application. This will be a Django application that will communicate with the AI assistant.
AI Assistant: The AI assistant that will analyze the architectural diagrams and other visual information in software repositories.
Database: The database that will store the chat history between the user and the AI assistant.

The backend API will be on the backend/app directory, and is going to correspond to the urls.py file. In this file we tell Django that, for example, to get the chat history, it needs to access the URL app/get_chat_history that will be linked to a function on the views.py that deals with that request. It's in the views.py file where the communication with the LLM occurs.

Current API

The project's API currently looks like the following:

/admin : When creating a Django project, it creates an Admin dashboard
/app : Our homepage, provides a description of the project and allows users to try the app
/get_content : When the user inputs a repository URL or images, this API is called to send the data to a function in the backend that analyzes the input and sends it to the LLM to be evaluated. It then returns the data generated by the LLM.
/get_chat_history : Retrieves the chat history from a given session in the database and shows it to the user.
/get_sessions: Retrieves all sessions stored in the database
/create_session: Creates a new session to start a new chat
/update_session_name: Allows for the user to customize the session name
/delete_session: Deletes a session

Security concerns

Some of the potential security vulnerabilities that could affect this system are the following:

Sensitive information exposure : Since Diagram Detective is scanning repositories, it might store sensitive information like IP adresses and the exposure of possible confidential information like this should be avoided.
Denial of Service (DOS) : Excessive amount of queries can overload the model, and quickly go over the rate limit that the gemini API provides.
Injection attacks: Since the system receives user inputs like images and urls, injection attacks are a possibility that should be considerated.
Insecure API Access: The backend API should be equipped with solid authentication measures.

Quality assurance

Describe which tools are used for quality assurance and link to relevant resources. Namely, provide access to reports for coverage and mutation analysis, static analysis, and other tools that may be used for QA.

How to use

To use our tool, copy the GitHub repository URL you want to analyze and paste it into the chat box. Run Diagram Detective, and it will provide a structured Markdown file, which you can download, with the analysis and inference conducted. After this, you can engage the tool in a conversational chat, to extract further insights, or use templated requests available through the interface, to obtain other useful information (e.g. link to literature about the pattern). You can also customize your level of expertise, to ensure answers are tailored more toward your needs. A Demo video, demonstrating all of this in action, shall be added here shortly.

Development Timeline

Throughout the development of our project, we were faced with some challenges that required changing the structure of the project.

We first started with a monolithic project, where we only used Django. A Django project offers tools for the backend, a template system for the frontend and an embedded SQLite database. We used this architecture to build the prototype which we presented.

Then, we faced the challenge of separating the different parts of the project into microservices, which resulted in the current architecture mentioned in this Wiki. We chose to separate the database, backend and frontend and use different technologies for each.

The previous structure is still present in the repository (directory aad) since this was the code used to build the prototype. The new architecture can be identified by the directories backend and frontend, the database folder is purposely missing because it stores sensible content that should not be pushed into the repository, and since we are using Django (it can build the database tables through the file models.py) we don't need the .sql file to generate and seed the database.

The assistant's current structure consists on backend (Django), a shared frontend (ReactJs) and a shared online database (Firestore).

How to contribute

Our tool is developed in Django and follows a microservices architecture, decoupled into a Python backend and a React frontend, integrated into the AI4SD ecosystem of tools. To contribute to the project, please contact one of the members below (preferably the Product Owner). Technical details about the code, building, and testing are specified in the Readme included in the tools repository, here

Contributions

Link to the factsheets of each team and of each team-member. For example:

Team 2
- David Cordeiro (PO)
- Manuel Serrano (SM)
- Ana Sofia Teixeira
- Eva Carvalho
- Tomás Ribeiro

Sprint 1 Retrospective

1. Sprint Goals

The goals for Sprint 1 were:

Containerize our assistant
Integrate it into the AI4SD ecosystem
Implement all user stories from the Sprint 1 planning

2. Completed Work

Assistant containerization
Architecture improvement suggestions
Custom level of expertise

3. Uncompleted Work

Rate limit warnings
AI explanation of how it reached its conclusion
Literature references on inferred patterns

Reasons: Bugs that arose during development. The 3 user stories are currently under review.

4. Challenges and Lessons Learned

We need to estimate the value of the user stories we are working on better, and make sure we are always working on what is most valuable.
We need to communicate more with the client and value his feedback more.
PO needs to pay more attention to the scrum board and the wiki.

5. Next steps

Fix the uncompleted user stories from Sprint 1
Begin implementing user stories chosen for Sprint 2
Start AI4SD integration

Sprint 2

Initial Scrum Board

sprint2_initial_1 sprint2_initial_2

Final Scrum Board

s2_final_backlog_1 s2_final_backlog_2

Items left in progress, were passed to sprint three's backlog.

Backlog Refinement

backlog_refinement_1

Subdivided the work item 'AI4SD Integration', into 4 more granular work items, for organizational purposes, and to be able to better accomodate the timing of some cloud pipeline decisions/developments (integration of secrets).

Documentation

backlog_item

All backlog items beeing worked on were converted to issues.
Every User Story has acceptance tests.

AI4SD Integration/Contributions

Frontend has been integrated.
Successfully migrated from our database system to firestore.
Completed an initial super-hero build.
Joined discussions and planning regarding Jarvis.

Assistant Specific Increment

Made several improvements to the frontend look, taking into account aesthetics, intuitiveness and cohesion with the overall frontend look, between 'web app' assistants.
There is a way for the user to now get an explanation from the AI on how it inferred the patterns that it inferred.
The provision of literature for identified patterns and the issueing of rate limit warnings as completed reviewing and is now properly integrated.

Demo

Sprint Retrospective

What Went Well? 🎉

Improved communication between team members.
Better involvement with other teams, and with the overall project.
Better estimation of what we were able to achieve during the sprint.

What Didn’t Go Well? ❌

Technical debt (secret integration) impacted new feature implementation.
Test features more thoroughly, and make sure to review code carefully and soon.

What Can We Improve? 🚀

Allocate time to address technical debt during each sprint.
Schedule a mid-sprint review to catch delays early/have more time to adjust.
Testing and code review.

Happiness Meters (😔 -> 😐 -> 😁)

David:

Eva - 😁
Sofia - 😁
Tomás - 😁
Manuel - 😁

Manuel:

Eva - 😁
Sofia - 😁
Tomás - 😁
David - 😁

Sofia:

Eva - 😁
Manuel - 😁
Tomás - 😁
David - 😁

Tomás:

Eva - 😁
Manuel - 😁
Sofia - 😁
David - 😁

Eva :

David- 😁
Manuel - 😁
Sofia - 😁
Tomás - 😁

Sprint 3