1MEIC02T01: CAVE (Code Architecture Visualization Engine) - FEUP-MEIC-DS-2024-25/ai4sd GitHub Wiki

The goal of the product is to generate architectural diagrams from descriptions of architectural patterns collected in software repositories, enabling developers to easily visualize, understand, and refine software architectures based on documented patterns and best practices.

The product vision is to create a tool that empowers software development teams to automatically generate architectural diagrams from GitHub repositories. This tool will significantly streamline the process of understanding, designing, and maintaining complex software systems by reducing manual effort in creating and interpreting architectural diagrams.

The following are examples of projects that generate diagrams, analyzed in terms of their strengths and weaknesses compared to the envisioned product:

• PlantUML

  • Pros: Open-source, widely used, integrates with many tools like GitLab, and offers text-based diagram creation.
  • Cons: Requires manual updates, which can be tedious for complex projects.

• Structurizr

  • Pros: Automatically generates C4 diagrams, offers a SaaS platform, integrates with codebases.
  • Cons: Requires specific documentation structure to work, limiting flexibility.

• Mermaid

  • Pros: Integrates well with markdown and documentation platforms like GitHub, automating visual documentation to an extent.

  • Cons: Still requires structured, domain-specific syntax. Lacks deep integration for software repository analysis and natural language processing.

• Lucidchart

  • Pros: Highly flexible for creating diagrams and offers a user-friendly interface.
  • Cons: Not directly integrated with codebases, requires manual input.

• Architexa

  • Pros: Provides seamless IDE integration for real-time architecture visualization. Includes features for interactive exploration and easy sharing, which improves collaboration among development teams.

  • Cons: Requires manual configuration for creating initial diagrams, limiting automation. Primarily designed for specific IDEs, which restricts flexibility for teams using other development environments or tools.

Compared to these projects, our product differentiates by focusing on converting unstructured natural language descriptions from repository documentation into accurate diagrams, saving time and reducing manual overhead for developers, and such tools, according to our research, aren't available.

The main downside compared to some existing tools is that it may take longer to gather data to refine the model in order to recognize a wide range of architectural patterns.

1. User Repository Selection: The user selects a GitHub repository for which they want to see an architectural diagram.

2. GenAI Processing – Repository Analysis and Diagram Generation: GenAI processes the repository information along with architectural pattern documentation to understand its structure and identify relevant architectural concepts. Based on this analysis, GenAI generates UML code that represents the repository’s structure and the identified architectural patterns.

3. Final Output – Display of Architectural Diagram: The UML code generated by GenAI is then processed by a tool to create an architectural diagram specific to the selected GitHub repository. This diagram visually represents the repository’s structure and highlights pattern-based insights, helping users better understand its architecture.

This high-level overview illustrates the GenAI process, from initial repository analysis to the generation of UML code and the final display of the architectural diagram. The overall workflow and component interactions are represented in Figure 1.

Figure 1: Domain Model

Our tool uses a modular architecture built upon the Django web framework, adhering to the Model-View-Controller (MVC) architectural pattern, specifically Django's Model-Template-View (MTV). This architecture ensures a clear separation of concerns, improving maintainability, scalability, and facilitating collaborative development.

The system is composed of three primary layers:

• Frontend - handles all interactions with the user through a web-based interface, processes user requests and serves as entry point to the workflow.

• Backend - serves as the core of the application, manages the workflow, defines the data models used and calls the necessary external services.

• External Services - handles the provided context data storage, repository processing, and integration with AI services.

Figure 2: Component Diagram

In the future, it is anticipated that our tool will be integrated as a microservice into a larger multi-tool application. To facilitate this integration, potential changes to our solution might include:

• Separating the frontend from the backend logic to create a headless service, this way enabling integration with various clients.

• Containerization of the application, aiming to assure consistency across different deployment environments.

• Standardizing the data formats of our API calls.

These future advancements might include new risks, like performance and scalability issues, as the tool might encounter performance bottlenecks when interacting with other services, as well as possible data breaches when communicating with other tools.

The development of this tool involved several key technologies, with some general restrictions given by the client and others selected by our team to best meet project goals. The client’s main restrictions included that the tool should be a web application or a VS Code extension. After evaluating the options, we chose to build a web app, prioritizing accessibility and ease of use. We justified this choice by considering that a web app allows broader platform compatibility, easier deployment, and straightforward maintenance compared to a more integrated VS Code extension, which could limit usability outside of the VS Code environment. Additionally, as a team, we felt more comfortable working within the web application domain compared to developing a VS Code extension, which aligned better with our skills and experience.

From the user's perspective, a web app provides several advantages: it can be accessed from any device with an internet connection, regardless of operating system, and does not require installing or configuring additional tools like a VS Code extension. The web app also facilitates easier updates, ensuring that users always have access to the latest features and improvements without needing to manually update or configure anything.

1. Django Framework (Python):

   • Choice: Django was selected as the primary framework for the application’s backend. Django’s Model-Template-View (MTV) architecture closely aligns with the Model-View-Controller (MVC) pattern, ensuring the separation of concerns and making the application more scalable and maintainable.
   • Justification: Django’s strong built-in support for database management and routing simplifies development.

2. GenAI (GPT-4-Turbo):

   • Choice: For now, we are using GPT-4-Turbo to analyze repository structure and generate UML code, but we are still evaluating other options to determine the best model for our needs.
   • Justification: When comparing Gemini with GPT-4-Turbo, we observed a significant difference in results, with GPT-4 being clearly superior. GPT-4-Turbo is more capable of generating structured UML diagrams for a specific GitHub repository with greater accuracy, outperforming Gemini in this regard. Using a large language model like GPT-4-Turbo allows us to automate a typically manual process, generating accurate diagrams faster and with minimal human input.

3. PlantUML:

   • Choice: PlantUML was selected for rendering UML diagrams from generated UML code.
   • Justification: As an open-source and widely used tool, PlantUML provides straightforward text-based diagram creation that integrates seamlessly into our workflow, enabling fast and dynamic diagram generation.

4. Frontend Technologies:

   • Choice: The frontend technology is currently undefined, as we plan to coordinate with other project groups using the web app to agree on a unified frontend technology. This collaboration will ensure consistency and facilitate easier integration across projects.
   • Justification: Selecting a shared frontend framework across teams will improve compatibility and collaboration, making the overall development process smoother as we integrate different parts of the application.

5. Database (SQLite):

   • Choice: For now, SQLite is used as the database to store user data, project information, and generated UML code.
   • Justification: SQLite’s lightweight and file-based structure makes it ideal for quick prototyping and development. It allows us to manage data efficiently during the early stages while keeping the option open to transition to a more scalable database like PostgreSQL if needed for production.

6. ChromaDB:

   • Choice: ChromaDB was selected as the database solution for storing and retrieving vector embeddings related to repository and UML data.
   • Justification: We decided to use ChromaDB to avoid using credits on VertexAI and to have greater control over our data management. ChromaDB is optimized for handling vector search and similarity queries, which is essential for our AI-driven processes. By using ChromaDB, we can efficiently manage large amounts of high-dimensional data while maintaining full control over storage and retrieval. This choice also enables us to fine-tune the system for better performance, speed, and scalability, without depending on external cloud services.

7. GitHub:

   • Choice: The GitHub is used to get a repository (selected by the user) for being analyzed.
   • Justification: Direct access to GitHub repositories allows the application an easier way to gather the code that the diagram is being built of. This integration improves the user experience by not manually uploading the code.

In Sprint 0, our team created a prototype to validate the feasibility of automatically generating architectural diagrams from GitHub repositories. This prototype demonstrated the integration between the GenAI model (GPT-4-Turbo), PlantUML, and the Django framework, laying the groundwork for the full application. The prototype confirmed that our chosen architecture and technology stack could meet the project’s requirements, providing a base for expanding the functionality in subsequent sprints.

The base prototype can be seen in: Sprint 0

We also explored the use of a vectorial database (chromaDB) to give the LLM some content about architectural patterns (not implemented in the prototype). The initial plan for the project during the development of the prototype, consisted in the following architecture:

Figure 3: Prototype architecture

What went well?

Application Containerization: We completed this task during the sprint, setting up the application to make deployment and integration more efficient.

API Development: An API was developed as planned, ensuring compatibility with the Avengers system and similar future services.

What went poorly?

We had the dream to build a service (vectorial database) that would receive as a query an architectural pattern small description based on a repository code. It would return some context and a UML diagram code template to give better context to the GenAi tool (Ultron) and make Ultron build better diagrams without many errors. Still, we failed to gather that kind of data/documentation during this sprint, so it was postponed to Sprint 2. The main challenge was finding sources with the necessary data and documentation during this sprint.

What can we do to improve?

Our communication and organization improved, with clearer task distribution helping everyone stay aligned and avoid overlapping or missed responsibilities.

What ideas do we have?

For the postponed task, the team will prioritize gathering the necessary data and documentation early in Sprint 2 to move forward with developing the vectorial database service. To improve our workflow, we plan to hold check-ins every three days to review progress and quickly address any challenges that arise.

End of sprint 1

By the end of Sprint 1, our Scrum Board reflected the following status, as shown in Figure 4:

Figure 4: Scrum board at the end of sprint 1

• Completed: Application containerization and API development

• Postponed: Development of the vectorial database service

The completed tasks demonstrate the team's focus and ability to deliver on key objectives. The postponed task highlights an area requiring further preparation and prioritization in Sprint 2. This clear visualization of our progress through the Scrum Board helped us evaluate performance effectively and set actionable goals for the next sprint.

In Figure 5, there is a board for tracking questions at the end of Sprint 1.

Figure 5: Questions Tracker

At the beginning of Sprint 2, as shown in Figure 6, our board was updated to reflect the new priorities and tasks for the sprint.

Figure 6: Scrum Board at the beginning of Sprint 2

What went well?

As communication between team members increased, we managed to complete more user stories than before, which made it possible to deploy the current version of the superhero backend as well as the respective user interface into the Avengers WebApp.

Some problems appeared during the deployment process and nevertheless, we found a way to bypass or even found a solution to it, which made the work more agile.

What went poorly?

Even though the communication between the team members was better, there was still misinterpretation about a user story which led to the postponement of it once again.

Also, there was a lack of invested time which made the work process occur next to the sprint deadline, and this contributed to the work not being well distributed and saturated some members.

We failed to communicate with other teams as much as was needed, this didn't lead to many consequences for the superhero itself, but maybe it limited the development of the whole project since we didn't contribute much to it.

Some technical issues arose, like storing and accessing secrets which made the superhero much limited in its functionalities.

What can we do to improve?

To improve as a team, we can focus on the following:

• Clearer Sprint Goals: Ensure every team member understands the sprint goals to stay aligned.

• Improve Communication: Have daily check-ins to ensure everyone knows the progress and challenges, and if everyone is dedicating the right amount of time to the tasks.

• Regular Feedback: Conduct short retrospectives at the end of each week to know if every team member is satisfied with the distribution of work.

What ideas do we have?

We need to continue integrating the assistant with the project and meet with the Avengers to handle general frontend bugs.

By the end of Sprint 2, our Scrum Board reflected the following status, as shown in Figure 7:

Figure 7: Scrum Board at the end of Sprint 2

• Clone the project using git clone.

• Run pip install -r requirements.txt to install all required libraries.

• First ensure Django Framework is installed;
• Start the server using python manage.py runserver
• Access the application at http://127.0.0.1:8000/

make build
make run

• Risk: Exposure of credentials or database information.
• Mitigation: Use environment variables and .env files to secure sensitive data.

• The tests folder includes scripts to validate core functionality, such as chromadb database interactions.

• Visit http://127.0.0.1:8000/create/ to create a diagram.
• Use /12/ to view an existing project with ID 12 (example).
• Diagram editing (/12/edit/).

• Access /admin using the credentials admin/admin (not recommended for production).

• Run python prompt.py to execute an example prompt. The system uses ChromaDB to enrich responses with contextual data.

• Clone the repository, install dependencies, and configure your environment as described above.

• Provide clear and concise descriptions in your PR, explaining the changes you’ve made.

• Team 1

  • Eduardo Areias
  • Francisco Ribeiro (PO)
  • José Costa
  • Marisa Azevedo (SM)
  • Tiago Simões