• Installation
• Configuration
• Implementation
• Usage
• Troubleshooting

• Docker:

  • Minimum version: Docker Desktop 4.0 or higher.
  • Ensure Docker is running with virtualization enabled.
  • Verify Docker installation:

    docker --version

    Figure 1: Docker Version Check for Windows
    

• Windows Subsystem for Linux (WSL):

  • Minimum version: WSL 2 is required. Ensure WSL 2 is enabled and configured by following this guide.
  • Verify WSL version:

    wsl --list --verbose

• Homebrew:

  • Minimum version: Homebrew 3.0 or higher.
  • Ensure Homebrew is installed by running:

    brew --version

    Figure 2: Homebrew Version Check
    

• Miniconda:

  • Ensure you're using Miniconda 4.11.0 or higher for Python 3.11 compatibility.
  • Verify the Miniconda version with:

    conda --version

RUN mamba create -n team2_env python=3.10 -y
SHELL ["mamba", "run", "-n", "team2_env", "/bin/bash", "-c"]

Figure 3: Create Mamba Environment

In your Dockerfile, add:

RUN /bin/bash -c "source ~/.bashrc && mamba install mistral -c conda-forge"

Ensure your requirements.txt file includes:

transformers>=4.0

Note: If using GPUs, ensure Docker has GPU support configured for performance optimization.

Install additional dependencies for integrating Mistral AI:

pip install langchain-mistralai

Securely store and load the API key from an .env file:

MISTRAL_API_KEY=
HF_TOKEN=

Figure 4: API Key Configuration

For generating responses, initialize the Mistral model:

def invoke_llm_for_response(query):
    """Generate a response with highlighted keywords and exclude sources if no information is provided."""
    llm = ChatMistralAI(model='open-mistral-7b', api_key=os.getenv("API_KEY"))

Figure 5: Mistral Model Initialization

1. Model Temperature and Max Tokens: Adjust the temperature and max_tokens parameters to control the randomness and response length of the chatbot.

model = ChatMistralAI(model='open-mistral-7b', temperature=0.7, max_tokens=150)

Explanation:

• Temperature: Controls randomness (0.0 for deterministic, 1.0 for more randomness).
• Max Tokens: Limits the response length to avoid excessively long outputs.

• Temperature: Controls the level of randomness in the response. A value of 0 means deterministic responses (the same response for the same input), while a value closer to 1 introduces more variation in the model’s answers.
• Max Tokens: Limits the length of the response. This can be adjusted to ensure responses are concise or more elaborate, depending on the use case.

Example:

model = ChatMistralAI(model='open-mistral-7b', temperature=0.7, max_tokens=150)

In practice:

• Use low temperature for factual responses (e.g., technical explanations).
• Use high temperature for creative tasks (e.g., generating stories or ideas).

If no context is found, pass a generic fallback context to the LLM:

if not selected_chunks:
        fallback_context = (
            "The context does not provide specific information relevant to the query. "
            "Generate a general response to help the user."
        )
        response = rag_chain.invoke({"context": fallback_context, "question": query})
        return format_response_with_highlights(response, [], [])

Figure 6: Fallback Context Implementation

Ensure all dependencies are installed and the .env file is properly configured with the API key. The following steps should be completed before running the chatbot service:

1. Verify that the required Python libraries are installed:

   pip install -r requirements.txt

2. Check that the environment variables are correctly set up in the .env file:

   MISTRAL_API_KEY=your_api_key_here
   HF_TOKEN=your_huggingface_token_here
   

To start the chatbot service, execute the following command in your terminal:

python start_chatbot_service.py

This will initialize the chatbot and begin listening for queries.

Once the chatbot service is running, users can query the chatbot through the command line, an API endpoint, or a user interface (if integrated). Below are different examples of queries and how the chatbot should respond.

• Query 1:
  User: "What are the core courses in the CSE major?"
  Chatbot Response:
  "The core courses include 'Intro to Programming,' 'Data Structures and Algorithms,' 'Discrete Mathematics,' and 'Computer Networks.'"

• Query 2:
  User: "What is artificial intelligence?"
  Chatbot Response:
  "Artificial Intelligence (AI) is a branch of computer science that focuses on creating machines or software that can perform tasks that typically require human intelligence, such as decision-making, problem-solving, and learning."

• Query 3:
  User: "What is quantum computing?"
  Chatbot Response:
  "I don't have information on quantum computing at the moment. Please refer to the Physics Department for more details."

• Query 4:
  User: "Can you tell me a joke?"
  Chatbot Response:
  "Sure! Why don't scientists trust atoms? Because they make up everything!"

• Query 5:
  User: "Tell me more about Data Structures."
  Chatbot Response:
  "Data structures are ways to store and organize data to perform operations efficiently. Common types include arrays, linked lists, stacks, queues, and trees."

• Query 6:
  User: "What are the prerequisites for Machine Learning?"
  Chatbot Response:
  "To get started with Machine Learning, it's helpful to have a foundation in linear algebra, calculus, statistics, and programming (typically in Python)."

• Query 7:
  User: "Can you explain the difference between supervised and unsupervised learning?"
  Chatbot Response:
  "Supervised learning involves training a model on labeled data, meaning the input data has corresponding output labels. Unsupervised learning, on the other hand, works with unlabeled data, where the model tries to find patterns or structures in the data without any predefined labels."

• Query 8:
  User: "What is the salary of a senior software engineer?"
  Chatbot Response:
  "Sorry, I don't have the exact salary data. You may want to consult salary websites like Glassdoor or Payscale for current figures."

• Query 9:
  **

User:** "Can you explain why the sky is blue?"
Chatbot Response:
"The sky appears blue due to Rayleigh scattering. Shorter wavelengths of light, like blue, scatter more than longer wavelengths, causing the blue light to be more visible."

• Query 10:
  User: "How to deploy an application on AWS?"
  Chatbot Response:
  "Unfortunately, I don't have detailed information on deploying applications on AWS at the moment. You can check out the official AWS documentation for step-by-step guides."

The chatbot supports multiple forms of user input, from simple queries to more complex interactions. If the chatbot cannot find relevant context or data, it will fallback to a generic response or direct the user to external resources.

For example:

if not selected_chunks:
    fallback_context = (
        "The context does not provide specific information relevant to the query. "
        "Generating a general response."
    )
    response = rag_chain.invoke({"context": fallback_context, "question": query})
    return format_response_with_highlights(response, [], [])

1. Docker Container Fails to Start

   • Issue: The Docker container may fail to start if certain system resources (like CPU or memory) are not sufficient, or if the Docker image is corrupted.
   • Solution:
     • Check System Resources: Ensure there’s enough available CPU and memory for the container. You can monitor system resources with:

       docker stats

     • Rebuild Docker Image: If the image is corrupted or outdated, rebuild it:

       docker-compose build --no-cache

     • Docker Version Compatibility: Sometimes, an outdated version of Docker may cause issues. Ensure you are using the latest stable version of Docker. You can check the installed version using:

       docker --version

     • Docker Network Issues: Ensure the container has proper network access. Sometimes, containers can fail to start if they can't connect to necessary networks or external APIs. To troubleshoot, restart Docker or try a different network mode:

       docker network ls

2. API Key Issues

   • Issue: The application fails to load or authenticate using the API key.
   • Solution:
     • Verify Environment Variables: Ensure the API key is properly set in the .env file and that it matches the expected format. For example:

       MISTRAL_API_KEY=your_api_key_here
       HF_TOKEN=your_huggingface_token_here

     • Check API Quotas: If you encounter an authentication error, make sure that the API key hasn’t exceeded the usage quota (e.g., for Hugging Face or Mistral).
     • Use the Correct Token Format: Ensure that the tokens or API keys are being passed correctly in the request headers. Double-check that you're using the correct syntax in the code:

       headers = {"Authorization": f"Bearer {API_KEY}"}

3. Missing Dependencies

   • Issue: The project fails to run due to missing Python or system dependencies.
   • Solution:
     • Reinstall Dependencies: Double-check that the requirements.txt file is up to date. You can reinstall the dependencies using:

       pip install -r requirements.txt

     • Check for System-Specific Dependencies: Sometimes, additional libraries are needed for specific systems (e.g., libcurl on Linux or Visual C++ Build Tools on Windows). Install any necessary system packages.
     • Dependency Conflicts: If there are version conflicts between libraries (e.g., incompatible versions of transformers and langchain), try upgrading or downgrading specific libraries:

       pip install --upgrade transformers
       pip install --upgrade langchain-mistralai

4. API Key Not Loading

If the application cannot access the API key, follow these steps:

• Verify .env File: Ensure the .env file is present in the project’s root directory and contains the correct API keys:

  MISTRAL_API_KEY=your_api_key
  HF_TOKEN=your_huggingface_token

• Ensure Environment Variables are Loaded: In Python, use dotenv to load environment variables before calling them in the code:

  from dotenv import load_dotenv
  load_dotenv()  # Load environment variables from .env file

• Double-check Path and File Location: Ensure that the .env file is in the correct directory and is readable by the script. If your project uses Docker, make sure the .env file is properly mapped into the container during the build process.

5. Model Response Delay or No Response

If you experience delays or no response from the model:

• Check System Resources: Ensure that your system has enough resources (CPU, memory, disk space) to run the model. Check Docker stats:

  docker stats

• Network Latency: If the model is cloud-based (e.g., Hugging Face), check for network issues that might delay API calls. You can use ping to check connectivity:

  ping api.huggingface.co

• Timeouts: If the model takes too long to respond, consider increasing the timeout value in your API requests or adjusting the model's configuration (e.g., temperature, max tokens) for more efficient responses.

6. Missing Configuration or Incorrect Setup

If the configuration is incorrect or missing, it may prevent the project from running smoothly.

• Verify Configuration Files: Double-check all configuration files (e.g., docker-compose.yml, .env) and ensure they match the project’s requirements.

  • For Docker, ensure the environment variables are passed correctly in the container.

• Model Initialization Issues: If you encounter issues when initializing the Mistral model, ensure that you’re using the correct API key and model identifier. Also, ensure the required dependencies (e.g., langchain-mistralai) are installed:

  pip install langchain-mistralai

• Environment Variables: Store API keys in environment variables and load them using dotenv.
• Encryption: Consider encrypting sensitive information in the .env file or using a vault system for production environments.
• HTTPS: Always use HTTPS to securely transmit API keys.

1. User Namespaces: Enable user namespaces to ensure Docker containers run with minimal privileges.
2. Limit Permissions: Use the least privilege principle by restricting containers to only the necessary resources.
3. Regular Updates: Regularly update Docker images to include the latest security patches. For example:

   docker pull <image_name>:latest

1. Forking the Repository

   • Step 1: Fork the repository to your GitHub account. This creates a personal copy of the project.
   • Step 2: Clone the forked repository to your local machine:

     git clone https://github.com/<your_username>/<repository_name>.git

   • Step 3: Navigate to the project directory:

     cd <repository_name>

2. Creating a New Branch

   • Always create a new branch for your work. This helps isolate your changes and keeps the main branch clean.
   • Create and switch to a new branch:

     git checkout -b feature/your-feature-name

3. Committing Changes

   • After making changes, ensure to add and commit them with meaningful messages:

     git add .
     git commit -m "Add feature: detailed troubleshooting for Docker issues"

4. Pushing Changes

   • Push your changes to your forked repository:

     git push origin feature/your-feature-name

5. Creating a Pull Request

   • After pushing changes, go to the repository on GitHub and open a pull request (PR) from your branch to the main repository’s main branch.
   • Provide a detailed description of the changes made and why they are necessary.

6. Review Process

   • Once the PR is submitted, the project manager or designated reviewer will check your changes.
   • If changes are requested, address them by modifying your code and pushing the updates to the branch. The PR will automatically update with the new changes.

7. Merging Pull Requests

   • After approval, the project manager will merge your changes into the main branch. If there are merge conflicts, resolve them by manually editing the conflicting files, committing, and pushing the changes.

8. Versioning

   • For version control, follow semantic versioning (MAJOR.MINOR.PATCH) to indicate changes:
     • MAJOR: Breaking changes that require changes to existing code.
     • MINOR: New features or enhancements that are backward-compatible.
     • PATCH: Bug fixes or small improvements.