Milvus Documentation

Last edited by csusb_fall2024_cse6550_team4

---

Table of Contents

1. Installation
2. Configuration
3. Implementation
4. Usage
5. Troubleshooting

---

1. Installation

This section outlines the steps to set up Milvus and the project environment using Docker.

---

Step 1: Build and Deploy the Milvus Environment

1. Navigate to the Project Repository:
   Clone the repository or navigate to the project folder:

   git clone https://github.com/DrAlzahraniProjects/csusb_fall2024_cse6550_team4.git  
   cd csusb_fall2024_cse6550_team4  
   

2. Build the Docker Image: Use the provided Dockerfile to build the project environment:

   docker build -t team4_chatbot . 
   

   

   Figure shows the milvus installation command in the docker file.

3. Run the Docker Container: Start the container with the following command:

   docker run -d -p 19530:19530 -p 5004:5004 -p 6004:6004 --name chatbot team4_chatbot
   

4. Verify Milvus Deployment:

   Ensure Milvus is active by executing the following command inside the container:

   docker exec -it chatbot bash  
   python -c "from pymilvus import connections; connections.connect(); print('Milvus connected')"  
   

Figure shows how the statement imports the Milvus integration module from the langchain-milvus package.

2. Configuration

Connecting to Milvus

To establish communication with Milvus, we use the pymilvus library in our project. The connection logic is defined in the bot.py file within the initialize_milvus function.

Here’s the relevant code snippet for connecting to Milvus:

from pymilvus import connections

def initialize_milvus(uri: str = MILVUS_URI):
    """
    Initialize the vector store for the RAG model

    Args:
        uri (str, optional): Path to the local Milvus db. Defaults to MILVUS_URI.

    Returns:
        vector_store: The vector store created
    """
    embeddings = get_embedding_function()
    vector_store = None

    # Connect to Milvus
    connections.connect("default", uri=uri)

    # Your collection setup and data handling logic goes here...
    print("Vector store initialization complete.")
    return vector_store

Key Steps:

Establishing the Connection: The connections.connect method connects to the Milvus server using the URI defined in the environment (MILVUS_URI).

Embedding Initialization: The function calls get_embedding_function() to configure embeddings for vector storage.

Vector Store Handling: The function sets up or retrieves the vector store used for managing document embeddings.

Confirmation: A log message confirms successful initialization of the Milvus vector store.

Figure shows how Milvus is initialized and connection is established.

Connection of milvus in jupyter notebook

3. Implementation

1. Creating the Collection and Embedding Setup:

The initialize_milvus function initializes the vector store for the chatbot, which includes the collection setup and document insertion. It connects to the Milvus database, checks if the collection exists, and either loads it or creates a new one.

def create_vector_store(docs, embeddings, uri):
    """
    Initializes or loads a vector store from Milvus, adding documents for similarity search.

    Args:
        docs (list): A list of documents to be stored in the vector store.
        embeddings : A function or model that generates embeddings for the documents.
        uri (str): Path to the local Milvus db.

    Returns:
        vector_store: The vector store created or loaded.
    """
    # Connect to the Milvus database
    connections.connect("default", uri=uri)

    # Check if the collection already exists
    if utility.has_collection("research_paper_chatbot"):
        print("Collection already exists. Loading existing Vector Store.")
        vector_store = Milvus(
            collection_name="research_paper_chatbot",
            embedding_function=get_embedding_function(),
            connection_args={"uri": uri}
        )
    else:
        # Create a new vector store and drop any existing one
        vector_store = Milvus.from_documents(
            documents=docs,
            embedding=embeddings,
            collection_name="research_paper_chatbot",
            connection_args={"uri": uri},
            drop_old=True,
        )
        print("Vector Store Created")
    return vector_store

Functionality:

Vector Store Creation: If the collection (research_paper_chatbot) already exists, it loads the existing vector store. If not, it creates a new collection and stores the documents passed to it.

Collection Initialization: The vector store is created by the Milvus.from_documents() method, which processes documents and stores them using the embeddings function.

Figure shows the creation of vector_store using Milvus.

2. Loading Existing Vector Store:

If the vector store is already initialized, the load_exisiting_db function can be used to load the store from Milvus.

def load_exisiting_db(uri=MILVUS_URI):
    """
    Load an existing vector store from the local Milvus database specified by the URI.

    Args:
        uri (str, optional): Path to the local Milvus db. Defaults to MILVUS_URI.

    Returns:
        vector_store: The vector store created.
    """
    vector_store = Milvus(
        collection_name="research_paper_chatbot",
        embedding_function=get_embedding_function(),
        connection_args={"uri": uri},
    )
    print("Vector Store Loaded")
    return vector_store

Figure shows loading of existing vector_store.

3. Document Processing and Insertion:

Documents are split into smaller chunks using split_documents() and then inserted into the vector store.

def split_documents(documents):
    """
    Split the documents into chunks for processing.

    Args:
        documents (list): List of documents to split.

    Returns:
        list: A list of chunks of documents.
    """
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=2000,  # Split the text into chunks of 1000 characters
        chunk_overlap=200,  # Overlap the chunks by 300 characters
        is_separator_regex=False,  # Don't split on regex
    )
    docs = text_splitter.split_documents(documents)
    return docs

This function ensures that documents are processed in manageable chunks and then stored in Milvus for efficient retrieval.

Milvus folder in our repository

4. Usage

Inserting Data into Milvus

Inserting data into the Milvus vector store is done after generating document embeddings. Below is the code snippet used in the project for inserting data into the vector store:

def create_vector_store(docs, embeddings, uri):
    """
    Initializes or loads a vector store in Milvus from documents and embeddings.

    Args:
        docs (list): The documents to be inserted into the vector store.
        embeddings: The model used to generate embeddings for the documents.
        uri (str): The URI of the local Milvus database.

    Returns:
        vector_store: The created or loaded vector store.
    """
    # Ensure the directory exists for the Milvus database
    head = os.path.split(uri)
    os.makedirs(head[0], exist_ok=True)

    # Connect to Milvus
    connections.connect("default", uri=uri)

    # Check if the collection exists; if it does, load it, else create a new one
    if utility.has_collection("research_paper_chatbot"):
        print("Collection already exists. Loading existing Vector Store.")
        vector_store = Milvus(
            collection_name="research_paper_chatbot",
            embedding_function=get_embedding_function(),
            connection_args={"uri": uri}
        )
    else:
        # Create a new vector store and insert documents
        vector_store = Milvus.from_documents(
            documents=docs,
            embedding=embeddings,
            collection_name="research_paper_chatbot",
            connection_args={"uri": uri},
            drop_old=True,  # Drop old collection if it exists
        )
        print("Vector Store Created")
    return vector_store

What This Code Does: It connects to the Milvus instance and checks if the collection (research_paper_chatbot) exists. If not, it creates a new collection and inserts the provided documents (docs), which are processed into embeddings. Important: Make sure your documents are converted into embeddings before passing them to this function.

Used milvus in docker file to have milvus ready and build inside the docker image

Querying Data from Milvus

Once data is inserted into the collection, you can query Milvus for relevant documents based on the user's input. Below is the code for querying data from the Milvus collection:

def query_rag(query):
    """
    Entry point for the RAG model to generate an answer to a given query.

    Args:
        query (str): The query string for which an answer is to be generated.
    
    Returns:
        str: The formatted answer with a unique source link (if available).
    """
    # Define the model
    model = ChatMistralAI(model='open-mistral-7b', api_key=MISTRAL_API_KEY, temperature=0.2)
    print("Model Loaded")

    prompt = create_prompt()

    # Load the vector store and create the retriever
    vector_store = load_exisiting_db(uri=MILVUS_URI)
    retriever = ScoreThresholdRetriever(vector_store=vector_store, score_threshold=0.2, k=3)  # Adjust k as needed
    
    try:
        # Set up document and retrieval chains
        document_chain = create_stuff_documents_chain(model, prompt)
        print("Document Chain Created")

        retrieval_chain = create_retrieval_chain(retriever, document_chain)
        print("Retrieval Chain Created")

        # Get relevant documents
        relevant_docs = retriever.get_relevant_documents(query)
        print(f"Relevant Documents: {relevant_docs}")
        
        # Generate response
        response = retrieval_chain.invoke({"input": query})
        response_text = response.get("answer", "No answer found.")

        # Collect unique links
        unique_links = set()
        for doc in relevant_docs:
            metadata = doc.metadata if hasattr(doc, "metadata") else {}
            source = metadata.get("source", "Unknown").split("/")[-1]
            page = metadata.get("page", "Unknown")

            # Ensure page is an integer
            try:
                page = int(page)
            except ValueError:
                page = 1  # Default to page 1 if invalid

            # Create a unique link
            if source != "Unknown":
                link = f'<a href="/team4/?view=pdf&file={data_dir}/{source}&page={page}" target="_blank" style="color : white">[more_info]</a>'
                unique_links.add(link)  # Adds only if link is unique

        # Append source links to response text
        if unique_links:
            response_text += f"\n\nSource: {''.join(unique_links)}"

        return response_text

    except HTTPStatusError as e:
        print(f"HTTPStatusError: {e}")
        if e.response.status_code == 429:
            return "I am currently experiencing high traffic. Please try again later."
        return f"HTTPStatusError: {e}"

What This Code Does: The function first retrieves relevant documents from Milvus based on the query input. The ScoreThresholdRetriever is used to filter out documents that don’t meet the relevance score. After retrieving the documents, the chatbot generates a response using the ChatMistralAI model and appends unique source links (e.g., PDF document references) to the response.

5. Troubleshooting

Faced problem while connecting to milvus vector database

Unable to load the milvus database locally

Milvus search failed raise error:"syncTimestamp Failed"

Connection Issues

If you encounter connection errors, confirm that the Milvus server is running and accessible at the specified host and port. You can verify this with the following command:

docker ps

This command checks if the Milvus container is active. Ensure that the container’s status and port mappings are correct.

Milvus Error When Using Collection Search: Connection Refused and Collection Not Loaded

Import Errors

If you experience import errors with pymilvus, make sure it is correctly installed:

pip install pymilvus

This command reinstalls the necessary library to resolve import issues.

Debugging Tips

• Check Collection Information: Use print statements to review the collection schema and configuration, ensuring the setup is accurate.

print("Collection info:", collection.schema)

• Verify Insertions: After adding data to the collection, confirm the entity count to ensure that all data was inserted correctly.

print("Total entities:", collection.num_entities)

• Reinstall Libraries: If issues persist, reinstalling pymilvus and updating dependencies can help resolve compatibility problems and ensure stable operation.

pip install --upgrade pymilvus