Chunk Client - gunpal5/Google_GenerativeAI GitHub Wiki

Introduction

The ChunkClient provides methods for interacting with the Gemini API's Chunks endpoint. This allows you to create, manage, and retrieve chunks within a document. Chunks are subdivisions of a document's content, enabling more granular control over semantic search and indexing. They are particularly useful for handling large documents by breaking them down into smaller, more manageable units.

Details

The ChunkClient offers the following functionalities:

Creating a Chunk

The CreateChunkAsync method creates a new chunk within a specified document.

using GenerativeAI.Clients;
using GenerativeAI.Types;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient

var parentDocument = "corpora/my-corpus-id/documents/my-document-id"; // Replace with the parent document name

var chunk = new Chunk
{
    Data = new ChunkData { StringValue = "This is the content of my chunk." },
    CustomMetadata = new List<CustomMetadata> { new CustomMetadata { Key = "key1", Value = "value1" } }, // Example metadata
    // ... other chunk properties ...
};

var createdChunk = await chunkClient.CreateChunkAsync(parentDocument, chunk);

if (createdChunk != null)
{
    Console.WriteLine($"Chunk created: {createdChunk.Name}");
}
else
{
    Console.WriteLine("Failed to create chunk.");
}

Listing Chunks

The ListChunksAsync method retrieves a list of chunks within a document.

using GenerativeAI.Clients;
using GenerativeAI.Types;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient

var parentDocument = "corpora/my-corpus-id/documents/my-document-id"; // Replace with the parent document name

var listChunksResponse = await chunkClient.ListChunksAsync(parentDocument); // You can provide pageSize and pageToken

if (listChunksResponse != null && listChunksResponse.Chunks != null)
{
    foreach (var chunk in listChunksResponse.Chunks)
    {
        Console.WriteLine($"Chunk Name: {chunk.Name}");
        Console.WriteLine($"Chunk Content: {chunk.Data?.StringValue}"); // Accessing content
    }
}
else
{
    Console.WriteLine("No chunks found.");
}

Getting a Chunk

The GetChunkAsync method retrieves a specific chunk by name.

using GenerativeAI.Clients;
using GenerativeAI.Types;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient

var chunkName = "corpora/my-corpus-id/documents/my-document-id/chunks/my-chunk-id"; // Replace with the chunk name

var chunk = await chunkClient.GetChunkAsync(chunkName);

if (chunk != null)
{
    Console.WriteLine($"Chunk Content: {chunk.Data?.StringValue}");
}
else
{
    Console.WriteLine("Chunk not found.");
}

Updating a Chunk

The UpdateChunkAsync method updates an existing chunk.

using GenerativeAI.Clients;
using GenerativeAI.Types;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient

var chunkName = "corpora/my-corpus-id/documents/my-document-id/chunks/my-chunk-id"; // Replace with the chunk name

var updatedChunk = new Chunk
{
    Name = chunkName, // Important: Include the name in the updated chunk object.
    Data = new ChunkData { StringValue = "This is the updated chunk content." },
    CustomMetadata = new List<CustomMetadata> { new CustomMetadata { Key = "key2", Value = "value2" } },
    // ... other properties to update ...
};

string updateMask = "data,customMetadata"; // Specify the fields to update

var resultChunk = await chunkClient.UpdateChunkAsync(updatedChunk, updateMask);

if (resultChunk != null)
{
    Console.WriteLine($"Chunk updated: {resultChunk.Name}");
}
else
{
    Console.WriteLine("Failed to update chunk.");
}

Deleting a Chunk

The DeleteChunkAsync method deletes a chunk.

using GenerativeAI.Clients;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient

var chunkName = "corpora/my-corpus-id/documents/my-document-id/chunks/my-chunk-id"; // Replace with the chunk name

await chunkClient.DeleteChunkAsync(chunkName);

Console.WriteLine($"Chunk deleted: {chunkName}");

Batch Creating Chunks

The BatchCreateChunksAsync method creates multiple chunks in a single request.

using GenerativeAI.Clients;
using GenerativeAI.Types;
using System.Collections.Generic;
using System.Threading.Tasks;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient
var parentDocument = "corpora/my-corpus-id/documents/my-document-id"; // Replace with the parent document name

var requests = new List<CreateChunkRequest>
{
    new CreateChunkRequest
    {
        Chunk = new Chunk
        {
            Data = new ChunkData { StringValue = "Chunk 1 content" }
        }
    },
    new CreateChunkRequest
    {
        Chunk = new Chunk
        {
            Data = new ChunkData { StringValue = "Chunk 2 content" }
        }
    },
    // ... more requests ...
};

var response = await chunkClient.BatchCreateChunksAsync(parentDocument, requests);

if (response != null && response.Chunks != null)
{
    foreach (var chunk in response.Chunks)
    {
        Console.WriteLine($"Created Chunk: {chunk.Name}");
    }
}
else
{
    Console.WriteLine("Failed to batch create chunks.");
}

Batch Updating Chunks

The BatchUpdateChunksAsync method updates multiple chunks in a single request.

using GenerativeAI.Clients;
using GenerativeAI.Types;
using System.Collections.Generic;
using System.Threading.Tasks;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient
var parentDocument = "corpora/my-corpus-id/documents/my-document-id"; // Replace with the parent document name

var requests = new List<UpdateChunkRequest>
{
    new UpdateChunkRequest
    {
        Chunk = new Chunk
        {
            Name = "corpora/my-corpus-id/documents/my-document-id/chunks/chunk1",
            Data = new ChunkData { StringValue = "Updated Chunk 1 content" }
        },
        UpdateMask = "data"
    },
    new UpdateChunkRequest
    {
        Chunk = new Chunk
        {
            Name = "corpora/my-corpus-id/documents/my-document-id/chunks/chunk2",
            Data = new ChunkData { StringValue = "Updated Chunk 2 content" }
        },
        UpdateMask = "data"
    },
    // ... more requests ...
};

var response = await chunkClient.BatchUpdateChunksAsync(parentDocument, requests);

if (response != null && response.Chunks != null)
{
    foreach (var chunk in response.Chunks)
    {
        Console.WriteLine($"Updated Chunk: {chunk.Name}");
    }
}
else
{
    Console.WriteLine("Failed to batch update chunks.");
}

Batch Deleting Chunks

The BatchDeleteChunksAsync method deletes multiple chunks in a single request.

using GenerativeAI.Clients;
using GenerativeAI.Types;
using System.Collections.Generic;
using System.Threading.Tasks;

// ... other code ...

var chunkClient = new ChunkClient(platform, httpClient, logger); // Initialize ChunkClient
var parentDocument = "corpora/my-corpus-id/documents/my-document-id"; // Replace with the parent document name

var requests = new List<DeleteChunkRequest>
{
    new DeleteChunkRequest
    {
        Name = "corpora/my-corpus-id/documents/my-document-id/chunks/chunk1"
    },
    new DeleteChunkRequest
    {
        Name = "corpora/my-corpus-id/documents/my-document-id/chunks/chunk2"
    },
    // ... more requests ...
};

await chunkClient.BatchDeleteChunksAsync(parentDocument, requests);
Console.WriteLine("Chunks batch deleted.");

Important Considerations

• Ensure proper authorization is configured before using the ChunkClient. See the Authentication page.
• Replace placeholder chunk names, IDs, and document names with actual values.
• Handle potential exceptions during API calls.
• Be mindful of rate limits when making frequent requests. See the official documentation for details.