Streaming - gunpal5/Google_GenerativeAI GitHub Wiki

This page explains how to stream content generation results using the Google Generative AI C# SDK. Streaming allows you to receive results incrementally as they are generated, which can be useful for displaying content in real-time or handling large responses. This page will detail the available StreamContentAsync methods and provide examples for each.

The SDK provides several overloaded StreamContentAsync methods for streaming content generation. These methods are asynchronous and return an IAsyncEnumerable<GenerateContentResponse>, which you can use to iterate over the streamed responses.

This is the most general method for streaming. You create a GenerateContentRequest (as described in the Generate Content page) and pass it to this method.

var request = new GenerateContentRequest();
request.AddText("Write a story about a robot learning to love.");

await foreach (var response in client.StreamContentAsync(request))
{
    Console.Write(response.Text()); 
}

This method streams content generation based on a simple text prompt.

string prompt = "Generate a list of 10 creative project ideas.";

await foreach (var response in client.StreamContentAsync(prompt))
{
    Console.WriteLine(response.Text()); 
}

This method allows you to stream content generation with a file input.

string prompt = "Describe what is happening in this image.";
string fileUri = "path/to/image.jpg"; // Or a URL
string mimeType = "image/jpeg";

await foreach (var response in client.StreamContentAsync(prompt, fileUri, mimeType))
{
    Console.WriteLine(response.Text()); 
}

This method allows you to stream content generation with a sequence of Part objects.

var parts = new List<Part>
{
    new Part { Text = "Translate 'Hello, world!' to Spanish." }
};

await foreach (var response in client.StreamContentAsync(parts))
{
    Console.WriteLine(response.Text()); 
}

This method allows streaming with pre-constructed Content objects.

var contents = new List<Content>
{
    new Content { Parts = new List<Part> { new Part { Text = "Explain the concept of quantum entanglement." } } }
};

await foreach (var response in client.StreamContentAsync(contents))
{
    Console.WriteLine(response.Text()); 
}

1. Asynchronous Iteration: The await foreach construct is crucial for processing the streamed responses asynchronously. This prevents blocking the main thread while waiting for more data.
2. Cancellation: Use the cancellationToken parameter to gracefully stop the streaming process if needed.
3. Error Handling: Implement appropriate error handling within the await foreach loop to manage potential exceptions during streaming.
4. Partial Responses: Be aware that each GenerateContentResponse in the stream might contain a partial response. You'll likely need to accumulate the content from multiple responses to get the complete generated text. The .Text() method helps simplify this by concatenating the text parts for you.

• GenerateContentRequest:
• GenerateContentResponse:
• Part:
• Content:
• Roles:

This page provides a comprehensive guide to streaming content generation using the Google Generative AI C# SDK. By using the StreamContentAsync methods and understanding the asynchronous nature of streaming, you can effectively integrate real-time content generation into your applications. The use of .Text() simplifies the retrieval of the generated text content.