NativeAOT and Trimming Guidelines - gunpal5/Google_GenerativeAI GitHub Wiki

This page outlines the guidelines for using the Google Generative AI SDK with NativeAOT and trimming. NativeAOT and trimming can significantly reduce the size and startup time of your application, but they require careful consideration of reflection and dynamic code generation.

The core features and code-generated function tools of the Google Generative AI SDK are designed to be compatible with NativeAOT and trimming without requiring manual changes. However, features that rely on JsonMode and reflection-based QuickTools necessitate explicit configuration of JsonSerializerContext.

When using JsonMode, you must create a JsonSerializerContext that includes all the types used in your JSON schema. This allows the NativeAOT compiler to generate the necessary serialization code at compile time, eliminating the need for runtime reflection.

using System.Text.Json.Serialization;
using Google.GenerativeAI;

// Example Data Class
public class MyData
{
    public string Name { get; set; }
    public int Age { get; set; }
}

[JsonSerializable(typeof(MyData))]
public partial class MyDataJsonContext : JsonSerializerContext { }

//Option 1 add to global type resolvers
DefaultSerializerOptions.CustomTypeResolvers.Add(MyDataJsonContext.Default);

//Option 2 add to GenerativeModel class instance
generativeModel.GenerateObjectJsonSerializerOptions = MyDataJsonContext.Default.Options

//Option 3 pass it as argument
GenerateContentRequest request = new GenerateContentRequest()
request.UseJsonMode<MyData>(MyDataJsonContext.Default.Options)

For QuickTools that rely on reflection, you must also provide a JsonSerializerContext that includes all class type arguments and return types used in the tools.

public class ToolInput
{
    public string InputText { get; set; }
}

public class ToolOutput
{
    public int Result { get; set; }
}

public ToolOutput MyToolMethod(ToolInput input)
{
   return new ToolOutput { Result = input.InputText.Length };
}

[JsonSerializable(typeof(ToolInput))]
[JsonSerializable(typeof(ToolOutput))]
public partial class MyToolJsonContext : JsonSerializerContext { }

//Option 1 add to global type resolvers
DefaultSerializerOptions.CustomTypeResolvers.Add(MyDataJsonContext.Default);

//Option 2 pass it into the constructor
var quickTool = new QuickTool(MyToolMethod,options:MyToolJsonContext.Default.Options)

You can pass the JsonSerializerContext in three ways:

1. Global Configuration: Add the JsonSerializerContext to DefaultSerializerOptions.CustomJsonTypeResolvers during application initialization. This makes the context available throughout the application.

   using System.Text.Json;
   using Google.GenerativeAI;
   
   public static void ConfigureJsonContext()
   {
       DefaultSerializerOptions.CustomJsonTypeResolvers.Add(MyDataJsonContext.Default);
   }

2. GenerativeModel Scope: Set the GenerateObjectJsonSerializerOptions property of a GenerativeAI instance. This makes the context available within the scope of the GenerativeModel class.

   using System.Text.Json;
   using Google.GenerativeAI;
   
   public async Task UseModelScopedContext(string apiKey, string modelName, string prompt)
   {
       var googleAI = new GenerativeAI(apiKey);
       
       var model = googleAI.GenerativeModel(modelName);
       model.GenerateObjectJsonSerializerOptions = MyDataJsonContext.Default.Options;
      
       //Use JsonMode specific overloads
       var obj = await model.GenerateObjectAsync<T>(...);
   }

3. Request/Response Level: Pass the JsonSerializerOptions to GenerateContentRequest.UseJsonMode<T>(options) and GenerateContentResponse.ToObject<T>(options).

   using Google.GenerativeAI;
   
   public async Task UseRequestSpecificContext(GenerativeModel model, string prompt)
   {
       var request = new GenerateContentRequest();
       request.AddText(prompt);
       request.UseJsonMode<MyData>(MyDataJsonContext.Default.Options);
   
       var response = await model.GenerateContentAsync(request);
       var myData = response.ToObject<MyData>(MyDataJsonContext.Default.Options);
   
       Console.WriteLine($"Name: {myData.Name}, Age: {myData.Age}");
   }

• Complete Type Registration: Ensure that all types used in JsonMode and QuickTools are registered in the JsonSerializerContext. Failure to do so will result in runtime errors.
• Performance: Pre-generating serialization code using JsonSerializerContext improves performance and reduces startup time in NativeAOT environments.
• Trimming: When trimming, the compiler will remove unused code. Providing a JsonSerializerContext ensures that necessary serialization code is preserved.
• Reflection Limitations: NativeAOT and trimming significantly restrict the use of runtime reflection. Rely on code generation and explicit type registration to avoid issues.