Function Calling - gunpal5/Google_GenerativeAI GitHub Wiki

Function calling allows you to extend the capabilities of Gemini by integrating your own custom functions. This feature lets Gemini call your code to retrieve information or perform actions based on user requests. This page explains how to define and use functions with Gemini using the Google Generative AI SDK for C#. This functionality requires the GenerativeAI.Tools NuGet package.

Google_GenerativeSDK supports several ways to define Function Tools.

1. Reflection based QuickTool and QuickTools.
2. FunctionToolAttribute based Function Tools.
3. Interface Based Function Tools

Integrating functions with Gemini involves several steps: defining the function (via interface, reflection, or attribute), creating tools and calls, and adding the tool to your GenerativeModel.

Quickly define function tools from methods using reflection. This approach simplifies the process of creating tools for simple functions.

var func = (async ([Description("Request to query student record")] QueryStudentRecordRequest query) =>
{
    return new StudentRecord
    {
        StudentId = "12345",
        FullName = query.FullName,
        Level = GradeLevel.Freshman,
        EnrolledCourses = new List<string> { "Math", "Physics", "Chemistry" },
        Grades = new Dictionary<string, double>
        {
            { "Math", 95.0 },
            { "Physics", 89.0 },
            { "Chemistry", 88.0 }
        },
        EnrollmentDate = new DateTime(2023, 1, 10),
        IsActive = true
    };
});

var quickFt = new QuickTool(func, "GetStudentRecordAsync", "Return student record for the year");
//Use QuickTools([F1, F2, F3]) for multiple methods 

var model = new GenerativeModel(GetTestGooglePlatform(), GoogleAIModels.Gemini2Flash);

model.AddFunctionTool(quickFt);

var result = await model.GenerateContentAsync("How's John Joe is doing in Senior Grade? in enrollment year 01-01-2024 to 01-01-2025").ConfigureAwait(false);

Define functions directly from methods using the FunctionToolAttribute. The code generator handles the rest, streamlining the tool creation process.

[FunctionTool(GoogleFunctionTool= true, MeaiFunction=true)]
[System.ComponentModel.Description("Get book page content")]
public static Task<string> GetBookPageContentAsync(string bookName, int bookPageNumber, CancellationToken cancellationToken = default)
{
    return there is a nice weather outside, and I am stuck at home.");
}

var model = new GenerativeModel(GetTestGooglePlatform(), GoogleAIModels.Gemini2Flash);

var tools = new Tools([GetBookPageContentAsync]);
model.AddFunctionTool(tools);

For Microsoft.Extensions.AI:

var tools = new Tools([GetBookPageContentAsync]);
chatOptions.Tools = tools.AsMeaiTools();

Create an interface that defines the functions you want to expose to Gemini.
Use the GenerateJsonSchema attribute from the GenerativeAI.Tools package on your interface.
Add Description attributes to your functions and their parameters to provide Gemini with information about their purpose.

New Parameters in GenerateJsonSchema:

The GenerateJsonSchema attribute now includes two optional boolean parameters: GoogleFunctionTool and MeaiFunction. These parameters generate extension methods AsGoogleFunctionTool() and AsMeaiTools() respectively, providing flexibility for different AI platforms.

using GenerativeAI.Tools;
using System.ComponentModel; // For Description attribute

[GenerateJsonSchema(GoogleFunctionTool = true, MeaiFunction = true)]
public interface IWeatherFunctions
{
    [Description("Get the current weather in a given location")]
    Weather GetCurrentWeather(
        [Description("The city and state, e.g. San Francisco, CA")]
        string location,
        Unit unit = Unit.Celsius);

    [Description("Get the current weather in a given location asynchronously")]
    Task<Weather> GetCurrentWeatherAsync(
        [Description("The city and state, e.g. San Francisco, CA")]
        string location,
        Unit unit = Unit.Celsius,
        CancellationToken cancellationToken = default);
}

public enum Unit
{
    Celsius,
    Fahrenheit,
    Imperial
}

public class Weather
{
    public string Location { get; set; } = string.Empty;
    public double Temperature { get; set; }
    public Unit Unit { get; set; }
    public string Description { get; set; } = string.Empty;
}

This example defines an interface IWeatherFunctions with two methods: GetCurrentWeather and GetCurrentWeatherAsync. The Description attributes provide context for Gemini. The Unit enum and Weather class define the data structures used by the functions. The GenerateJsonSchema attribute with GoogleFunctionTool = true and MeaiFunction = true will now generate the respective extension methods.

Create a class that implements the defined interface. This class will contain the actual logic for your functions.

public class WeatherService : IWeatherFunctions
{
    public Weather GetCurrentWeather(string location, Unit unit = Unit.Celsius)
    {
        return new Weather
        {
            Location = location,
            Temperature = 30.0, // Replace with actual logic
            Unit = unit,
            Description = "Sunny", // Replace with actual logic
        };
    }

    public Task<Weather> GetCurrentWeatherAsync(string location, Unit unit = Unit.Celsius,
        CancellationToken cancellationToken = default)
    {
        return Task.FromResult(new Weather
        {
            Location = location,
            Temperature = 22.0, // Replace with actual logic
            Unit = unit,
            Description = "Sunny", // Replace with actual logic
        });
    }
}

This example implements the IWeatherFunctions interface. The methods provide placeholder implementations; you should replace these with your actual logic.

Use the AsTools and AsCalls extension methods (from the GenerativeAI.Tools package) on an instance of your implemented class to create the necessary tools and calls objects. Then create a GenericFunctionTool instance, passing the tools and calls.

Using the newly generated extension methods:

If you defined your interface with GoogleFunctionTool = true, you can use AsGoogleFunctionTool() directly. For M.E.A.I, use AsMeaiTools().

var service = new WeatherService();
var model = new GenerativeModel(apiKey: "YOUR_API_KEY");
var googleTool = service.AsGoogleFunctionTool(); // Generated if GoogleFunctionTool = true, Use `new GenericFunctionTool(service.AsTools(),service.AsCalls())` if GoogleFunction = false `(default)`
model.AddFunctionTool(googleTool);

//For Meai
var meaiTools = service.AsMeaiTools(); // Generated if MeaiFunction = true
chatOptions.Tools = meaiTools;

You can customize how Gemini interacts with your functions using the FunctionCallingBehaviour property of the GenerativeModel.

model.FunctionCallingBehaviour = new FunctionCallingBehaviour
{
    AutoCallFunction = false, // Gemini will suggest the function call but not execute it automatically.
    AutoReplyFunction = true,  // Gemini will automatically generate a response after the function call.
    AutoHandleBadFunctionCalls = false // Gemini will not attempt to handle errors from incorrect calls.
};

• FunctionEnabled: Enables or disables function calling.
• AutoCallFunction: If true, Gemini will automatically call the identified function. If false, Gemini will suggest the function call in the response, and you may choose to execute the function yourself.
• AutoReplyFunction: If true, Gemini will attempt to generate a response after the function call. If false, Gemini will return the result of the function call, and you may construct your own response.
• AutoHandleBadFunctionCalls: If true, Gemini will attempt to handle errors from incorrect function calls.

Make a request to Gemini that would trigger one of your functions.

var result = await model.GenerateContentAsync("What is the weather in San Francisco today?");
Console.WriteLine(result.Text);

Gemini will analyze the request and, based on the descriptions you provided, may choose to call your function.

Choosing the right type of function tool depends on your specific needs and constraints. Here's a comparison to help you decide:

1. Interface-based tools (using GenerateJsonSchemaAttribute) and Method-based tools (using FunctionToolAttribute):

• NativeAOT/Trimming Compatibility:
  • These tools are fully compatible with NativeAOT and trimming, making them ideal for performance-sensitive applications.
• Interface-based tools:
  • Useful when you need to define a reusable set of tools with multiple methods.
  • Define the toolset once and reuse it across your solution.
  • Use Cases: Plugin Systems, Microservices Communication, Domain-Specific Languages (DSLs), Data Access Layers, Workflow Automation, AI-Driven Data Validation (defining data validation rules as an interface), Personalized Learning Platforms (interfaces representing different learning modules).
• Method-based tools (using FunctionToolAttribute):
  • Best for scenarios with a small number of methods.
  • Suitable for class-scoped or static methods.
  • Use Cases: Command-Line Interface (CLI) Integration, Configuration Management, Hardware Control, Simple Utility Functions, Real-Time Data Analysis (wrapping single analysis methods), Context-Aware Notifications (methods that trigger specific notifications based on context).

2. Reflection-based QuickTool and QuickTools:

• NativeAOT/Trimming:
  • These tools are not recommended when NativeAOT or trimming is crucial.
• Use Cases:
  • Ideal when NativeAOT/Trimming is not a priority.
  • Useful for calling functions from third-party libraries where you don't control the source code.
  • Provides the ability to define anonymous functions, which is valuable for testing prompts and rapid prototyping.
  • Dynamic Scripting, Rapid Prototyping of API Integrations, Debugging and Testing Scenarios, Event Handling, Data Transformation Pipelines, AI-Powered Data Scraping (dynamically wrapping scraping functions), User Input Sanitization (quickly creating tools for input validation from untrusted sources), A/B Testing of Function Calls (quickly swaping out function logic for testing).

In summary:

• For maximum performance and compatibility with NativeAOT/Trimming, use interface-based tools or method based tools with FunctionToolAttribute.
• For rapid prototyping, testing, or integrating with third-party libraries, QuickTool and QuickTools offer flexibility.
• Interface based tools for reusable tool sets, and method based tools for smaller sets of methods.