ProGuide Create a Pro Assistant Function - Esri/arcgis-pro-sdk GitHub Wiki

Language:      C#
Subject:       Framework
Contributor:   ArcGIS Pro SDK Team <[email protected]>
Organization:  Esri, http://www.esri.com
Date:          04/09/2025
ArcGIS Pro:    3.6
Visual Studio: 2022

This ProGuide explains the step by step process on how to create an ArcGIS Pro Assistant Extension to extend ArcGIS Pro's Assistant. The ArcGIS Pro Assistant can enhance productivity by helping you complete GIS tasks efficiently. It is embedded in ArcGIS Pro and uses artificial intelligence to answer your questions, generate queries, and perform actions. The built-in functionality of the Pro Assistant can be extended via the SDK by creating your own custom Assistant Extension class. An Assistant Extension class can contain one or more Assistant functions that will be invoked by the Assistant using AI.

In this topic

Background
Prerequisites
Online Administrator Settings
Step 1 Create a new Addin Project
Step 2 Run the SDK Assistant Extension Item Template
Step 3 Examine the Code
Step 4 Examine the Config.daml
Step 5 Change Assistant Extension and Function Descriptions
Step 6 Create Layout Content Custom Class
Step 7 Update Extension Declaration in the Config.daml
Step 8 Override Built in Assistant Extensions
Step 9 Compile
Step 10 Debug
Step 11 Run the Pro Assistant
Step 12 Invoke Your Extension via the Chat

Background

In ArcGIS Pro, a Pro Assistant Extension is represented as a C# class and should focus on providing one type of functionality for example map-navigation, table editing, or, in the case of this guide, layout creation. The extension, sometimes referred to as "an agent", contains one or more Assistant Functions, each of which will perform a discrete task. Each Assistant Function is responsible for handling particular tasks, allowing language models to interact with and utilize your existing APIs and services effectively. Under the hood, the Pro Assistant uses a built-in feature of many modern LLMs called function calling which lets the LLM determine which functions to call and what arguments to pass in based on the user request. The LLM communicates back to the Pro Assistant which can then auto-invoke the function, or functions, selected by the LLM to execute whatever are the relevant tasks in ArcGIS Pro. The implementation of an Assistant Function uses "standard" implementations of the Pro SDK public apis (same as any other addin).

In this example, we are going to author a custom Assistant Extension containing an Assistant Extension Function to create a layout using the current map.

Prerequisites

Ensure you have ArcGIS Pro 3.6 installed along with Pro Assistant ProAssistant.msi and the Pro SDK. When installing Pro, ensure that the optional install for AI models is enabled.

ProAssistantInstall

ProAssistantInstall2

Online Administrator Settings

In your ArcGIS Organization, the following administrator settings must be enabled:

ProAssistantEnableOnline

Also, ArcGIS Pro (Beta) must not be blocked on the security settings tab. The easiest way is to disable block of Esri apps.

ProAssistantDisableOnlineBlock

However, if organization policy requires that Esri Beta apps be blocked, then use the "Manage blocked Esri apps and capabilities" button to add an exception for ArcGIS Pro.

Refer to Configure AI assistants and Blocked Esri apps for more information.

If, when running the Pro Assistant, you get a 401 error in the chat response then there is probably an issue with your portal/organization admin permissions.

Step 1 Create a new Addin Project

Create a new addin project. Call it CreateLayoutAIExtension. Store it in the folder of your choice.

Step 2 Run the SDK Assistant Extension Item Template

From the right-click visual studio project context menu, select "Add->New Item...". Click on the (newly installed) "ArcGIS Pro Assistant Extension (Beta)" item template to create a new AI Assistant extension. Call your AI Assistant extension "CreateLayoutExtensionClass".

ProSDKAIAssistantTemplate

Step 3 Examine the Code

The template will have generated a new AI Assistant extension class called CreateDefaultLayoutClass containing an AI assistant function called CreateDefaultLayout. We will inspect the code.

[Description("CreateLayoutExtensionClass class for ArcGIS Pro Assistant Extension")]
internal class CreateLayoutExtensionClass : AIAssistantExtension
{
   [AIAssistantFunction, Description("CreateLayoutExtensionClassFunction description")]
   public static AIFunctionResult CreateLayoutExtensionClassFunction(
     [Description(
       "CreateLayoutExtensionClassFunction optionalTextString parameter description")] string optionalTextString = "")
    {
        //TODO: CreateLayoutExtensionClassFunction implementation goes here
        System.Diagnostics.Debug.WriteLine(
         $"Project Name: {Project.Current.Name}. Executing CreateLayoutExtensionClassFunction with parameter {optionalTextString}: {DateTime.Now}");
        return new AIFunctionResult("CreateLayoutExtensionClassFunction was executed");
    }
}

Some important points of interest: AI assistant extension class functions are declared as public static. They return type of ArcGIS.Desktop.Internal.Core.Assistant.AIFunctionResult*. They are decorated with the [AIAssistantFunction, ...] attribute. This is what identifies a "generic" function as being an "AI Assistant extension function". The "default" AI Assistant extension function auto-generated by the template also includes an optional function parameter string optionalTextString. Note that it too has a [Description("...")] attribute, same as its parent function (whose description is included within the AIAssistantFunction attribute).

* Pro Assistant is still Beta at 3.6. Therefore, addins extending the AI Assistant will be using "internal" namespaces as currently.

Step 4 Examine the Config.daml

We will inspect the Config.daml. Notice that the item template also added registration "code", or "xml", into the Config.daml:

 <categories>
   <updateCategory refID="esri_core_ai_assistant_extension_category">
     <insertComponent id="CreateLayoutAIExtension_CreateLayoutExtensionClass" 
                                                              className="CreateLayoutExtensionClass">
       <content>
         <searchDescriptions>
          <searchDescription>CreateLayoutExtensionClass class for ArcGIS Pro Assistant Extension</searchDescription>
         </searchDescriptions>
         <aiAssistantFunction name="CreateLayoutExtensionClassFunction">
           <searchDescriptions>
             <searchDescription>CreateLayoutExtensionClassFunction description</searchDescription>
           </searchDescriptions>
           <!--<dependencies>-->
           <!--Add a dependency Pro Assistant Function to your function-->
           <!--<dependency refID="esri_mapping_assistant_extension|GetAllLayers"/>-->
           <!--</dependencies>-->
         </aiAssistantFunction>
       </content>
     </insertComponent>
   </updateCategory>
 </categories>

Some important points of interest: AI assistant extension classes and their child AI assistant functions must be registered in the esri_core_ai_assistant_extension_category category. The registration includes an <insertComponent .../> element for the class and one <aiAssistantFunction .../> element per child AI assistant functions. In this case the template generates just a single function: "CreateLayoutExtensionClassFunction" so there is one <aiAssistantFunction .../> child element in this case. The class name attribute of the insertComponent element must match the class name of the extension function - "CreateLayoutExtensionClass" in this case. The name attribute on the aiAssistantFunction element must match the name of the AI assistant extension function - "CreateLayoutExtensionClassFunction" in this case. We will be changing the various "description" attributes in the following steps.

Step 5 Change Assistant Extension and Function Descriptions

We are going to add the code to our function to generate a default layout (via the AI Assistant). First, change the description attribute of the AIAssistantExtension class, CreateLayoutExtensionClass, as follows:

   [Description("Create, add, or insert a layout using the active current map.")]
   internal class CreateLayoutExtensionClass : AIAssistantExtension
   {

Next, change the description of the Assistant Function and the description of the optional input parameter - see below. Change the name of the optional parameter to be optionalLayoutTitle and give it a default value of "Default Layout". Your code should match the code shown below:

[Description("Create, add, or insert a layout using the current active map.")]
internal class CreateLayoutExtensionClass : AIAssistantExtension
{
   [AIAssistantFunction, Description("Create, add, or insert a layout using the current active map. " +
"The user can optionally include a layout name")]
   public static AIFunctionResult CreateLayoutExtensionClassFunction(
     [Description("The optional name or title of the layout")] 
   string optionalLayoutTitle = "Default Layout")
   {
       //TODO: CreateLayoutExtensionClassFunction implementation goes here
       System.Diagnostics.Debug.WriteLine(
          $"Project Name: {Project.Current.Name}. Executing CreateLayoutExtensionClassFunction " +
          $"with parameter {optionalTextString}: {DateTime.Now}");
       return new AIFunctionResult("CreateLayoutExtensionClassFunction was executed");
   }

Note these changes:

The extension and function description attributes have been changed to "Create, add, or insert a default layout using the current active map". This is important. These descriptions will be used by the LLM to determine the purpose of our extension and function.
The function parameter description has been changed to "The optional name or title of the layout" which will also be used by the LLM to determine the purpose of the parameter - i.e. to allow a user to specify the name of the layout. Note also that, if no name is provided, it defaults to "Default Layout".

Next, we need to add the actual implementation of our custom assistant function.

Step 6 Create Layout Content Custom Class

Add the layout generation code to the body of the method. This is "standard" layout creation code using LayoutFactory and ElementFactory same as would any other addin code needed for layout generation. Note also the use of QueuedTask.

   [AIAssistantFunction, Description("Create, add, or insert a layout using the current active map. " +
"The user can optionally include a layout name")]
   public static async Task<AIFunctionResult> CreateLayoutExtensionClassFunction(
     [Description("The optional name or title of the layout")] 
   string optionalLayoutTitle = "Default Layout")
{
   //Get the active map view.
   var map = MapView.Active.Map;

   var layout = await QueuedTask.Run(() =>
   {
      var layout = LayoutFactory.Instance.CreateLayout(8.5, 11, LinearUnit.Inches);
      layout.SetName(optionalLayoutTitle);

      //Add a map frame
      var mf_env = EnvelopeBuilderEx.CreateEnvelope(0.77, 1.3, 7.7, 9.7);

      var mf = ElementFactory.Instance.CreateMapFrameElement(
		layout, mf_env, map, "Default Map", false);

     //Add a title
     var title_sym = SymbolFactory.Instance.ConstructTextSymbol(
		ColorFactory.Instance.BlackRGB, 36, "AvenirNext LT Pro Medium", "Italic");
     var title_text = new Coordinate2D(0.77, 9.7);

     ElementFactory.Instance.CreateTextGraphicElement(
		layout, TextType.PointText, title_text.ToMapPoint(), title_sym,
		optionalLayoutTitle, "Title Text1", false);

      //Add a north arrow
      var na_frame = EnvelopeBuilderEx.CreateEnvelope(7.1, 1.3, 7.7, 1.9);

      var north_arrow_info = new NorthArrowInfo()
      {
          MapFrameName = mf.Name
      };

      ElementFactory.Instance.CreateMapSurroundElement(
          layout, na_frame, north_arrow_info, "", false,
          new ElementInfo() { Anchor = Anchor.BottomRightCorner});

      //Add a scale bar
      var sb_frame = EnvelopeBuilderEx.CreateEnvelope(3.09, 0.914, 5.4, 1.41);

      var sbar_info = new ScaleBarInfo()
      {
          MapFrameName = mf.Name
      };

      ElementFactory.Instance.CreateMapSurroundElement(
          layout, sb_frame, sbar_info, "", false,
          new ElementInfo() { Anchor = Anchor.BottomMidPoint});

      return layout;
   });

   //GUI thread
   await FrameworkApplication.Current.Dispatcher.InvokeAsync(async () =>
   {
      await ProApp.Panes.CreateLayoutPaneAsync(layout);
   });

   return new AIFunctionResult($"Layout '{optionalLayoutTitle}' created and opened");
}

Additionally, we changed the return type from AIFunctionResult to Task<AIFunctionResult> reflecting that this particular method will be asynchronous. This is always usually going to be the case as most of the Pro SDK public api requires use of a QueuedTask

Step 7 Update Extension Declaration in the Config.daml

We will now update the extension and function entries in the Config.daml. We are going to add a series of <searchDescription> elements to both the child <content><searchDescriptions>...</searchDescriptions></content> of the "parent" <insertComponent ...> element representing the extension, as well as the child <searchDescriptions>...</searchDescriptions> element of the <aiAssistantFunction ...> for the assistant function. Search descriptions in the Config.daml are used "up front" by the Pro Assistant semantic search. Unlike the [Descriptions(....)] we added in the code-behind, <searchDescriptions>...</searchDescriptions> are not used by the LLM.

For our search descriptions we are looking for multiple phrases, each of which covers the different natural language alternatives that a user could type into the chat, to invoke our function. The more possibilities we add, the better our potential "score" against relevant user prompts requesting creation of a layout using the current map. Here are the descriptions we are going to add:

<searchDescription>Create, add, or insert alayout using the default map</searchDescription>
<searchDescription>Create, add, or insert a layout using the current map</searchDescription>
<searchDescription>Create, add, or insert a layout using the current active map</searchDescription>
<searchDescription>Create, add, or insert a layout using the active map</searchDescription>
<searchDescription>Create, add, or insert a new layout</searchDescription>

We will use the same search descriptions for the extension and the function*. Your Config.daml entry should now look like this:

<categories>
 <updateCategory refID="esri_core_ai_assistant_extension_category">
  <insertComponent id="CreateLayoutAIExtension_CreateLayoutExtensionClass" className="CreateLayoutExtensionClass">
   <content>
    <searchDescriptions>
      <searchDescription>Create, add, or insert alayout using the default map</searchDescription>
      <searchDescription>Create, add, or insert a layout using the current map</searchDescription>
      <searchDescription>Create, add, or insert a layout using the current active map</searchDescription>
      <searchDescription>Create, add, or insert a layout using the active map</searchDescription>
      <searchDescription>Create, add, or insert a new layout</searchDescription>
    </searchDescriptions>
    <aiAssistantFunction name="CreateLayoutExtensionClassFunction">
     <searchDescriptions>
       <searchDescription>Create, add, or insert alayout using the default map</searchDescription>
       <searchDescription>Create, add, or insert a layout using the current map</searchDescription>
       <searchDescription>Create, add, or insert a layout using the current active map</searchDescription>
       <searchDescription>Create, add, or insert a layout using the active map</searchDescription>
       <searchDescription>Create, add, or insert a new layout</searchDescription>
     </searchDescriptions>
     <!--<dependencies>-->
     <!--Add a dependency Pro Assistant Function to your function-->
     <!--<dependency refID="esri_mapping_assistant_extension|GetAllLayers"/>-->
     <!--</dependencies>-->
    </aiAssistantFunction>
   </content>
 </insertComponent>
</updateCategory>

*As of 3.6 beta, the search descriptions that are child elements of the <aiAssistantFunction><searchDescriptions> are not currently used.

Step 8 Override Built in Assistant Extensions

To ensure that other assistant extensions do not intefere with our own assistant extension, we can override them. In this case, we want to override the built-in layout extension functions and built-in geoprocessing functions. We do not want the LLM to select built-in functions for layout creation, we want it to select ours. Overriding built-in extensions ensures that they are omitted from consideration by the LLM. To override an extension and/or one of its functions, we use the <overrides><override refID="extension_daml_id"></override> element. To omit just a single function from within an extension, use the daml id of the assistant extension paired with the pipe | symbol plus the daml id of the function (to be omitted or overriden). Add the following section to the <content ...> of your assistant extension:

 <overrides>
   <override refID="esri_geoprocessing_assistant_extension"/>
   <override refID="esri_layoutFactory_assistant_extension|CreateLayout"/>
 </overrides>

Your Config.daml entry should now look like this:

<categories>
 <updateCategory refID="esri_core_ai_assistant_extension_category">
  <insertComponent id="CreateLayoutAIExtension_CreateLayoutExtensionClass" className="CreateLayoutExtensionClass">
   <content>
    <searchDescriptions>
      <searchDescription>Create, add, or insert alayout using the default map</searchDescription>
      <searchDescription>Create, add, or insert a layout using the current map</searchDescription>
      <searchDescription>Create, add, or insert a layout using the current active map</searchDescription>
      <searchDescription>Create, add, or insert a layout using the active map</searchDescription>
      <searchDescription>Create, add, or insert a new layout</searchDescription>
    </searchDescriptions>
    <overrides>
      <override refID="esri_geoprocessing_assistant_extension"/>
      <override refID="esri_layoutFactory_assistant_extension|CreateLayout"/>
    </overrides>
    <aiAssistantFunction name="CreateLayoutExtensionClassFunction">
     <searchDescriptions>
       <searchDescription>Create, add, or insert alayout using the default map</searchDescription>
       <searchDescription>Create, add, or insert a layout using the current map</searchDescription>
       <searchDescription>Create, add, or insert a layout using the current active map</searchDescription>
       <searchDescription>Create, add, or insert a layout using the active map</searchDescription>
       <searchDescription>Create, add, or insert a new layout</searchDescription>
     </searchDescriptions>
     <!--<dependencies>-->
     <!--Add a dependency Pro Assistant Function to your function-->
     <!--<dependency refID="esri_mapping_assistant_extension|GetAllLayers"/>-->
     <!--</dependencies>-->
    </aiAssistantFunction>
   </content>
 </insertComponent>
</updateCategory>

Step 9 Compile

Compile and fix any errors.

Step 10 Debug

Run the addin in the debugger. Open any Pro project and make sure that it has an active map (otherwise your function method will be disabled). Open the AI Assistant dockpane. You will find it on the Help tab. You must sign-in to your arcgis online account in-order to use the Pro Assistant. Also, Pro Assistant (Beta) must have been enabled for use by your organization or you will get a 401 permission denied error. Refer to the Online Administrator Settings section above.

ProAssistantTab

Step 11 Run the Pro Assistant

Start a "Pro Assistant action session". This will switch the Pro Assistant from executing Pro Help (the default) in response to your chat entries to executing Pro actions (as implemented by Pro assistant extension functions). You can initiate an action session either via the burger button "ArcGIS Pro Actions" option or by clicking the "ArcGIS Pro Actions - Take action in ArcGIS Pro..." button on the assistant dockpane.

ProAssistantActionSession

Step 12 Invoke Your Extension via the Chat

Type into the chat "Create a layout using the current map". Hit enter. You should see a new layout pane generated (via your ai assistant function). Feel free to experiment with different phrasing and specifying a title string, or not.