Development flow - Azure/reliability-workbook GitHub Wiki

Fork the repository

In GitHub fork the repository from the latest branch to be your own. Verify the branch to work on.

Clone the repository

I am using Visual Studio Code to work on this solution. Install your favorite GIT, and Terraform extensions. Here are a few I like:

Review variables.tf

In this example I will walk through the files needed to update a section like “Databases”. First look at the variables.tf file. There were issues deploying to other Azure regions at one point. EastUS region was working so I continued to use this in my deployments. The resource group can be changed, but I would recommend using one that isn’t used for anything else so you can clear the entire group out when deploying the workbook during your development for testing. default = { location = "EastUS" name = "rg-workbook" }

To push to Azure you also have to change the variable to default = true. This will be completed with Terraform Apply commands later. variable "deploy_community_edition_to_azure" { type = bool default = true }

Review database.tf (Example)

To add a new database instance to the Database tab we will use the PostgreSQL as an example to show the changes that are needed. Open the databases.tf file in the explorer window. This databases.tf file consists of names of the KQL logical and file names that need to be mapped. The \build\template_kql\database/(new file name) will need to be created to match these names properly otherwise errors will be displayed during the terraform checks.

The next section of the database.tf file is mapping the KQL names to json nodes that are inside the \build\templates\database.tpl.json file that is used for creating the dashboard template.

Save this file after your changes are made. Next create these new blank files under the \build\template_kql\database\ file path.

Review extend_resource.kql

Before working on the the new files with content for the KQL to be used it would be good to work on a common file that is used for all the KQL queries in the workbook. This file is under \build\template_kql\common\extend_resource.kql. This file contains shared KQL used for all dashboards. For example it defines Resource Group and Environment “tag” variable that can be used on all resources so if the resource is in dev or prod it will flag this appropriately. There is a “where” clause that was needed to be changed to add in dbforpostgresql/servers and flexibleservers to query these resource types.

The rest of this document needs to be reviewed depending on the resource. For example, if the resource will have a state to be viewed change the case statement to include postgresql. It could be recommended to change only a few at a time to ensure that the KQL works properly when this is deployed.

Modify New KQL file

Next move back to the new files that were created for the KQL queries. For example the database_postgresqlflexible_resources_details.kql file, the content of this file would be: ${extend_resource} | project Type, SkuName, SubscriptionId, Name, State, Location, ResourceGroup, Zones, AvailabilityZone, AutomaticFailover, ReplicationRole, GeoBackup = tolower(Backup), Environment | where Type == 'microsoft.dbforpostgresql/flexibleservers'

This query above is using the common extend_resource.kql with any new “project” query values or where options.

Another KQL file is a summary file per resource that is querying items of interest that would count valuable items to be displayed.

The database_postgresqlflexible_resource_details_summary.kql example contains this KQL. ${extend_resource} | project Type, SkuName, SubscriptionId, Name, State, Location, ResourceGroup, Zones, AvailabilityZone, AvailabilitySet, FaultDomain, OSDisk, ReadReplica, AutomaticFailover, MultipleWriteLocations, StorageAutogrow, ReplicationRole, GeoBackup = tolower(Backup), Environment | where Type == 'microsoft.dbforpostgresql/flexibleservers' | summarize ResourceTotal=count(Name), NotRunning=dcountif(Name, State !in~ ("ready")), NotRecommendedSKU=dcountif(Name, SkuName == "burstable"), NoAVZone=dcountif(Name, AvailabilityZone == "not configured"), NoHA=dcountif(Name, AutomaticFailover == "not configured"), NoGeoBackup=dcountif(Name, GeoBackup == "not configured")

Save these changes to each file.

Review the template file

Moving onto the template. The templates are located under \build\templates. In this scenario we will look at the databases.tpl.json file.

Find an area to add in the new dashboard tab. I took a sample from MySQL and then made changes to make it fit for PostgreSQL. Change the gridSettings formatters sections to match the columns that will be displayed.

Some areas you may want to show icons or colors to what is displayed in the dashboard. In this example I wanted to flag skus that were not production. Burstable sku is going to show an orange color background.

This example is of showing icons vs. colors in the json file.

Below link is a good reference to learn about the Icon rendering options. Azure Workbooks rendering options - Azure Monitor | Microsoft Learn

Here is an example of colors and icons.

Deploy changes to Azure

When you are ready to deploy your changes locally to your Azure subscription make sure all the changed files are saved properly. In VScode go to the Terminal tab. Publishing using Linux WSL is recommended if using a Windows machine.

Running Visual Studio Code on Linux WSL - Visual Studio Marketplace Make sure you have Terraform installed. Install | Terraform | HashiCorp Developer Change your environment variable path for terraform or copy the .exe to the \build folder. Run the following commands

1. terraform.exe fmt -check (this checks spaces and formatting inside the file if it comes back with a file name run the command below)
   • terraform.exe fmt databases.tf
2. terraform.exe init
3. terraform.exe validate -no-color
   • Success! The configuration is valid.
4. terraform.exe plan -out tfplan
5. terraform.exe apply tfplan (this command will push the dashboards to Azure resource group)

The "apply" can take a few minutes to show in the Azure Portal and it seems that Azure portal doesn’t refresh this well. Try to refresh your entire Azure portal to see the files show up. Go to the resource group and you will see all the files listed.

Push Changes to GitHub

After you are happy with the changes you have made and successfully published to your own portal for verification, you are now ready to commit the changes back to Github. In VSCode go to the side panel for changes that have been made.

Once you stage the file that you need to update it should show up in the Staged Changes. Only stage the files that you need, otherwise you may push other files that are not needed by the dashboard.

Add a comment for source control and hit the drop down to select commit and push. If you are familiar with GIT command line feel free to use what you are used to using.