ArcGIS Enterprise Upgrades - Esri/arcgis-gitops GitHub Wiki

This document is a draft.

This document describes the upgrade procedure for base ArcGIS Enterprise site on AWS deployed using the ArcGIS GitOps templates.

Upgrading ArcGIS Enterprise is a complex process that may involve updating or modifying core infrastructure resources, such as networking, configuration management software, deployment infrastructure, operating systems, and third-party software, in addition to the ArcGIS Enterprise software itself. Even if these upgrades are not strictly necessary, it is advisable to keep all software and infrastructure up to date.

The upgrade process for ArcGIS Enterprise is described in the ArcGIS Enterprise documentation.

This guide covers scenarios where the ArcGIS Enterprise site was created using an earlier version of the GitOps templates. Each release of the templates supports a range of ArcGIS Enterprise versions, which are listed in the README file of the templates. An upgrade is supported by a template if both the current and target versions are supported by the template release.

If the site was not created using the templates, migration to the GitOps templates is also possible if certain settings of the original site match those used by the templates (see settings that must match between environments), that allows using WebGISDR tool to transfer data between the sites.

New sites created using the latest version of the templates with default settings also may introduce changes in infrastructure and application configuration that do not match those of the original site. However, the new templates are expected to provide an option to keep backward compatibility with the original site settings for the ArcGIS Enterprise versions supported by the templates.

A reliable upgrade process requires a working Business Continuity/Disaster Recovery (BC/DR) plan. See Backups and Disaster Recovery procedures for site's deployments recommended for the templates.

The standby upgrade site is a replica of the primary site that will be upgraded to the target ArcGIS Enterprise version. The upgrade site should be created using the latest stable version of the templates from arcgis-gitops repository used to create the site.

This guide assumes that the primary site matches the latest commit on the main branch of the private site's repository.

Login to GitHub and clone the private sites's repository. Replace "MyOrg" and "MySite" with the actual GitHub organization and repository names.

git clone [email protected]:MyOrg/MySite.git

cd MySite

Create a backup branch (for example "arcgis-11.3") from the main branch.

git branch <backup-branch>

Push the backup branch to the repository.

git push origin <backup-branch>

Add the arcgis-gitops repository as "upstream" remote.

git remote add upstream https://github.com/Esri/arcgis-gitops.git

git fetch upstream

Replace <release branch> with the arcgis-gitops repository's release branch name (for example v0.2.0) and checkout the release branch.

git checkout <release branch>

Push the release branch to the site's repository.

git push origin <release branch>

Create an upgrade branch from the release branch and push it to the site's repository.

git checkout -b upgrade

git push origin upgrade

Enable the workflows and update the site configuration to match the primary site.

Create "config-update" branch to merge the configuration from the primary site into the upgrade branch leveraging the GitHub merge tool.

git checkout -b config-update

Enable the workflows by copying the site's workflow files to .github/workflows/.

cp -r aws/arcgis-site-core/workflows/* .github/workflows/

For example, for Windows:

cp -r aws/arcgis-enterprise-base-windows/workflows/* .github/workflows/

or for Linux:

cp -r aws/arcgis-enterprise-base-linux/workflows/* .github/workflows/

If the workflows were modified/customized, the relevant changes also must be made in the config-update branch. In particular, check the scheduled workflows that may require changes in the schedule settings.

Copy the configuration and authorization files from the main branch to the config-update branch.

git checkout main config/

Change "site_id" in all the JSON config files of the config directory (config/aws/) to a new site id, for example gis114.

The "site_id" value must contain up to 6 lowercase letters, numbers, and hyphens. It must start with a letter and end with a letter or number.

Change "is_upgrade" property in application.tfvars.json configuration files of the site's deployments to false.

If the templates were modified/customized, the relevant changes also must be merged into the config-update branch.

Consider upgrading the operating system along with the application upgrades by changing the "os" property in image.vars.json and application.tfvars.json configuration files.

Commit and push the changes to the config-update branch.

git add --all

git commit -m "Update site configuration"

git push origin config-update

Create a pull request to merge the config-update branch to the upgrade branch. Review the changes, merge the pull request, and delete the config-update branch.

New versions of the templates may require changes in the IAM policies, the core AWS resources, and the Chef automation resources.

Compare the IAM policies specified for the new release of the templates with policies in the AWS account. Update the policies if required.

To prevent the upgrade process from accidental changes to the primary site, consider using environment secrets and separate IAM users with different policies for primary and standby sites. The IAM policies should the use conditions for ArcGISSiteId tag values to restrict access to the resources of the sites.

Run "site-core-aws" workflow with the upgrade branch to create the core AWS resources such as VPC, subnets, and IAM roles for the standby site.

To make sure that the workflow runs do not destroy any critical resources, first run the workflows with terraform_command set to "plan" and check the logs for the changes that will be performed when the workflow runs with terraform_command set to "apply".

Run "site-automation-chef-aws" workflow with the upgrade branch to create Chef automation resources such as Chef Cookbooks for ArcGIS and CINC client.

Replicate the primary deployment to a standby upgrade deployment created using the new version of the template.

Run "image" workflows with the upgrade branch for each deployment of the standby upgrade site to build EC2 AMIs for the deployments.

Run "infrastructure" workflows with the upgrade branch for each deployment of the standby upgrade site to provision AWS resources for the deployments.

Run "application" workflows with the upgrade branch for each deployment of the standby upgrade site to configure the deployment's applications.

Run "backup" workflows with the main branch for each deployment of the standby upgrade site to back up the deployments.

For base enterprise deployments set "backup_restore_mode" workflow input to "full" to create a full backup of the base enterprise deployment.

Run "restore" workflows with the upgrade branch to restore the backup to the upgrade deployment.

For base enterprise deployments set "backup_restore_mode" workflow input to "full" to restore the backup to the upgrade deployment.

Upgrade the standby site to the target ArcGIS Enterprise version. The upgrade process involves deactivating the primary site, creating an incremental backup of the primary site's base deployment, restoring the standby upgrade site's deployments from the backups, and running an in-place upgrades in the upgrade site's deployments.

These steps involve the system downtime and should be performed during a scheduled maintenance window.

Deactivate the primary site or change it to read-only mode to prevent any changes to the data during the upgrade process.

During the downtime, consider routing the site's domains to a static "Under maintenance" site.

Run "enterprise-base-windows-aws-backup"/"enterprise-base-linux-aws-backup" workflow with the primary branch and "backup_restore_mode" workflow input set to "incremental" to create an incremental backup of the primary base deployment.

Run "enterprise-base-windows-aws-restore"/"enterprise-base-linux-aws-restore" workflow with the upgrade branch and "backup_restore_mode" workflow input set to "incremental" to restore the incremental backup to the upgrade base deployment.

Add authorization files for the new ArcGIS Enterprise version to config/authorization/<ArcGIS version> directory of the upgrade branch and update the authorization files paths in application.tfvars.json files of the site's deployments.

Change "arcgis_version" property in application.tfvars.json file to the new ArcGIS Enterprise version and "is_upgrade" property to true.

Optionally, set "*_patches" properties in application.tfvars.json file to the lists of patch file names that must be installed with the upgrade.

Update "arcgis_version" and the "*_patches" properties in image.vars.json to match the values in application.tfvars.json file. Though the image.vars.json file is not used in the upgrade process, it is recommended to keep the values in sync for consistency.

Commit the changes to the upgrade branch and push the branch to GitHub.

Run "application" workflows with the upgrade branch for each deployment of the standby upgrade site to in-place upgrade the applications in the deployments to the new ArcGIS Enterprise version.

Activate the standby upgrade site to make it the primary site.

After the upgrade is completed successfully, perform the following steps to complete the upgrade process.

Run "test" workflows with the upgrade branch for each deployment of the standby upgrade site to test the upgraded deployment.

Run "destroy" workflows with the main branch for each deployment of the standby upgrade site and for the core infrastructure to delete the old site.

The upgrade branch cannot be simply merged into the main branch, because the branches do not have common history. Instead, either the main branch must be reset to the upgrade branch, or content of the upgrade branch copied to the main branch. Copying the content preserves history of the main branch.

Checkout the main branch.

git checkout main

Create upgrade-merge branch from the main branch.

git checkout -b upgrade-merge

Remove all the files in the branch.

git rm -rf .

Copy the content of the upgrade branch to the upgrade-merge branch.

git checkout upgrade -- .

Commit and push the changes to the upgrade-merge branch.

git add --all

git commit -m "Upgrade to ArcGIS Enterprise <version>"

git push origin upgrade-merge

Create a pull request to merge the upgrade-merge branch to the main branch. Review the changes, merge the pull request, and delete the upgrade-merge branch.

Upgrading an ArcGIS Enterprise site using the GitOps templates involves a series of well-defined steps to ensure a smooth and reliable transition to the new version. By adhering to these steps and performing the upgrade during a scheduled maintenance window, you can achieve a successful upgrade with minimal disruption to your ArcGIS Enterprise site.

Always ensure that you have a reliable backup and recovery plan in place before starting the upgrade process.

For further details and troubleshooting, refer to the Upgrade ArcGIS Enterprise documentation.