Challenge 3: Building a deployment workflow - alan9259/animal-adoption-portal GitHub Wiki
Challenge 3: Building a deployment workflow The customer is happy with the state of the project and would you to establish Test and Prod environments. This challenge leverages GitHub Actions to create a release and then deploy the compiled assets. DevOps concepts covered:
Releasing Deploying Following on from the previous challenge the code has been built and artifacts are ready to be released. Compiled assets should be released and deployed through a separate workflow that utilizes GitHub Actions.
For the following exercises you may choose to collaborate on a single release workflow, or each team member can create their own release workflow & associated resources:
Exercise 1 Before we can deploy our code, we need a Web Application in Azure to deploy it to. In this exercise, we will be preparing our Azure Resources and provide a GitHub workflow with the permissions it needs to publish to our test Resource Group.
Log into Azure portal.azure.com and create an Azure Web Application in your test environment. Find and visit the URL of your new Web App. You should see a default page. Download the Publish Profile Settings of your Web App. Add the contents of the Publish Profile Settings file as a secret with an appropriate name. NOTE - If you have decided to work on your own release workflow, then make sure that you have created your own Azure Web Application that you can deploy to. You may share an App Service Plan & Service Connection.
Hints Two Resource Groups have already been created for your team to use, they represent your Test and Production environments. We recommend sharing a single App Service Plan between all the components in each environment. Use S1 as the resource tier for the App Service Plan. Make sure you pay close attention to the region where you are deploying the resources. Ensure that the two environments are completely separate, having production services accidentally accessing test data can lead to some pretty dire consequences! Your Web App URL can be found by opening the resource in Azure. It is on the overview page under the URL heading. Helpful resources [Recommended naming and tagging conventions](Recommended naming and tagging conventions) Azure App Service Publish Profile Credentials Creating encrypted secrets for a repository Exercise 2 Now it's time to deploy the code to our Azure Web App. We will accomplish this by taking the published artifacts from our build job and deploying it to our test Azure resources using a release job.
Create a new job inside the existing workflow for deploying to the test environment. Update the new job to take your published artifacts from Challenge 2 and deploy it to your newly created Web App. Ensure that your release workflow will only automatically trigger when your build job is successful and changes were made to the master branch. Check that the deployment worked, you should see an error page saying that an environment variable is missing, rather than the default page that is shown when nothing is deployed to an Azure App Service. Hints You do not need to release into your production environment at this stage. A release workflow would typically be defined in its own file, however, due to a limitation with the download artifact action (an artifact only being able to be accessed within the same workflow) the build and release steps will be in the same file. Use the needs and if properties to apply conditions to jobs and steps. Helpful resources Download a Build Artifact action Workflow syntax - needs Workflow syntax - if Events that trigger workflows GitHub Action for deploying to Azure Web App Action samples for deploying to Azure Web Apps Exercise 3 Our release job should be deploying to the Web App in the test environment, we should now ensure that our Web App has the required configuration to run.
Update the Web Application settings in the Azure portal. Refresh the Web App and check that it loads correctly. In the Web App, go to the login page and test that the global password you set is working. Hints After releasing, certain settings will need to be set in order to run the application. You can find these in the AnimalAdoption.Web.Portal\appsettings.Development.json file Helpful resources Configure an App Service app in the Azure portal Exercise 4 Sometimes the person building the DevOps workflow may not have direct access to the resources in Azure and won't be able to manually set the Web Application settings.
Discuss with your team whether the two Application Settings values required for the TAA portal to function should be stored as environment variables, repository secrets, or a mixture of the two. Add the variables and/or secrets to your workflow job and/or repository. Find an action in the GitHub Actions marketplace that will allow you to configure the Application Settings of an Azure Web app and set it up to populate the values from your variables/secrets. This may require setting up other actions and secrets as a prerequisite. Test that the Application Settings were successfully applied and that the portal still works. Hints The action to update App Service Settings has a dependency, follow the instructions provided by either action to create an Azure Service Principal that can be used to authenticate with Azure and update your App Settings. If this fails, ensure that you are logged in using az login. You can also try creating the Service Principal by itself, then assigning the role in a separate command, or creating it through the Azure Portal. Make sure you store a copy of the Azure Service Principal's Client Id, as you will need it in the next exercise. If you don't want to build the JSON array for the app-setting-json property manually, you can copy it from the Advanced edit section of the Configuration page in the Azure Portal. Inline JSON within a step needs to be on a single line and surrounded by single quotes. Inline variables in the JSON will need to be surrounded by double quotes. Alternatively, you can add the whole JSON array as a secret variable and reference that. You may want to add a variable for your test Web App name since it is used in multiple places. Your variables will have to be accessed using the ${{ context.var_name }} syntax, rather than the $var_name syntax used by Bash. Before running your updated workflow you should "break" the portal by removing the Application Settings configuration, and ensuring that you get an error related to missing environment variables. You may have to refresh the page several times to clear the cache. Helpful resources Using variables and secrets in a workflow env context Azure Login action Azure App Service Settings action How and why applications are added to Azure AD Exercise 5 When applications are leveraging external services (such as Databases, external APIs etc), often these services are secured with security keys or credentials. Azure has a service for this called Azure Key Vault. This pattern is especially useful when production keys are managed externally from the development team.
In this exercise, we will store the secrets in a Key Vault and then update the workflow to leverage the Key Vault for secure deployments.
Start by creating a Key Vault resource inside the test Resource Group in Azure. Create the App Settings required by the Web App as secrets in your Key Vault. Extend your release job to retrieve all the keys from your Key Vault. Update your App Settings step to use the Key Vault variables. Hints There are a few methods of integrating Azure KeyVault. Experiment with each method and see what works best for your use case. You will need to give your Service Principal access to get secrets from Azure Key Vault. This can be done during the creation of the Key Vault or once it has been created by going to Azure Portal > {Your Azure Key Vault} > Access policies > Add Access Policy. When selecting a Principal you will need to know the Name / Client Id of your Service Connection. This is the Name / Client Id of the Service Principal you created in the previous exercise. Make sure you assign the appropriate Secret permissions, you can use the Secret Management template. Don't forget to remove any secrets or environment variables that have been superseded by the Key Vault secrets. Make sure to clear out the existing Application Settings against your web app before running the workflow. The Key Vault step needs to occur after the Azure login step, otherwise, it will be unable to authenticate. Helpful resources About Azure Key Vault Add a secret to Key Vault Azure Key Vault - Get Secrets action Embedding Keyvault dependencies in the application Exercise 6 Doing all manual steps to set up a new environment can be tedious, require a lot of documentation and introduce an element of risk via human error. By using Azure Resource Manager (ARM) templates you can define your Azure Infrastructure as code and ensure repeatable deployments without having to manually create any resources.
Follow the earlier instructions for creating a Service Principal to create a Service Principal for the production Resource Group. Make sure you add a secret to your repository for this. Get the Object Ids for your Azure Active Directory (AAD) account and the new Service Principal that you just created. This can be done through the Azure Portal: for users (AAD > Users > Search by email > Click), for the Service Principal (AAD > App registrations > Search by name > Click > Managed application in local directory > Click). Download the ARM template from here and add it in the root of your repository. Replace in the objectId property of the Key Vault definition with the Object Id of one of your team members (so they are granted access). Use the Azure Deploy tool to validate the ARM template. You do not need to Purchase/Deploy, simply clicking Save and being presented with the Purchase GUI means that validation was successful. Add a new job to your workflow, it should have a step to deploy the ARM template. The deployment can be configured using Azure CLI commands, or a marketplace action. You will need to set some parameters for the ARM template. Test the workflow once this has been configured. Validate that the deployment has worked and that only the team member who is configured can access the Keys, Secrets, and Certificates sections of the Key Vault resource. Manually add the Application Setting secrets to the production Key Vault. Update the production job to download the secrets from the production Key Vault to test that the Access Policy for the Service Principal has been configured correctly. Extend the ARM template to have Access Policies for each of your team members by using their object id. Once the workflow has run, check that all team members can now access the Keys, Secrets, and Certificates sections of the Key Vault resource. Update the production job to deploy the TAA portal into the production Resource Group. Update the deploy to test job to deploy the ARM template into the test Resource Group, with the appropriate values passed through for the parameters. Replace Publish Profile authentication for the Web App deploy task for test and production with Service Principal authentication. Refer to the documentation for the Azure WebApp action for guidance. Remove any repository secrets that are no longer required. Hints The deploy to productions job should only run if the deploy to test job is successful. It is recommended to create environment variables for easy management of parameters and arguments (such as Resource Group name). To share variables across jobs you will have to add them at the root level of the workflow. You can test your ARM template in the Azure Portal: Azure Deploy. You can debug deployment failures by going into {your Resource Group} > Deployments > {select the failed deployment} > Operation details. You want to use Incremental mode for your ARM template deploy. Make sure that you use the correct Service Principal for the associated Resource Group, i.e. Resources deploying to your production Resource Group will need to reference your production Service Principal. Sensitive values such as Tenant GUID, Subscription GUID, and Service Principal Object Ids should be stored as secrets in your Key Vault or as a repository secret. To pass the ARM template between jobs you will have to publish it as an artifact, then download it inside each job that requires it. Key Vaults are deployed in soft-delete mode by default, which means that even after a Key Vault has been deleted, the name is reserved until the retention period expires or the az key vault purge command (docs) is run to permanently delete it. Helpful resources What are ARM templates? Azure Resource Manager deployment modes Using workflow commands to access toolkit functions Workflow syntax for GitHub Actions Upload a Build Artifact action Download a Build Artifact action Deploy Azure Resource Manager templates by using GitHub Actions GitHub Action for Azure Resource Manager (ARM) deployment Azure WebApp action Extra information Information You can see which App Service Plan your Web App is using by opening your Web App in Azure by going to Overview > App Service Plan. If you go to Settings > App when viewing the App Service Plan you can see all of the App Services that are associated with the selected plan. If you have deleted the original App Service and Key Vault resources in a Resource Group to test a full deploy for the ARM template, the original Publish Profile will no longer be valid because the resource it is associated with no longer exists. We recommend updating your Web App deployments to use the relevant Service Principal for authentication, instead of using a publish profile. This will prevent the need to get the Publish Profile each time. Don't forget to remove the Publish Profile secrets when you are done. Links Export template in Azure Portal Exercise 7 Functional testing is used to test features or workflows of software applications by feeding in input data and examining the expected output against the actual output. Where unit tests focus on a single "unit" (block or method) of code, functional tests are more concerned with user/business requirements i.e. can a user with a valid account sign in to your site using the login page.
Like unit tests, functional tests can also be automated. Automated functional tests should be used in conjunction with unit and manual testing. Let's add a functional test job that ensures that the test Web App is working successfully before we deploy to production.
Install Google Chrome if it is not already installed. Download the appropriate version of ChromeDriver for the version of Chrome that you have installed. Copy chromedriver.exe to the root of your FunctionalTests project folder. Add the following snippet to AnimalAdoption.Web.Portal.FunctionalTests.csproj: PreserveNewest Build the solution and ensure that chromedriver.exe has been copied into AnimalAdoption.Web.Portal.FunctionalTests\bin<target-configuration>\netcoreapp3.1. Add a new (system) environment variable called ANIMAL_ADOPTION_FUNCTIONAL_TEST_PATH and set its value to the URL of the TAA test Web App. If you are using: Visual Studio: Right click on the FunctionalTests project and run the tests. Visual Studio Code: Install the .NET Core Test Explorer extension. Run the tests. Ensure that the existing functional test runs. Add a new functional test. Add a job to your workflow that runs the functional tests, it should run after the TAA Web App has been deployed to the test Resource Group, and prevent the TAA Web App from being deployed to the production environment if it fails. Hints After adding your local environment variables, you maybe have to restart your IDE for them to found. You will have to export all of the compiled assets of the Function Tests project as an artifact, then use this artifact inside your new job. The new job will need to have an environment variable set for ANIMAL_ADOPTION_FUNCTIONAL_TEST_PATH, and should only run if the deploy to test environment job was successful. You don't have to pass the environment variable to the run command, it will automatically be picked up. You can download and run VSTest.console.exe locally to debug the commands. Don't forget to update the dependency for your release to production job to only run if the functional test job is successful. Helpful resources Create and Modify Environment Variables on Windows UI Tests with Selenium. Kinds if automated tests Setup VSTest.console.exe VSTest.Console.exe command-line options Exercise 8 It is important to have a rollback plan if something goes wrong with a deployment. This can be as easy as redeploying a previous release, but some deployment targets allow simpler rollbacks by leveraging built-in methods for rolling back with minimal downtime.
Azure App Services allow you to deploy to specific deployment slots that run on the same infrastructure and allow instantaneous switching between new and old code.
Create a new slot named "staging" in your production environment. Test the staging slot to ensure that you are shown the default .NET Core landing page. You should also verify that the TAA web portal is still running on the production slot. Update your release to production job to deploy to your staging slot. Swap the production and staging slots. If you selected the production slot as the value for "Clone settings from" when setting up your staging slot, the UI will not show any changes in the preview interface. After the changes are live - immediately test the website again. You will have to hard refresh your browser. Update the workflow to deploy back to the production slot and run it again. Swap the production and staging slots back. Double check the traffic values to ensure the swap has occurred. Test that the production slot web app functions as expected Add the following snippet to the end of your ARM template, which will automatically recreate the "staging" slot as part of the resource deployment: json { "type": "Microsoft.Web/sites/slots", "apiVersion": "2018-11-01", "name": "[concat(parameters('webAppName'), '/staging')]", "location": "Australia East", "dependsOn": [ "[resourceId('Microsoft.Web/sites', parameters('webAppName'))]" ], "properties": { "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', parameters('appServicePlanName'))]" } } You don't need to add a defintion for the production slot as this is done by default. Hints Slots can be automatically swapped by configuring "auto swap". This is currently unsupported for Web Apps on Linux, as per this documentation, this page was last updated 04/30/2020. Slots can be configured to divert a percentage of traffic to allow monitoring while only affecting a small portion of your user base. You may want to create an environment variable inside the release to production job that holds the value of the slot to deploy to. This variable can then be referenced in the name and configuration of the deploy step. You will have to surround the name of the deploy step in quotes if you want the variable insertion to work correctly e.g. - name: 'Deploy the TAA portal to the ${{ env.PROD_WEB_APP_SLOT_TO_DEPLOY_TO }} slot of the Production Web App'. Helpful resources Set up staging slots Deploy to App Service using GitHub Actions