DevOps Best Practices (AZ‐400) - amitbhilagude/userfullinks GitHub Wiki

  1. Analysing Metrics in Azure
    1. Create a Dashboard and Check Metrics in Azure Monitor
    2. Set Application Health Endpoints into Azure Application Insights Availability test. Further, Create alerts or Actions on those tests
    3. Create baseline metrics. The best way to start the Baseline metrics is to go though Azure Monitor Insights, E.g. Application that has an App insights dashboard, Go through individual resources, etc.
    4. Use the above baseline Metric for performance testing to see the difference.
    5. Azure Monitor and App insights allow you to set a different alert. Those alerts can be static and dynamic. Static alert has a fixed value e.g. CPU goes higher with % trigger the alert however dynamic uses Machine learning and detects automatically when to set the alert.
    6. Azure Monitor also has smart detection feature to give a recommendation based on the machine learning
    7. Azure Application insights support dependency map can help to find out all calls from different system and failure rate
  2. Source Control Strategy
    1. Two Options Github and TFVC. Recommended is Github
    2. Repository sharing with Submodules: Submodules is best option in case you need to reused another repo e.g. Build script is one repo and every repo needs it then you create submodule
  3. Branching strategy
    1. Policies must be set in terms reviewer, restrictions etc.
    2. Three options for Branching strategy: Trunk based, Feature(Task) based and release based. First one is used for small team, second one is good for CI\CD and medium size, third one is good for large team which different code per customer.
  4. Repository Strategy
    1. Handling Large repository
    2. Branch policies and Security
    3. Branch lock and restrictions
    4. Rollback or reset is reset particular commit which will overwrite all above commits and you will never see them again in history
    5. If the branch is deleted, you can restore it but the only way to restore it to search with the exact name.
  5. Versioning Strategy
    1. Use of semantic versioning Major.Minor.patch-tag
  6. Build Pipeline Automation
    1. Recommended tools for security scanning checkmark or Snyk or WhiteSource Bolt or Black duck
    2. Test automation Unit testing, functional testing e.g. Individual microservice, Integration testing and UI testing using salenium
    3. Code coverage. Available in Visual Studio as well as tasks in the Build pipeline in C#. An alternate option is sonarqube
    4. The Health Status of the build pipeline will be monitored using built-in charts
    5. Pipeline performance can be improved by enabling caching.
    6. Cost vs Efficiency: A cost optimization solution could be having minimal agents however that will make the developer idle as pipelines will stay in a queue. so the preferred option is to have multiple agents which will reduce the time and improve the efficiency.
    7. The best practice to have YAML template created and store in same or other repository which can be used by multiple pipelines
    8. Create a variable group that can be used in multiple pipelines. It will have keyvault secret integration feature.
    9. Build and package deployment can be included into Same pipeline with different stages.
    10. Use of Azure Policy to enforce compliance in Azure e.g. naming convention, tags, etc.
  7. Deployment Scripts and Template
    1. ARM template vs Terraform :
      1. The ARM template is Azure Specific. Terraform is for multiple cloud provider
      2. Provisioning the latest resource is always available in ARM but Terraform may not have immediately.
      3. Terraform maintains state but ARM template is not.
      4. Terraform has clean up commands. ARM template doesn't have clean up commands
    2. Bicep
      1. Human readable markup of ARM template
      2. Internally it converts this mark up into ARM template only before converting to ARM.
    3. Include common checks e.g. ARM template JSON validation using linting added into build pipeline
    4. Multiple options for creating SQL DB through Azure Pipeline e.g. SQL scripts or DACPAC (File can be exported from SQL)
  8. Release Pipeline
    1. Release pipeline supported Gated deployment and Approval
    2. It is a good option instead of adding into YAML. The version and history of release pipeline is manged through Azure DevOps.
    3. Azure App configuration: Recommended option for Centralised settings(Key-Value pair) and Feature toggle.Feature toggle support different target like group of people, % etc. Integration of feature flag will take care it automatically.
    4. Release gates and Approval: Couple of pre and post condition can be applied and allow to approve the stage
  9. Managing sensitive information in Automation
    1. Service Principal
    2. System Identity and Managed Identity
    3. Store secrets in Key vaults and integrate Key vault in Build pipeline with below option
      1. Use of Variable Group
      2. Read secrets directly from Key vault in Pipeline
    4. Create plain text secret and lock it in Azure DevOps
    5. Creating service connection and manage the permission for that service connection
  10. Security and Compliance Scanning
    1. Container Dependency scanning. Two Options
      1. Integrate into the Build Pipeline https://marketplace.visualstudio.com/items?itemName=Qualys-Inc.build-release-task
      2. ACR takes care automatically using Qualys https://azure.microsoft.com/en-in/updates/scan-container-images-for-vulnerabilities-in-azure-security-center/
    2. Security Scanning
      1. Two options: As part of CI, use of different tools mentioned above which is active test and nightly build which will become passive test
    3. Compliance Scanning
      1. Sonarqube
  11. Monitoring DevOps Environment
    1. Azure Monitor: One place to see all Logs
    2. Log Analytics: Use of Workspace to see logs. The workspace will be centralized per region or in the resource context which means you can only see data if you have access to the Resource
    3. Azure Application Insights: APM tool to get more insight into application level like exception, request, AJAX call, etc. Distribution\End to end tracing will be done with the Correlation Id
    4. Expected one instance per region for log analytics.
    5. Infrastructure performance monitoring key indicators
      1. High-level performance indicator
        1. Request rate
        2. Traffic rate
        3. Transaction rate
        4. Latency Rate
      2. Low-level performance indicator
        1. Memory
        2. Number of Thread
        3. CPU utilization
        4. Disk IO
        5. Network IO
    6. Setup alerts, Notification and Action Group. Additionally, alert can be notified in Teams channel using Action and Logic app.
  12. Configuring communication and collaboration
    1. Dashboard and widgets with set of charts
    2. Set of team analytics charts
    3. Cost Management: User of Alerts, budget, Cost center and azure advisor on the cost. There is also template in Cloud Adaption Framework https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/govern/cost-management/template
    4. Link Work items in every PR
    5. Microsoft Team Integration with Azure DevOps to manager all communication at the central place. https://azuredevopslabs.com/labs/vstsextend/teams/
  13. Documentation
    1. Onboarding
    2. Dependency Tracking
    3. Release Notes
    4. Semantic versioning standards