Define and Implement Continuous Integration - VishalPatangay/My-devops-repo GitHub Wiki

Create a build pipeline with Azure Pipelines

What is Azure Pipelines?

Microsoft Azure Pipelines is a cloud service that you can use to automatically build, test, and deploy your code project. You can also make it available to other users. And it works with just about any language or project type.

What is continuous integration?

Continuous integration (CI) is the process of automating the build and testing of code every time a team member commits changes to version control.

CI encourages developers to share their code and unit tests by merging their changes into a shared version control repository after every small task completion.

Manage Build Dependencies

Why should I build a package? There are advantages to building a package as opposed to duplicating the code.

One reason to create a package instead of duplicating code is to prevent drift. When code is duplicated, each copy can quickly diverge to satisfy the requirements of a particular application. Another reason to build a package is to provide a consistent way to build and test that package's functionality. When multiple applications can benefit from the same code, the advantages far outweigh the disadvantages.

Where are packages hosted? You can host packages on your own network, or you can use a hosting service. A hosting service is often called a package repository or package registry. Many of these services provide free hosting for open source projects.

A package feed refers to your package repository server. This server can be on the internet or behind your firewall, on your network. For example, you can host your own NuGet feeds by using hosting products such as Azure Artifacts and MyGet. You can also host packages on a file share.

Who can access packages? Many package feeds provide unrestricted access to packages. For example, you can download Json.NET from nuget.org, without the need to sign in or authenticate.

How are packages versioned? The versioning scheme depends on the packaging system you use.

For example, NuGet packages use Semantic Versioning .

Semantic Versioning is a popular versioning scheme. Here's the format:

Major.Minor.Patch[-Suffix]

How do i create an Azure package and use it in pipeline

Create the package First, we need to create a project in Azure Artifacts. Callout 1 We can do this from Azure DevOps.

Then, we create a pipeline in Azure Pipelines that connects to the GitHub repo for the package code. Then the pipeline builds the code, packages it, and pushes the package to Azure Artifacts. Callout 2

We need to update the application that consumes this package to point to the Azure Artifacts feed that we created. Callout 3

After that, we update the pipeline that creates our application. The update allows us to use our Azure Artifacts feed to pull the new package dependency, and build as normal. Callout 4

  1. What is a package? You can host packages on a network share or use a web service. Authenticated users can download your package and use it in their own applications.
  2. Say you've built a package that you want to share publicly. What's the easiest way to do that? Hosting services such as NuGet make it easy for you to upload packages, as well as for others to discover and use them. 3.What is Azure Artifacts?You can configure your feeds to allow public access or access only from select users.

Setup Azure Env:

https://docs.microsoft.com/en-us/learn/modules/manage-build-dependencies/3-set-up-environment What is a remote in Git: A remote is a Git repository where team members collaborate (like a repository on GitHub). What is Origin: Origin specifies your repository on GitHub. When you fork code from another repository, it's common to name the original remote (the one you forked from) as upstream. git remote -v -> To list the remote repositories.

Create a Package Feed in Azure

Setup Azure Artifacts Feeds host your packages and let you control permissions

From Azure DevOps, go to the Artifacts tab, and then select + Create feed.

Select connect to feed.

Create Package pipeline

  1. Create a fork - The first step is to fork the mslearn-tailspin-spacegame-web-models repository so you can work with and modify the source files.
  2. Clone your fork locally - To clone the mslearn-tailspin-spacegame-web-models projects to your computer:

Go to your fork of the mslearn-tailspin-spacegame-web-models project on GitHub.

Select Clone or download. Then select the button next to the URL that's shown to copy the URL to your clipboard.

From Visual Studio Code, go to the terminal window and run this git clone command. Replace the URL that's shown with the contents of your clipboard.

How do you access a Branch:

  1. git fetch upstream models-package - to fetch a branch
  2. git checkout -b models-package upstream/models-package - To switch to the branch

https://docs.microsoft.com/en-us/learn/modules/manage-build-dependencies/6-consume-package Stage, commit, and push your changes to GitHub. git add . git commit -m "Add reference to Models package" git push origin models-package

Origin - Github repo

https://docs.microsoft.com/en-us/learn/modules/manage-build-dependencies/7-push-a-change

Create a branch: Create a branch named add-game-style, which is based off the master branch. git checkout -b add-game-style

Types of packages Earlier in this module, we discussed some of the formats that are available. Here's more information about them.

NuGet.

NuGet packages are used for .NET code artifacts. These artifacts include .NET assemblies and related files, tooling and, sometimes, metadata. NuGet defines the way packages are created, stored, and consumed. A NuGet package is essentially a compressed folder structure with files in the ZIP format and has the .nupkg extension. See also, An introduction to NuGet .

NPM.

An NPM package is used for JavaScript. An NPM package is a file or folder that contains JavaScript files and a package.json file that describes the metadata of the package. For node.js, the package usually contains one or more modules that can be loaded once the package is consumed. See also, About packages and modules .

Maven.

Maven is used for Java-based projects. Each package has a Project Object Model file that describes the metadata of the project and is the basic unit for defining a package and working with it. See also, Apache Maven Project .

Docker.

Docker packages are called images and contain complete, self-contained deployments. Most commonly, a Docker image represents a software component that can be hosted and executed by itself, without any dependencies on other images. Docker images are layered and might be dependent on other images. See also, Docker .

Configure the pipeline to access security and license ratings There are several tools available from third parties to help you assess the security and license rating of the software packages you use.

Some of these tools scan the packages as they are included in the build or CD pipeline. During the build process, the tool scans the packages and gives instantaneous feedback. During the CD process, the tool uses the build artifacts and performs scans. Two examples of such tools are WhiteSource Bolt and Black Duck . With Azure DevOps, you use build tasks to incorporate scanning into your pipeline.

To learn how to incorporate WhiteSource Bolt into your pipeline, see Scan open source components for vulnerabilities and license ratings in Azure Pipelines .https://docs.microsoft.com/en-us/learn/modules/scan-open-source/

Host your own Build agent in Azure pipelines What are build agents and agent pools? A build agent is a system that performs build tasks. Think of it as a dedicated server that runs your build process. https://docs.microsoft.com/en-us/learn/modules/host-build-agent/2-choose-a-build-agent Imagine that you have an Azure Pipelines project that receives build requests many times per day. Or perhaps you have multiple projects that can each use the same type of build agent. You can organize build agents into agent pools to help ensure that there's a server ready to process each build request.

When a build is triggered, Azure Pipelines selects an available build agent from the pool. When you use a Microsoft-hosted agent, you specify the VM image to use from the pool. Here's an example from your existing build configuration that uses an Ubuntu 18.04 build agent: pool: vmImage: 'ubuntu-18.04' demands:

  • npm When you use a Microsoft-hosted agent, you use vmImage to specify which type of system you need. The demands section specifies which software or capabilities you require the build machine to have. Azure Pipelines supports these operating systems:

Windows macOS Linux (Ubuntu, Red Hat Enterprise Linux, and CentOS)

  1. Let's say you're building a video game. The build process takes two hours to run and uses 18 to 20 GB of disk space to compile the game assets. What kind of build agent might you use? Although the build is completed within the allowed time, you might need a self-hosted agent to accommodate the amount of disk space that your build requires.
  2. Let's say you're building an application that runs on macOS, Linux, and Windows. How might you build the app for each platform you target? You can configure one pipeline to build on multiple platforms. YOu need not have multiple pipelines for each OS.
  3. A self-hosted build agent:You can run a self-hosted agent anywhere. The agent must meet the requirements and be able to connect to Azure Pipelines.

Exercise(to perform a POC whether using a private build server helps than Microsoft hosted agents): Setup the Devops project environment in Azure. https://docs.microsoft.com/en-us/learn/modules/host-build-agent/3-set-up-environment

Setup the project locally Open VS code> Bash shell> run cd ~ to navigate to the folder from which you want to work from. Configure Git:(https://help.github.com/articles/set-up-git) A fork is a copy of a repository. Forking a repository allows you to freely experiment with changes without affecting the original project.

Create a build agenthttps://docs.microsoft.com/en-us/learn/modules/host-build-agent/4-create-build-agent

Create an Ubuntu virtual machine on Azure to serve as your build agent. Create an agent pool in Microsoft Azure DevOps. Create an access token to authenticate your agent with Azure DevOps. Configure your agent with the software that's required to build the Space Game website. Configure your agent to connect to Azure DevOps so that it can receive build jobs. Verify that the agent is connected to Azure DevOps and ready to receive build jobs.

Create the agent pool Recall that an agent pool organizes build agents. In this section, you create the agent pool in Azure DevOps. Later, you'll specify the name of the agent pool when you configure your agent so that it can register itself to the correct pool.

In Azure DevOps, go to the Space Game - web - Agent project.

Select Project settings.

Under Pipelines, select Agent pools.

Locating Agent pools in the menu

Select Add pool.

In the Add pool window: https://docs.microsoft.com/en-us/learn/azure-devops/host-build-agent/media/4-project-settings-agent-pools.png

Under Pool to link, select New. Under Pool type, select Self-hosted. Under Name, enter MyAgentPool.

Create a personal access token For your build agent to register itself with Azure DevOps, you need a way for it to authenticate itself.

To do that, you create a personal access token. A personal access token, or PAT, is an alternative to a password. You can use the PAT to authenticate with services such as Azure DevOps.