What is Azure Pipelines?

Azure Pipelines, a core component of Azure DevOps, automates the process of building, testing, and deploying code. It integrates continuous integration, continuous testing, and continuous delivery to streamline the software delivery lifecycle. With support for all major programming languages and project types, Azure Pipelines ensures your code is efficiently built, tested, and deployed to any environment.        

Benefits of Azure Pipelines

Azure Pipelines offers a fast, reliable, and secure way to automate the build process. It ensures consistent, high-quality code that is ready for deployment.

Key benefits include:

• Multi-language and Cross-platform Support: Compatible with any programming language and platform.
• Flexible Deployment: Simultaneously deploys to multiple targets.
• Seamless Azure Integration: Easily integrates with Azure services for streamlined deployments.
• Multi-OS Builds: Supports builds on Windows, Linux, and macOS machines.
• GitHub Integration: Natively connects with GitHub for version control and workflows.
• Open-Source Friendly: Fully supports open-source projects.

Getting Started with Azure Pipelines

To set up continuous integration and continuous delivery (CI/CD) for your code, sign up for an Azure DevOps organization and Azure Pipelines. This enables you to manage high-performance pipelines for seamless deployments.

For more details, see What is Azure Pipelines?

You can sign up using either a Microsoft account or a GitHub account.

Sign Up with a Microsoft Account

Follow these steps to sign up for Azure DevOps with a Microsoft account:

• Update Your Account: Please make sure your Microsoft account is active by signing in.
• Access Azure Pipelines: Go to Azure Pipelines and select Start Free, then sign in with your Microsoft account.

• Begin Setup: Click Continue to proceed with Azure Pipelines.

• Create Your Organization: Enter a name for your organization. Choose a host location from the drop-down menu. Enter the CAPTCHA characters displayed. Select Continue to complete the setup.

• Access Your Organization: Use the following URL anytime to sign in: https://dev.azure.com/{yourorganization}

You'll then be prompted to create your first project and start building pipelines.

Creating a Project in Azure Pipelines

You can create either a public or private project based on your needs. For more details about public projects, see What is a public project?

Follow these steps to create a project:

• Enter Project Details: Provide a project name. Choose the visibility (public or private). Optionally, add a project description.

• Create the Project: Click Create Project to proceed.

Note:

• Project names cannot contain special characters such as / : \ ~ & % ; @ ' " ? < > | # $ * } { , + = [ ].
• The name cannot start with an underscore or a period, end with a period, and must be 64 characters or fewer.

Visibility Options:

• Public: Anyone on the internet can view the project.
• Private: Only users you grant access can view and collaborate.

Once the project is created, you'll be prompted to select the services you want to use.

You're now ready to create your first pipeline or invite others to collaborate on your project.

Inviting Team Members (Optional)

You can invite others to collaborate on your project by adding their email addresses to your organization and project.

To invite team members, follow these steps:

• Access Organization Settings:

• Complete the Invitation Form:
• Complete the Invitation: Once you've filled out the details, select Add to send the invitation.

Create Your First Pipeline: Get the Python Sample Code

To begin, fork the following repository into your GitHub account.

https://github.com/Microsoft/python-sample-vscode-flask-tutorial

Create Your First Python Pipeline

Follow these steps to set up your first Python pipeline in Azure Pipelines:

1. Access Your Azure DevOps Project:

• Sign in to your Azure DevOps organization and navigate to your project.
• Go to Pipelines and select New Pipeline or Create Pipeline if it's your first one.

2. Connect Your GitHub Repository:

• When prompted, select GitHub as the source location.
• You may be redirected to GitHub for authentication—enter your GitHub credentials if required.
• From the list of repositories, select your desired repository.
• If prompted, approve the installation of the Azure Pipelines app by selecting Approve & Install.

3. Configure the Pipeline:

• Azure Pipelines will automatically analyze your repository and suggest the Python package pipeline template.
• Once the pipeline YAML appears, review the configuration.
• When ready, select Save and run.

4. Commit the YAML File:

• You'll be prompted to commit a new azure-pipelines.yml file to your repository.
• After reviewing the commit message, click Save and run again.

5. Monitor Your Pipeline:

• To watch the pipeline execution, select the build job.
• This pipeline was automatically generated based on the Python package template detected in your code.

6. Customize Your Pipeline:

• You now have a working YAML pipeline (azure-pipelines.yml) in your repository, ready for customization.
• To make changes, go to the Pipelines page, select your pipeline, and choose Edit.

View and Manage Your Pipelines

• To view and manage your pipelines, navigate to the Pipelines section from the left-hand menu. This will take you to the pipeline's landing page.

• From the pipelines landing page you can view pipelines and pipeline runs, create and import pipelines, manage security, and drill down into pipeline and run details.

Choose Recent to view recently run pipelines (the default view), or choose All to view all pipelines.

Select Runs to view all pipeline runs. You can optionally filter the displayed runs.

• Select a pipeline run to view information about that run: You can choose to Retain or Delete a run from the context menu

View pipeline details

The details page for a pipeline allows you to view and manage that pipeline.

Choose Edit to edit your pipeline. For more information, see the YAML pipeline editor. You can also edit your pipeline by modifying the azure-pipelines.yml file directly in the repository that hosts the pipeline.

Viewing Pipeline Run Details

The pipeline run summary provides real-time and completed run statuses, allowing you to track progress and outcomes effectively.

Key features of the summary pane include:

• Run Status: Monitor the pipeline's status during execution and after completion.
• Job and Stage Details: View detailed insights into each stage and job.
• Artifact Downloads: Easily download generated artifacts.
• Linked Resources: Access associated commits, test results, and work items.

Jobs and Stages: The Jobs pane offers an overview of your pipeline's stages and job statuses. Depending on your pipeline configuration, the pane may display multiple tabs for stages and jobs.

For example, a pipeline with Build and Deploy stages will show each stage's progress. You can explore individual pipeline steps by selecting the corresponding job from the Stages or Jobs pane.

Choose a job to see the steps for that job.

From the steps view, you can review the status and details of each step. From the More actions. you can toggle timestamps or view a raw log of all steps in the pipeline.

Cancel or Re-run a Pipeline

• Cancel a Running Pipeline: If the pipeline is currently running, select Cancel to stop the execution.
• Re-run a Completed Pipeline: Once the run is complete, choose Run New to trigger the pipeline again.

Pipeline Run: More Actions Menu

The More Actions menu offers several management options for your pipeline runs, including:

• Download Logs: Retrieve detailed logs for troubleshooting.
• Add Tags: Organize runs by adding relevant tags.
• Edit Pipeline: Modify the pipeline configuration.
• Delete Run: Remove the pipeline run if no longer needed.
• Configure Retention: Set retention policies for the run.

Add a Status Badge to Your Repository

Displaying a status badge in your repository helps showcase your commitment to code quality by highlighting the success of your Azure Pipeline.

Copy the Status Badge:

1. In Azure Pipelines, go to the Pipelines page and select the pipeline you created.
2. Click the More Actions menu (···) and choose the Status badge.
3. In the Sample Markdown section, copy the provided Markdown code.

Add the Badge to GitHub:

1. In your GitHub repository, navigate to the list of files and open Readme.md.
2. Click the pencil icon to edit the file.
3. Paste the status badge Markdown at the beginning of the file.
4. Commit the change to the main branch.

The status badge will now appear in your repository's description.

Enable Badge Access for Private Projects (Optional):

1. In Azure DevOps, go to Project Settings (bottom left corner).
2. Under Pipelines, open the Settings tab.
3. Toggle off the Disable anonymous access to badges option under General to allow public visibility.

Because you just changed the Readme.md file in this repository, Azure Pipelines automatically builds your code, according to the configuration in the azure-pipelines.yml file at the root of your repository. Back in Azure Pipelines, observe that a new run appears. Each time you make an edit, Azure Pipelines starts a new run.

Customize your pipeline

Understanding the azure-pipelines.yml File

The pipeline configuration is defined using a YAML file, typically named azure-pipelines.yml, located at the root of your repository. This file outlines the steps for building, testing, and deploying your code.

To access and edit the pipeline YAML:

1. Go to the Pipelines page in Azure Pipelines.
2. Select the pipeline you created.
3. Open the More actions menu (···) and choose Edit to access the YAML editor.

Pipeline Execution Overview

This pipeline automatically triggers whenever your team pushes changes to the main branch or creates a pull request. It runs on a Microsoft-hosted Linux machine and includes a single step: executing the Maven task.

Change the Build Platform

You can choose to build your project using:

• Microsoft-hosted agents: Pre-configured with SDKs and development tools.
• Self-hosted agents: Custom environments with specific tools tailored to your project needs.

Change the platform to build on

1. Go to the Pipelines page and select your pipeline.
2. Click Edit from the pipeline’s main page or the build summary.

By default, the pipeline runs on a Linux agent, but you can modify the agent pool in the YAML file as needed.

To choose a different platform like Windows or Mac, change the vmImage value:

Add Steps to Your Pipeline

Enhance your pipeline by adding additional scripts or tasks. A task is a pre-configured script designed for specific actions like building, testing, publishing, or deploying your application.

For Java projects, the Maven task handles testing and result publishing. You can also include tasks to publish code coverage reports or perform other operations.

To add steps:

1. Open the YAML editor for your pipeline.
2. Append the following code snippet at the end of your azure-pipelines.yml file.

Build Across Multiple Platforms

You can build and test your project on multiple platforms using the strategy and matrix features. These allow you to define multiple configurations for your pipeline runs. Additionally, you can use variables to pass data throughout the pipeline conveniently.

In this example, we'll use a variable to specify the desired image for the build.

To configure multi-platform builds:

1. Open your azure-pipelines.yml file.
2. Replace the existing content with the following configuration:

with the following content:

• Select Save and then confirm the changes to see your build run up to three jobs on three different platforms.

Each agent can run only one job at a time. To run multiple jobs in parallel you must configure multiple agents. You also need sufficient parallel jobs.

Build Using Multiple Versions

To build your project with different language versions, you can define a matrix of versions along with a variable. This approach allows you to either:

• Build with multiple Java versions on a single platform, or
• Run different Java versions across multiple platforms.

To build with multiple versions:

1. Open your azure-pipelines.yml file.
2. Add the following matrix configuration before the Maven task and after the vmImage definition:

Then replace this line in your maven task:

with this line:

• Make sure to change the $(imageName) variable back to the platform of your choice.
• If you want to build on multiple platforms and versions, replace the entire content in your azure-pipelines.yml file before the publishing task with the following snippet:

• Select Save and then confirm the changes to see your build run two jobs on two different platforms and SDKs.

Customize CI Triggers

Pipeline triggers automatically initiate a pipeline run based on specific events. For example, you can use the trigger keyword to initiate a pipeline whenever updates are pushed to a branch.

By default, YAML pipelines include a CI trigger for the default branch (usually main). You can further customize triggers for:

• Specific branches
• Pull request (PR) validation

To enable PR validation, replace the trigger: step with pr:, as shown in the examples below. This ensures the pipeline runs for each PR update.

To configure triggers: Add one of the following code snippets at the beginning of your azure-pipelines.yml file.

You can specify the full name of the branch (for example, main) or a prefix-matching wildcard (for example, releases/*).

Pipeline Settings

You can view and configure pipeline settings from the More Actions menu on the pipeline details page.

Here’s what you can manage:

• Manage Security: Control user access and permissions for the pipeline.
• Rename/Move: Edit the pipeline name or change its folder location.

• Status Badge: Add a status badge to your repository to display the build status.
• Delete: Permanently remove the pipeline along with all associated builds and artifacts.
• Scheduled Runs: Set up and manage recurring pipeline runs.

Pipeline Settings Configuration

From the Pipeline Settings pane, you can configure the following options:

🔄 Processing of New Run Requests

Control how your pipeline handles new run requests:

• Enabled (Default): Allows all trigger types, including manual runs, to start immediately.
• Paused: Run requests are queued but not executed. When re-enabled, runs start in queue order.
• Disabled: This prevents new runs and disables all triggers. Build policies that are disabled show an "Unable to queue Build" message in PR overviews, marking them as broken.

📄 YAML File Path

Specify an alternative path for your pipeline's YAML file. This is useful if you move or rename the YAML file.

🔗 Automatically Link Work Items

Link work items to pipeline runs based on associated changes.

• When enabled, specify a target branch or use it * to link work items for all branches (default).

Create Work Item on Failure

YAML pipelines don’t include a built-in Create work item on failure option like classic build pipelines. This is because YAML pipelines can have multiple stages, making a pipeline-level setting less practical.

To implement this functionality in a YAML pipeline, you can use one of the following methods:

• REST API: Utilize the Work Items - Create API call.
• Azure CLI: Use the az boards work-item create command at the desired stage of the pipeline.

Example Workflow:

In the following example, two jobs are defined:

1. Primary Job: Executes the main pipeline tasks.
2. Failure Handling Job: If the primary job fails, this job runs and automatically creates a bug in the same Azure DevOps project.

Build a YAML Pipeline with Stages

Using multiple stages in your YAML pipeline helps isolate different parts of the process, enhancing quality control, security, and visibility. It also reduces risk by ensuring each stage runs independently.

In this guide, you'll learn how to create a more advanced YAML pipeline with multiple stages and conditions. The example pipeline includes:

• Build Stage: Compiles the code and generates artifacts.
• Test Stage: Run automated tests to ensure code quality.
• Deploy Stage: Deploys the application to the target environment.
• Optional Stages: Alternate deployments and rollback options for quick recovery if issues arise.

This approach improves pipeline reliability and stability. While the example provided works for most cases, you can further customize it to meet specific language or platform requirements.

1. Create a Starter Pipeline

• In your Azure DevOps project, navigate to Pipelines and select Create Pipeline.
• Choose GitHub as the source location for your code.
• On the Select a Repository screen, pick your desired repository.
• On the Configure your pipeline screen, select Starter pipeline.
• Save the pipeline.

2. Add the Build Stage

The starter pipeline is just a placeholder. To create a functional pipeline, you'll need to replace the default code with a new YAML configuration that defines a Build stage.

• Open the pipeline editor.
• Remove all the existing code.
• Replace it with the following YAML, which includes a Build stage.

3. Add the Test Stage

The Test stage runs tests on your project and publishes the results to Azure DevOps. For more details, refer to the Publish Test Results task.

This stage will only execute if the Build stage completes successfully and cannot be skipped.

Add the following configuration after the Build stage in your pipeline to include the Test stage.

The Test stage includes an attribute that sets Skippable to false. When you set isSkippable to false, a stage can't be skipped. The option to skip the stage is also disabled in the Azure DevOps UI.

4. Deploy to Staging

Add the DeployToStaging stage after the Test stage in your pipeline.

This stage depends on the successful completion of the Test stage and includes two distinct jobs:

1. DeployStagingJob: This job handles the deployment of your application to staging resources, such as a staging server.
2. DeployStagingJobWithValidation: This job focuses on validating the staging deployment and requires manual approval.

The DeployStagingJobWithValidation job includes a Manual Validation task, which pauses the pipeline until a user manually approves the stage. This ensures that changes undergo thorough review before progressing, enhancing security and mitigating potential risks.

Note that the manual approval process runs on the server pool, as manual validations only execute in this environment.

5. Deploy to Production

Add the DeployToProduction stage after the DeployToStaging stage in your pipeline.

In this stage, the application is deployed to production only if the DeployToStaging stage completes successfully and the source branch is either main or release.

The DeployToProduction stage includes two key jobs:

1. DeployProductionJob: This job handles the deployment of your application to the production environment.
2. wait for validation: This job introduces an additional manual approval step for enhanced security and quality control.

The Manual Validation task in the waitForValidation job ensures that a second human review occurs before the deployment proceeds, similar to the validation process used in the DeployToStaging stage.

View the entire YAML pipeline

Here's the entire pipeline with all of the referenced stages.

➡ 𝐹𝑜𝑙𝑙𝑜𝑤 Ankit Ranjan Ankit Ranjan (DevOps Engineer) 𝑓𝑜𝑟 𝑚𝑜𝑟𝑒 Azure 𝑡𝑖𝑝𝑠 and subscribe to my Azure Newsletter: