Selenium UI tests in an Azure DevOps pipeline

Overview

Running Selenium UI tests in an Azure DevOps pipeline is challenging part of Test automation process. This guide describes how to run Selenium UI tests in CI-CD using Azure DevOps. The following are different steps required to do so. For complete understanding of the overall prcoess of CICD, do visit this tutorial.

Prerequisites

1. Project/Website, Web application developed in .Net
2. Test Project, Test Project contains selenium tests to test the web application
3. Release pipeline, Release pipeline, to build, deploy and test the application

Note: Unit and integration tests are usually run in a build process, preventing the deployment if the tests fail.

UI tests are run after deployment on stagging or QA server and before deployment on the production server.

Selenium UI tests in an Azure DevOps pipeline

Following are the steps you need to perform to integrate selenium script in CI/CD.

Push Web Application along with Test project to git:

The first step is to make sure that your project code as well as test project is uploaded to git repo, your test project is a class library project of project to be tested. This is how your project will look like in Visual Studio:

On Git it will look like

Setting UP CI

We need the following task to setup CI, this CI will build both the projects.

trigger:
  branches:
     include:
       - QA
pool:
  vmImage: 'windows-latest'
variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

To build your project that is .NET Core

To build your test project

- task: DotNetCoreCLI@2
  inputs:
    command: 'build'
    projects: '**/Portal.tests.csproj'
    arguments: '--configuration $(BuildConfiguration) --output $(build.artifactstagingdirectory)'

.NET Core, To publish your test project

- task: DotNetCoreCLI@2
  inputs:
    command: 'publish'
    publishWebProjects: false
    projects: '**/Portal.tests.csproj'

Setting up CD

Our UI tests will run as part of a release pipeline after deployment, my CD looks like

The artifacts you will get from the CI, in stages we have two sections: one is responsible for code deployment, and, the 2^nd section is responsible for executing UI tests. Let’s see the steps of 2^nd section it looks like this:

It contains two task first Visual Studio Test

steps:
- task: VSTest@2
  displayName: 'Test Assemblies '
  inputs:
    testAssemblyVer2: |
     **\* tests*.dll
     !**\*TestAdapter.dll
     !**\obj\**
    uiTests: true
    distributionBatchType: basedOnAssembly
  continueOnError: true

Secondly it contains the Publish Test Results

steps:
- task: PublishTestResults@2
  displayName: 'Publish Test Results **/TEST-*.xml'
  inputs:
    testResultsFormat: NUnit
    mergeTestResults: true   continueOnError: true

Report

once you trigger the pipeline, it will build, deploy, and execute the tests and you can see the result in the report as shown below.

Summary | Selenium UI tests in an Azure DevOps pipeline

Integration of Selenium UI tests in an Azure DevOps pipeline is a bit challenging but it can be done easily if proper steps are followed. The document briefly describes all the required action to set up UI tests in CI-CD.