GitHub actions — CI with proprietary / licensed framework— ArcGIS pro as example

Abstract

While developing a solution that uses ESRI ArcGIS pro solution, I have to find a way to run

• Business logic tests
• Regression
• Integration
• CI

With the help of windows docker containers and CoreHost approach, build framework dependent integration tests that run on pull requests, merges, and other trigger events automatically on the CI pipeline. In this essay, I describe the approach.

The problem

To run the following code snippet, one needs to

var queryPolygonMercator = GeometryEngine.Instance.Project(queryPolygon, SpatialReferences.WebMercator);

• Have ArcGIS pro installed on the host
• Have a valid license configured
• Must run as a standalone application, no option to run AddIn in a test runner

The above is valid would you want to build NuGet packages for internal code reuse.

Doing automated tests not practical this way. I will not repeat what is now a conventional wisdom

• Martin Fowler
• Roy Osherov
• Michael Feathers

The authors discuss unit testing. Yet, in my case, integration and regression are the subjects.

Testing pipeline

CI with no human interaction

Code reuse and granularity using NuGet packages

Solution outline

• Create a container image that satisfies ESRI prerequisites
• Avoid the need to reference DLLs

o ArcGIS.CoreHost.dll

o ArcGIS.Core.dll

• Ease up the Initialization syntax [STAThread]

Side note

As a general rule, one would separate the code which, uses the framework (ESRI) from the one which is not. A different article will follow.

I am using GitHub actions as a CI — yet any other solution will be similar.

Docker image to build and test ArcGIS pro code

The result will have

1. ArcGIS pro installation
2. Full .net 4.8 SDK framework

One would need a full windows based container image to start, yet the .net framework Dockerfile Is based on widows server core.

Thus create a clean image.

File similar to this

2. To take advantage of multi-stage build, I created a separate image only for installation assets. Use of volume would be possible — but not on GitHub actions

3. Here is the Dockerfile that creates the image that can run our ArcGIS pro code.

https://gist.github.com/dkuida/7bfa46e3dbb3ca9f438a10f7b0f00085#file-arcbase-dockerfile

• Licensing — one could use named or single license- yet then you say goodbye to it for other uses
• Layers of container yet need to be optimized- to reduce volume size

4. The last is an executer image that will execute arbitrary script mounted into it

https://gist.github.com/dkuida/7bfa46e3dbb3ca9f438a10f7b0f00085#file-executingcontainer-dockerfile

At this point, there is an image available to run your code. Let me show how.

Putting your docker image to use

I created a GitHub action that (might share it on a different occasion)

• Mounts source code to under test into c:/workspace

Executes the PowerShell script that does the magic

https://gist.github.com/dkuida/7bfa46e3dbb3ca9f438a10f7b0f00085#file-deploy-ps1

That alone allows one to compile and deploy NuGet package.

Reminder — ESRI require to

1. Host.Initialize before any ArcGIS code

very slow — and since xUnit creates new instance before every test — is a nightmare to do. (ClassFixture is an option though )

2. [STAThread] — will kill your parallel run — and forces to use [StaFact]

3. Reference DLL with copy local — will break code reuse and sharing via NuGet.

To say the least, it is not productive

Let’s address one by one

STAThread

There is a lot of background conversation on the matter

• The problem
• One option
• Or just the most basic implementation

ArcGIS.*.dll dependencies and Host.Initialize

Based on an example provided by ESRI community, which shows how to resolve dll in runtime.

I would want to Initialize the Core — thus resolve the DLL dependencies beforehand

// call in constructor
m_Resolver = new CoreHostResolver(m_Logger, 50);
// validate initialization where needed before ArcGIS core
var isSuccess = await m_Resolver.InitialisedTask;

Here is the implementation

https://gist.github.com/dkuida/7bfa46e3dbb3ca9f438a10f7b0f00085#file-corehostresolver-cs

You would need to reference the actual DLL. Make sure that you set the following in project file

• not copy it to the build ( and as a result to the NuGet )
• not depend on a particular version

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <TargetFrameworks>net48;netstandard2.0</TargetFrameworks>
        <RepositoryUrl></RepositoryUrl>
        <Configurations>Release;Debug</Configurations>
        <Platforms>x64</Platforms>
        <LangVersion>8</LangVersion>
        <PackageId>Inf.ArcGis.Threader</PackageId>
        <Authors></Authors>
        <PackageTags>ArcGisPro, Core, Threading</PackageTags>
        <PackageVersion>0.0.1</PackageVersion>
        <Version>0.0.1</Version>
    </PropertyGroup>

    <ItemGroup>
      <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="3.1.5" />
    </ItemGroup>

    <ItemGroup Condition="'$(TargetFramework.TrimEnd(`0123456789`))' != 'net'">
        <PackageReference Include="Microsoft.Win32.Registry" Version="4.7.0" />
    </ItemGroup>

    <ItemGroup>
      <Reference Include="ArcGIS.Core,  Culture=neutral, PublicKeyToken=8fc3cc631e44ad86" >
        <Private>False</Private>
        <SpecificVersion>False</SpecificVersion>
<HintPath>C:\Program Files\ArcGIS\Pro\bin\ArcGIS.Core.dll</HintPath>
        
      </Reference>
      <Reference Include="ArcGIS.CoreHost, Culture=neutral, PublicKeyToken=8fc3cc631e44ad86" >
        <Private>False</Private>
        <SpecificVersion>False</SpecificVersion>
        <HintPath>C:\Program Files\ArcGIS\Pro\bin\ArcGIS.CoreHost.dll</HintPath>
      </Reference>
    </ItemGroup>


</Project>

Use our executer container to build and publish the NuGet package

Now you can use it in

• tests
• standalone CoreHost applications
• Worker / Hosted services
• .net ASP.NET Core services ( while communicating using gRPC with a desktop or other components )

Test

[Fact]
public async Task GivenSpatialQueryRequest__WhenQueried_ReturnsFieldsAndGeometry()
{
 var resolved = await m_CoreHostResolver.InitialisedTask;
 if (!resolved)
 {
  m_Logger.LogCritical("didnt find arcGIS");
  Assert.False(true);
  return;
 }

// not real or valid coordinates
 var coordinates = new[]
 {
  new Coordinate2D(1, 1), new Coordinate2D(1, 2),
  new Coordinate2D(2, 2), new Coordinate2D(2, 1)
 };
var spatialReference = SpatialReferenceBuilder.CreateSpatialReference(3857);
 var queryPolygon = PolygonBuilder.CreatePolygon(coordinates,
  spatialReference);
// your SUT code with validation
// assertions

}

Summary

Using a layered approach, we achieved a code that can be

• tested in CI
• potentially — code break down based on a business domain into separate independent packages
• deployment pipeline with automated build and validation
• can run using GitHub actions ( with no need for dedicated CI to maintain )
• a step forward moving away from .net framework — 4.8 is the last version that ceases to be supported by Microsoft in July 2021 ( another article will follow)