How to create a multi-project pipeline in Gitlab Community Edition

A while ago my team wanted to finally have a fully integrated project pipeline that includes all of our automation tests. The problem that we faced was that the Java-Selenium testing codebase lives outside of the product application’s codebase.

With two entirely separate projects living on Gitlab there seemed to be no clear way to create the multi-project pipeline that we needed. We were getting value from the Selenium project by manually running the automated test suite from our local machines whenever a new change was pushed to our test environment. But after some time we gained confidence in our test suite and saw the value in having these tests join the existing unit and integration test as a part of an automated build pipeline. The only thing that we wanted to manually control was when we went into production. This stage remained a manual job. Based on this it seemed that the pipeline which we needed would have our application project (Project A) firing off a message to our Selenium project (Project B). This project would in turn run the regression suite and send the test result back to project A.

Having a vision of what needed to be down, I began my journey into finding a solution to the problem which we faced.

Angies Jones once said that, “If you’ve achieved something great, write about it!” and thankfully many people had done just that. Taking the advice from my peer, I began with the simplest implementation of the solution.

What this looked like was was just a simple curl from Project A to the Selenium one.

stages:
  - trigger_selenium_project

trigger:
  stage: trigger_selenium_project
  script:
    - apk update && apk add curl
    - curl -s 
      -X POST 
      -F token=$CI_JOB_TOKEN 
      -F ref=master 
      -F "variables[TRIGGERER_PIPELINE_ID]=${CI_PIPELINE_ID}" 
      https://gitlab.com/api/v4/projects/15700800/trigger/pipeline

Since this was a proof of concept, the stage on the other project was even more basic:

stages:
  - run_triggered_stage
  
run triggered stage:
  stage: run_triggered_stage
  script:
    - echo "Stage triggered from Project A."

With my Gitlab files created in either project I was ready for my first trial run. I pushed my build. The build ran. The build failed.

I checked the logs and found a strange message: Your browser sent an invalid request. But what could that mean? I went back and checked my project IDs and confirmed that my curl was correct. It was. So what could have caused this error?

After sifting through some more articles I had found that each of my Gitlab projects knew that I owned them and yet did not know that they could communicate with each other. This is where an access token came in handy. Thankfully, I found a blog post on zerotoprod.com which gave me a very clear indication of how to overcome this latest problem. I needed to create a unique ACCESS TOKEN and add it as a CI/CD variable in both of my Gitlab projects. With that step complete I was sure that this was definitely going to work.

I pushed my build. The build ran. The build passed. Yes!

Adding the access token to both projects was key to ensuring that they both knew that they have the same owner. I felt great to have solved all of my organization’s problems; and with such a with little effort. So I took my success story back to my team, awaiting the praise and adoration that I knew that I had deserved. I presented my work and awaited the responses. The first once was, “Wow, do your Selenium tests run that quickly?”

Darn. I had forgotten while I begun this in the first place.

Adding the access token to both projects was key to ensuring that they both knew that they have the same owner.

So I went back to the drawing board. The first thing to was send me test instructions from one project to the next. Maven made this easy by having simple commands, including the one that I needed:

variables:
  SELENIUM_TESTS: "mvn clean integration-test"

Adding this to curl was just as easy:

trigger:
  stage: trigger_selenium_project
  script:
    - apk update && apk add curl
    - curl -s 
      -X POST 
      -F token=$CI_JOB_TOKEN 
      -F ref=master 
      -F "variables[TRIGGERER_PIPELINE_ID]=${CI_PIPELINE_ID}" 
      -F "variables[SELENIUM_TESTS]=$SELENIUM_TESTS"
      https://gitlab.com/api/v4/projects/15700800/trigger/pipeline

Configuring the test project was a breeze as well:

stages:
  - run_selenium_tests
  
run selenium tests:
  stage: run_selenium_tests
  only:
    variables:
     - $SELENIUM_TESTS
  allow_failure: true
  script:
    - echo "Testing run started."
    - $SELENIUM_TESTS
    - echo "Testing run complete."

I used the ONLY tag because I did not want any future build or deploy stages to run every time that I needed to run the regression test pack.

The result?

I had done it, I thought. I had successfully created a multi-project pipeline on Gitlab. This meant that I could automatically run the selenium tests whenever there was a new build on the application project. This was amazing. However, there was a new problem now; how would Project A ever know the result of the tests in Project B?

What I had created was a simple pipeline that kicked off my regression test suite and did not care about the result. Fire and forget.

Since the goal was to have the UI tests as a part of Project A’s CI pipeline, it meant that there needed to be a way to check the test result of Project B before moving onto the next stage. Back to zerotoprod.com.

The suggestion here was to add two new stages. With a quick copy and paste Project B’s pipeline now had a new look.

stage: send_test_feedback
  only:
    variables:
      - $SELENIUM_TESTS
  allow_failure: false
  script:
    - echo "Send test feedback."
    - apk add curl jq
    - >
      GITLAB_MANUAL_JOB=$(curl --fail --header "PRIVATE-TOKEN: $GITLAB_API_KEY" "https://gitlab.com/api/v4/projects/15695625/pipelines/${TRIGGERER_PIPELINE_ID}/jobs" | jq '.[] | select (.name == "test_result") | .id');
      echo $GITLAB_MANUAL_JOB;
      curl -X POST --header "PRIVATE-TOKEN: $GITLAB_API_KEY" "https://gitlab.com/api/v4/projects/15695625/jobs/${GITLAB_MANUAL_JOB}/play" -F "variables[PROJ_B_PIPELINE_ID]=${CI_PIPELINE_ID}" | jq

Let’s break this stage down:

GITLAB_MANUAL_JOB=$(curl --fail --header "PRIVATE-TOKEN: $GITLAB_API_KEY" "https://gitlab.com/api/v4/projects/15695625/pipelines/${TRIGGERER_PIPELINE_ID}/jobs" | jq '.[] | select (.name == "  test_result") | .id');

This searches Project A for the latest pipeline with a stage named test_result and gets the ID of this stage and puts it into a variable named GITLAB_MANUAL_JOB.

echo $GITLAB_MANUAL_JOB;

This prints the ID of the stored in the variable named GITLAB_MANUAL_JOB.

curl -X POST --header "PRIVATE-TOKEN: $GITLAB_API_KEY" "https://gitlab.com/api/v4/projects/15695625/jobs/${GITLAB_MANUAL_JOB}/play" -F "variables[PROJ_B_PIPELINE_ID]=${CI_PIPELINE_ID}" | jq

Lastly, the line above triggers the tests_passed stage in Project A. Now let’s have a look at the new stage in Project B’s pipeline.

test_result:
  stage: test_result
  allow_failure: false
  when: manual
  script:
    - apk add curl jq
    - >
        GITLAB_TEST_JOB="$(curl -v -H "Content-Type: application/json" -H "PRIVATE-TOKEN: $GITLAB_API_KEY" https://gitlab.com//api/v4/projects/15700800/pipelines?per_page=1&page=1 )"; 
    - if [[ $GITLAB_TEST_JOB== *success* ]]; then echo "Tests passed"; else echo "Tests failed" && exit 1; fi

The first line in the script finds the latest job in Project B’s pipeline and puts the entire Gitlab Json message into a variable named GITLAB_TEST_JOB. The second line is a simple if statement that scans through the Json message and searches for the string success in it. Success is Gitlab’s indicator that a pipeline had passed.

Pro tip: Do not name the pipeline in your testing project super_successful_testing_pipeline. This will always return a true.

At last, I had my test project sending back a response to my application project.

Finally, I had figured out a way to have our application project kick off tests in a separate Selenium project and continue with the pipeline once the test have passed.

THE CODE

Project A

image: alpine

variables:
  PROJ_A_ID: 15695625
  SELENIUM_TESTS: "mvn clean integration-test"


stages:
  - compile
  - build
  - trigger_selenium_tests
  - test_result
  - trigger_deploy

compile:
  stage: compile
  script:
    - echo compile

build:
  stage: build
  script:
    - echo build

trigger_selenium_tests:
  stage: trigger_selenium_tests
  allow_failure: true
  script:
    - apk update && apk add curl
    - curl -s 
      -X POST 
      -F token=$CI_JOB_TOKEN 
      -F ref=master 
      -F "variables[TRIGGERER_PIPELINE_ID]=${CI_PIPELINE_ID}" 
      -F "variables[SELENIUM_TESTS]=$SELENIUM_TESTS" 
      -F "variables[PROJ_A_ID]=PROJ_A_ID" 
      https://gitlab.com/api/v4/projects/15700800/trigger/pipeline


test_result:
  stage: test_result
  allow_failure: false
  when: manual
  script:
    - apk add curl jq
    - >
        GITLAB_MANUAL_JOB="$(curl -v -H "Content-Type: application/json" -H "PRIVATE-TOKEN: $GITLAB_API_KEY" https://gitlab.com/api/v4/projects/15700800/pipelines?per_page=1&page=1 )";
    - echo $GITLAB_MANUAL_JOB;    
    - if [[ $GITLAB_MANUAL_JOB  == *success* ]]; then echo "Tests passed"; else echo "Tests failed" && exit 1; fi
    
    
trigger_deploy:
  stage: trigger_deploy
  script:
    - echo "Put steps to deploy code to the next environment here.";

Project B

cache:
  paths:
    - maven.repository/

stages:
  - run_selenium_tests
  - send_test_feedback
  
run selenium tests:
  stage: run_selenium_tests
  image: markhobson/maven-chrome
  only:
    variables:
     - $SELENIUM_TESTS
  allow_failure: true
  script:
    - echo "Testing run started."
    - $SELENIUM_TESTS
  after_script:
    - echo "Testing run complete."

send test feedback:
  stage: send_test_feedback
  image: alpine
  only:
    variables:
      - $SELENIUM_TESTS
  allow_failure: false
  script:
    - echo "Send test feedback."
    - apk add curl jq
    - >
      GITLAB_MANUAL_JOB=$(curl --fail --header "PRIVATE-TOKEN: $GITLAB_API_KEY" "https://gitlab.com/api/v4/projects/15695625/pipelines/${TRIGGERER_PIPELINE_ID}/jobs" | jq '.[] | select (.name == "test_result") | .id');
      echo $GITLAB_MANUAL_JOB;
      curl -X POST --header "PRIVATE-TOKEN: $GITLAB_API_KEY" "https://gitlab.com/api/v4/projects/15695625/jobs/${GITLAB_MANUAL_JOB}/play" -F "variables[PROJ_B_PIPELINE_ID]=${CI_PIPELINE_ID}" | jq