Tag Archives: continuous integration

Build Agent Infrastructure Testing in GoCD

In this post I would like to describe a simple technique for reducing the waiting time and stress related to build agent environment volatility when using Continuous Integration / Continuous Delivery tools like GoCD, via infrastructure testing.

The Problem

Given a modern CI server, such as GoCD, and a set of dedicated build machines (agents), it is possible to improve software development agility. Automated build/test/deploy pipelines, built to reflect the value stream, bring transparency and focus into the software delivery activities.

CI automation is software itself, and is thus susceptible to errors. Configuration management can optimize the set up of the environment, in which the build agents run. However, when computational resources are added to a CI infrastructure, i.e. to parallelize the build and thus reduce feedback times, a missing environment dependency can cause stress and pain that CI is trying to eliminate.

Consider a pipeline where a complete cycle (i.e. with slow integration tests followed by a reporting step at the end or long check-outs in the beginning) takes a significant amount of time. If one of the last tasks fails due to a configuration or an environment issue, the whole stage fails. The computational resources have been wasted just to find out that a compiler is missing. This can easily happen when there is variation in the capabilities of the build agents.

Improvement idea: fail fast! Don’t wait for environment or infrastructure mismatches

GoCD Resources as Requirements and Capabilities

If a step in a build pipeline requires a certain compiler or a particular environment, this can be conveniently expressed in the configuration of GoCD as a build agent resource. A resource can be seen as a requirement of a pipeline step that is fulfilled by a corresponding capability of a build agent.

Consider the following set-up with two build agents – one running on Windows, another one on Linux. Some tasks could be completely platform-independent, such as text processing, and thus could be potentially performed on any machine with a required interpreter installed.


Build Agent is the Culprit

We have set up our environment, and have successfully tested our first commit, but the second one fails:


The code is the same, why did the second build fail? The builds ran on different agents, but I expected them to behave similarly…


Oh, that’s embarrassing. While yes, the script is platform-independent, there’s no executable named python2 in my Windows runner environment path.

With a one-file repository and a simple print statement this failure did not cause much damage, but as mentioned earlier, real life builds failing due to a missing executable might be costly.

Unhappy Picture


Infrastructure Test Pipeline

In order to fail fast in situations where new agents are added to a CI infrastructure, or their environment is volatile, I propose to use a single independent pipeline that checks the assumptions that longer builds depend upon.

If a build step requires python in the path, there should be a test for it that gives this feedback in seconds without much additional waiting time. This can be as easy as calling python --version, which will fail with a non-zero return code if the binary is missing. More fine-grained assertions are possible, but should still remain fast.

If a certain binary should not be in the path, this can be asserted as well. The same goes for environment variables and file existence. Dedicated infrastructure testing tools, such as Serverspec could also be used, but having a response time under a minute is crucial in my view.

Run on All Agents

In order to validate the consistency of the CI infrastructure, the validation tasks should run on all agents that advertise a corresponding resource. This is where, in my view, the real power of GoCD comes to light, and the concepts used in it fit in the right places.

GoCD will run the test tasks on all agents that fulfill all resource requirements for the task.


Test fails

Now that we have all the tests, running them gives quick and precise feedback:


Checking out the job run details reveals the offending agent. Note the test duration: under 1 second.


Fixing the Infrastructure


Whatever the resolution of the infrastructure problem, when the infrastructure test has a good coverage of the prerequisites for a pipeline, adding new agents to the CI infrastructure should become as much fun as TDD is: write an infrastructure test, see it fail, fix the infrastructure, feel the good hormones. Add new build agents for speed — still works — great!


Note how the resources that are available only on one machine are only run on one corresponding machine.


Happy Pictures and Developers

happy pipeline

When to Test

It is an open question, when to test the infrastructure. With the system being composed of the CI server and agents, the tests should probably run on any global state change, such as

  • added/removed/reconfigured agents
  • automatic OS updates (controversial)
  • restarts
  • network topology changes

It is also possible to schedule a regular environment check. Having the environment test pipeline be the input for other pipelines unfortunately will not do in the following sequence of events:

  • environment tests pass
  • faulty agents are added
  • downstream pipeline is triggered
  • environment failure causes a pipeline to fail

In any case, there is a REST API available for the GoCD server should automating the automation become a necessity.


I would like to thank all the great minds, authors and developers who have worked and are working to make lives of developers and software users better. Tools and ideas that work and provide value are indispensable.  The articles and the software linked in this blog entry are examples of knowledge that brings the software industry forward. I am also very grateful to my current employer for letting me learn, grow and make a positive impact.



Setting travis-ci with github for a c++ project for the first time


Here, I’m trying to use travis-ci, c++, github, CATCH, premake together with my undoredo-cpp library to reduce entropy, try out continuous integration and behavior-style tests.

As a “one-man show” programmer at least at home, I’m trying to keep the discipline of writing tests first. “Growing Object-Oriented Software Guided by Tests” is perhaps a good, although a comparatively dry book for those who are not yet convinced. The blog post by Phil Nash about his latest version of the c++ single-header testing framework CATCH, moved me to finally get my hands on the free continuous integration service travis-ci, along with CATCH with a goal to rewrite the tests for my undo-redo c++ adventure in a more behavior-driven-style.

The undo-redo library is already there, and the tests as well – in gtest (see the master branch). I’d label them “explorative” at the moment since there are just too many assertions per test case, which means I’m repeating myself.

Starting continuous integration at travis-ci for c++

To start my CATCH-“BDD” exploration I’ve setup the branch first: https://github.com/d-led/undoredo-cpp/tree/catchmoci. At my landing page the project is switched on for catching the commit hooks:


The following configuration file .travis.yml is placed for travis-ci to know what to do with my non-conforming repo:

language: cpp

   - catchmoci

before_script: ls

  - make -C Build

As in all TDD practice, the build fails due to the reason that the build doesn’t work yet at all. Adding a status image to my README shines:


Fixing the build

The makefiles are created using premake4, which is a single-file makefile generator based on lua. Unfortunately, I couldn’t force the CI-virtual machine execute my binary premake4, so I had to add the generated makefiles. Now that the make process works, the tests still don’t compile:


Fixing the tests

Once the bulk of the assertions have been rewritten for CATCH, the build still failed due to an ambiguity in serializing std::nullptr_t. Fortunately, Phil has thought of (or rather tested) that, and has a macro which can be defined for the build, fixing it: CATCH_CONFIG_CPP11_NULLPTR.

Voila, travis-ci vm is happy:


Just checking locally if test reporting is fine by adding a spurious test temporarily:


The test-rewrite has been successful and all pass, the badge is green and I can go to bed


But! It’s not the end of the story! BDD! Mock-objects!