Category Archives: Programming

C++ version of ruby’s Integer::times via user-defined literals

Motivation

It occurred to me that I was missing Ruby’s 42.times do ... in C++. While a loop is still imperative, the inspiration to write the folowing code and the post came after watching the talk “Declarative Thinking, Declarative Practice” by Kevlin Henney. Thus, let it be a “declarative imperative”. While there’s no explicit intended use for the following code, it could make some tests read a bit less verbose. An implementation should be dependency-free, minimalist and copy-paste-able.

42_times([]{
    do_something();
});

First version

Luckily, C++11/C++14 feels like a new language[1. the original quote by Bjarne Stroustrup is about C++11] and has a nice feature that allows to fulfill wishes: user-defined literals. Thus, having settled on the implementation strategy, here’s how “.times” can look like in C++ as of today:

#include <iostream>

struct execute {
    const unsigned long long n;

    template<typename Callable>
    void operator() (Callable what) {
        for (auto i = 0; i < n; i++)
            what();
    }
};

execute operator"" _times(unsigned long long n) {
    return execute{n};
}

int main() {
    3_times([]{
        std::cout << "bla" << std::endl;
    });

    auto twice = 2_times;

    twice([]{
        std::cout << "blup" << std::endl;
    });
}

g++ -std=c++14 example.cpp & ./a.out

bla
bla
bla
blup
blup

I’d guess, this idea occurred to a number of people already

Update 1: loop index

@Ideone

With two simple options: with a counter and without the counter available in the closure:

#include <iostream>
#include <functional>

struct execute {
    const unsigned long long n;

    void operator() (std::function<void()> what) {
        for (auto i = 0; i < n; i++)
            what();
    }

    void operator() (std::function<void(unsigned long long)> what) {
        for (auto i = 0; i < n; i++)
            what(i);
    }
};

execute operator"" _times(unsigned long long n) {
    return execute{n};
}

int main() {
    3_times([]{
        std::cout << "bla" << std::endl;
    });

    auto twice = 2_times;

    twice([]{
        std::cout << "blup" << std::endl;
    });

    3_times([](unsigned long long i) {
        std::cout << "counting: " << i << std::endl;
    });
}

bla
bla
bla
blup
blup
counting: 0
counting: 1
counting: 2

Update 2: benchmark

A simple benchmark attempt shows that the modern C++ compilers can optimize a for loop that doesn’t produce side effects better than the code above. However, not all code needs to sacrifice readability for performance. If the loop is not on the critical path, it probably doesn’t need to be optimized, especially, if it’s just a test.

After some tinkering to disable optimizations for the function call, here’s a hayai-based benchmark:

struct TimesVsLoop : public ::hayai::Fixture{
    static int v;

    static void do_something() {
        v = v % 41 + 1;
    }

    static void do_something(unsigned long long i) {
        v = (i + v) % 41 + 1;
    }
};

int TimesVsLoop::v = 0;

BENCHMARK_F(TimesVsLoop, Times_Without_Index, 10, 100)
{
    1000000_times([]{
        do_something();
    });
}

BENCHMARK_F(TimesVsLoop, Loop_Without_Index, 10, 100)
{
    for (int i = 0; i < 1000000; i++) {
       do_something();
    }
}

BENCHMARK_F(TimesVsLoop, Times_With_Index, 10, 100)
{
    1000000_times([](unsigned long long i){
        do_something(i);
    });
}

BENCHMARK_F(TimesVsLoop, Loop_With_Index, 10, 100)
{
    for (int i = 0; i < 1000000; i++) {
       do_something(i);
    }
}

int main()
{
    hayai::ConsoleOutputter consoleOutputter;

    hayai::Benchmarker::AddOutputter(consoleOutputter);
    hayai::Benchmarker::RunAllTests();
    return 0;
}

g++ 1_benchmark.cpp -std=c++14 -O3 -I/usr/local/include/hayai && ./a.out

A performance hit is observable, but is probably insignificant in relevant use-cases:

Update 3: a short metaprogram

In order to address the issue of the std::function overhead, Kirk Shoop (@kirkshoop) has posted an alternative version with a small metaprogram:

struct execute {
    const unsigned long long n;

	template<typename F, class Check = decltype((*(F*)nullptr)())>
    void operator() (F what, ...) {
        for (auto i = 0; i < n; i++)
            what();
    }

	template<typename F, class Check = decltype((*(F*)nullptr)(0))>
    void operator() (F what) {
        for (auto i = 0; i < n; i++)
            what(i);
    }
};

This version appears to perform as good, if not sometimes better (an OS fluke, perhaps?) than the raw loop:

Update 4: CppCast & benchmark @ Github

This blog entry was briefly discussed in CppCast’s episode Catch 2 and C++ the Community with Phil Nash. Further ideas, reflections and analyses are welcome!

Benchmark code: cpp_declarative_times/github.com

Update 5: yet more declarative & responses

Jon Kalb has posted a very clear article with an in-depth take on a traditional C++ approach to implementing N_times(...). The Reddit responses point to the Boost.Hana implementation. These responses are in a way in tension with the original intent of writing a minimum sufficient amount of code to achieve the ruby-like syntax. However, indeed, a typical C++ solution involves metaprogramming partly to ensure that other uses apart from the one originally in mind are safe. The discussion of that phenomenon should be part of a separate article.

The other mentioned (metaprogramming) approaches inspire me to create an even more special [2. as in, opposite to generic], but still a “character-lightweight” solution. Coming back to declarative thinking and explicit and clear intent, a much simpler resolution of the two overloads can be achieved by simply creating another operator for the index overload:

struct execute_with_index {
    const unsigned long long count;

    template<typename CallableWithIndex>
    void operator() (CallableWithIndex what)
    {
        for (auto i = 0; i < count; i++)
            what(i);
    }
};

execute_with_index operator"" _times_with_index(unsigned long long count)
{
    return { count };
}

1000000_times_with_index([](unsigned long long i) {
    do_something(i);
});

this addition to the initial version solves the problem with the overloads and avoids both metaprogramming and std::function.

Next update: a more generic loop and CRTP.

Build Agent Infrastructure Testing in GoCD

In this post I would like to describe a simple technique for reducing the waiting time and stress related to build agent environment volatility when using Continuous Integration / Continuous Delivery tools like GoCD, via infrastructure testing.

The Problem

Given a modern CI server, such as GoCD, and a set of dedicated build machines (agents), it is possible to improve software development agility. Automated build/test/deploy pipelines, built to reflect the value stream, bring transparency and focus into the software delivery activities.

CI automation is software itself, and is thus susceptible to errors. Configuration management can optimize the set up of the environment, in which the build agents run. However, when computational resources are added to a CI infrastructure, i.e. to parallelize the build and thus reduce feedback times, a missing environment dependency can cause stress and pain that CI is trying to eliminate.

Consider a pipeline where a complete cycle (i.e. with slow integration tests followed by a reporting step at the end or long check-outs in the beginning) takes a significant amount of time. If one of the last tasks fails due to a configuration or an environment issue, the whole stage fails. The computational resources have been wasted just to find out that a compiler is missing. This can easily happen when there is variation in the capabilities of the build agents.

Improvement idea: fail fast! Don’t wait for environment or infrastructure mismatches

GoCD Resources as Requirements and Capabilities

If a step in a build pipeline requires a certain compiler or a particular environment, this can be conveniently expressed in the configuration of GoCD as a build agent resource. A resource can be seen as a requirement of a pipeline step that is fulfilled by a corresponding capability of a build agent.

Consider the following set-up with two build agents – one running on Windows, another one on Linux. Some tasks could be completely platform-independent, such as text processing, and thus could be potentially performed on any machine with a required interpreter installed.

agents

Build Agent is the Culprit

We have set up our environment, and have successfully tested our first commit, but the second one fails:

another_agent

The code is the same, why did the second build fail? The builds ran on different agents, but I expected them to behave similarly…

build_fails

Oh, that’s embarrassing. While yes, the script is platform-independent, there’s no executable named python2 in my Windows runner environment path.

With a one-file repository and a simple print statement this failure did not cause much damage, but as mentioned earlier, real life builds failing due to a missing executable might be costly.

Unhappy Picture

unhappy

Infrastructure Test Pipeline

In order to fail fast in situations where new agents are added to a CI infrastructure, or their environment is volatile, I propose to use a single independent pipeline that checks the assumptions that longer builds depend upon.

If a build step requires python in the path, there should be a test for it that gives this feedback in seconds without much additional waiting time. This can be as easy as calling python --version, which will fail with a non-zero return code if the binary is missing. More fine-grained assertions are possible, but should still remain fast.

If a certain binary should not be in the path, this can be asserted as well. The same goes for environment variables and file existence. Dedicated infrastructure testing tools, such as Serverspec could also be used, but having a response time under a minute is crucial in my view.

Run on All Agents

In order to validate the consistency of the CI infrastructure, the validation tasks should run on all agents that advertise a corresponding resource. This is where, in my view, the real power of GoCD comes to light, and the concepts used in it fit in the right places.

GoCD will run the test tasks on all agents that fulfill all resource requirements for the task.

run_on_all_g

Test fails

Now that we have all the tests, running them gives quick and precise feedback:

infrastructure_test_fails

Checking out the job run details reveals the offending agent. Note the test duration: under 1 second.

infrastructure_test_agent

Fixing the Infrastructure

resources_modified

Whatever the resolution of the infrastructure problem, when the infrastructure test has a good coverage of the prerequisites for a pipeline, adding new agents to the CI infrastructure should become as much fun as TDD is: write an infrastructure test, see it fail, fix the infrastructure, feel the good hormones. Add new build agents for speed — still works — great!

infrastructure_test_passes

Note how the resources that are available only on one machine are only run on one corresponding machine.

 

Happy Pictures and Developers

happy pipeline

When to Test

It is an open question, when to test the infrastructure. With the system being composed of the CI server and agents, the tests should probably run on any global state change, such as

  • added/removed/reconfigured agents
  • automatic OS updates (controversial)
  • restarts
  • network topology changes

It is also possible to schedule a regular environment check. Having the environment test pipeline be the input for other pipelines unfortunately will not do in the following sequence of events:

  • environment tests pass
  • faulty agents are added
  • downstream pipeline is triggered
  • environment failure causes a pipeline to fail

In any case, there is a REST API available for the GoCD server should automating the automation become a necessity.

Acknowledgments

I would like to thank all the great minds, authors and developers who have worked and are working to make lives of developers and software users better. Tools and ideas that work and provide value are indispensable.  The articles and the software linked in this blog entry are examples of knowledge that brings the software industry forward. I am also very grateful to my current employer for letting me learn, grow and make a positive impact.

 

 

A unit test suite is a spiderweb

A moment’s epiphany in a highly caffeinated brain; a metaphor:

a unit test suite is a spiderweb![1. When done well]

Why?

  • It’s lightweight
  • It’s sufficiently robust
  • It catches a casual bug passing by
  • It’s most useful if the maintainer reacts to a bug catch immediately
  • It needs occasional maintenance
  • It’s not watertight, but good enough
  • Some are beautifully designed, some look like a tight cloth, some are a mess
  • …insert your analogy…

There is an already established XP metaphor of a safety net for the people and the business, to which I fully subscribe. This “lightweight net” metaphor, I hope, adds another flavor to the test-suite-as-a-net analogies, especially, to lightweight unit test suites, by describing which qualities unit test suites can have.

We now know that small (biological) bugs and other small creatures can cause a lot of harm, just as tiny software bugs can and do. [1. As a corollary: one can catch larger things by being a large cat, and applying different powerful strategies, but that requires quite a muscle mass, and a good connection to the ground. P.S. no lightweight vs. non-lightweight preference comparison intended.]

The analogy might have been a cheap trick of my brain. Nonetheless, it is now externalized, and I can go to sleep.

Build Status