Tag Archives: virtual time

Why wait forever for the tests? Fast tests of slow software.

Time is volatile

Imagine writing a cron-like functionality that should produce some side-effect, such as cleanup. The intervals between such actions might be quite long. How does one test that? One can surely reason about the software, but given a certain complexity, test should be written, proving that certain important scenarios work as intended.

It’s common that software depends on time flow as dictated by the physical time flow, reflected via some clock provider. However, resetting the time to a year ahead won’t make the CPU work faster and make all the computations it should have performed within that year. A clock is also a volatile component that can be manipulated, thus if time is an issue, it’s probably a good idea not to depend on it directly, following the Stable Dependencies Principle and the Dependency Inversion Principle.

Luckily, there is an abstraction for time, at least in Reactive Extensions (Rx), which is the Scheduler.

Slow non-tests

Here’s a slow Groovy non-test, waiting for some output on the console using RxGroovy:

import rx.*
import java.util.concurrent.TimeUnit

def observable = Observable
	.just(1)
	.delay(5, TimeUnit.SECONDS)

observable.subscribe { println 'ah, OK, done! Or not?' }

Observable
	.interval(1,TimeUnit.SECONDS)
	.subscribe { println 'still waiting...' }

println 'starting to wait for the test to complete ...'

observable.toBlocking().last()

Running it produces the following slow-ticking output:

oldnontest 1

Interpreting such tests without color can be somewhat challenging 2.

Fast tests

Now let’s test something ridiculous, such as waiting for a hundred days using Spock. Luckily, RxJava & RxGroovy also do implement the test scheduler, thus enabling fast tests using virtual time:

import spock.lang.Specification

import rx.Observable
import rx.schedulers.TestScheduler
import java.util.concurrent.TimeUnit


class DontWaitForever extends Specification {
    def "why wait?"() {
        setup:
            def scheduler = new TestScheduler()

            // system under test: will tick once after a hundred days
            def observable = Observable
                .just(1)
                .delay(100, TimeUnit.DAYS, scheduler)
            def done = false

        when:
            observable.subscribe {
                done = true
            }

            // still in the initial state
            done == false

        and:
            scheduler.advanceTimeBy 100, TimeUnit.DAYS

        then:
            done == true
    }
}

fasttest 3

just checking, advancing the time by 99 days results in a failure:

just_checking

Delightful, groovy colors!

Source

github.com/d-led/dont_wait_forever_for_the_tests

  1. Caputured with the wonderful pragmatic tool LICEcap by the Reaper developers
  2. Here, the ‘still waiting’ subscription is terminated after the first subscription ends. Try exchanging the order of the subscribe calls.
  3. Building using Gradle

Deterministic Testing of Concurrent Behavior in RxCpp

A Retrospective

After getting inspired by The Reactive Manifesto, it is hard not to get excited about Reactive Extensions. Such excitement has lead to a series of hello-world articles and some code examples. While Reactive Extensions take over the programming world in C#, Java and JavaScript, it seems, the world of C++ is slow to adopt RxCpp.

The new ReactiveX Tutorial link list is a great place to start learning and grokking. This article is an attempt to bring RxCpp closer to C++ developers who might not see yet, how a reactive programming model might help writing better, more robust code.

Testing concurrency with RxCpp

A previous article showed how to test ViewModels in C# by parameterizing the ViewModels with a scheduler. In a UI setting, the scheduler usually involves some kind of synchronization with the GUI thread. Testing keystrokes arriving at certain speed would require some effort to simulate events, probably leading to brittle tests. With the scheduler abstraction, the concurrent behavior of a component is decoupled from physical time, and thus can be tested repeatedly and very fast. This was the C# test:

(new TestScheduler()).With(scheduler =>
{
    var ticker = new BackgroundTicker(scheduler);

    int count = 0;
    ticker.Ticker.Subscribe(_ => count++);
    count.Should().Be(0);

    // full control of the time without waiting for 1 second
    scheduler.AdvanceByMs(1000);
    count.Should().Be(1);
});

Show Me The Code

Without further ado, the C++ version is not very far from the C# version. In a simple test, we can parameterize a sequence of integer values arriving at specified intervals (a ticker) with a coordination (why coordination and not scheduler, read in the RxCpp developer manual:

auto seq = rxcpp::observable<>::interval(
            std::chrono::milliseconds(1),
            some_scheduler
);

The deterministic test scheduler API is currently available through a worker created on the test scheduler:

auto sc = rxcpp::schedulers::make_test();
auto worker = sc.create_worker();
auto test = rxcpp::identity_same_worker(worker);

The rest should read like English:

int count = 0;

WHEN("one subscribes to an observable sequence on the scheduler") {
  auto seq = rxcpp::observable<>::interval(
              std::chrono::milliseconds(1),
              test // on the test scheduler
             ).filter([](int i) { return i % 2; });

  seq.subscribe([&count](int){
    count++;
  });

  THEN("the sequence is not run at first") {
    worker.sleep(2 /* ms */);

    CHECK(count == 0);

    AND_WHEN("the test scheduler is advanced manually") {

      THEN("the sequence is run as expected") {
        worker.advance_by(8 /* ms */);
        CHECK(count == 5);
      }
    }
  }
}

The full test can be seen @github, and is built on Travis CI

ReactiveUI 6 and ViewModel Testing

Testability and ReactiveUI

ReactiveUI XAML example

In the previous articles about ReactiveUI I’ve claimed without bringing any evidence that writing ViewModels using ReactiveObjects brings about testability. While the aspects of testing Rx and ReactiveUI have been treated at length in the respective authors’ blogs linked herein, this post is intended as a quick glance for the impatient online surfer at the hello-world testing code, which has been written “post-mortem” 1 as a follow-up to the previous articles.

An update to ReactiveUI 6

Paul Betts and contributors have been busy simplifying and extending the library 2. There are some extension methods now that help creating observables from properties, and transforming observables to properties. In the example ViewModel from previous articles, there’s an observable stream of strings that is simply transformed into a property defined as follows:

ObservableAsPropertyHelper<string> _BackgroundTicker;
public string BackgroundTicker
{
    get { return _BackgroundTicker.Value; }
}

In the constructor, the helper is now initialized without using strings:

public WordCounterModel(IObservable<string> someBackgroundTicker)
{
someBackgroundTicker
    .ToProperty(this, x => x.BackgroundTicker, out _BackgroundTicker);
...
}

instead of string-based error-prone

_BackgroundTicker = new ObservableAsPropertyHelper<string>(
	someBackgroundTicker, _ => raisePropertyChanged("BackgroundTicker")
);

For the actual changes in ReactiveUI, consult Paul Betts’ insightful log.

A simple test

Since the tests have been written after writing the example code, I’ve been searching for the “Generate Unit Test” context menu in Visual Studio 2013. The context menu is not there, but luckily some enthusiasts recreated the functionality partly: → Unit Test Generator.

After the initial set-up and

failedtest

here’s the simple test of the word count property:

[TestMethod]
public void WordCounterModelTest()
{
    var mock = new Mock<IObservable<string>>();
    var vm = new WordCounterModel(mock.Object);

    vm.WordCount.Should().Be(0);

    vm.TextInput = "bla!";
    vm.WordCount.Should().Be(1);

    vm.TextInput = "bla, bla!!";
    vm.WordCount.Should().Be(2);
}

In it, one can observe the use of MOQ for mocking a dummy and FluentAssertions for beautifully readable Spec/BDD-style assertions 3.

So, there’s no UI involved, and the UI is dogmatically bound from XAML with almost no code-behind.

Testing time series

The hello world example program simulated a dependency on some timed series of strings, ticking every second. While this is not specific to ReactiveUI, let’s make use of the test scheduler 4. For that, the time series should optionally depend on an injected IScheduler:

public class BackgroundTicker
{
    IScheduler scheduler = Scheduler.Default;

    public BackgroundTicker(IScheduler other_scheduler = null)
    {
        if (other_scheduler != null)
            scheduler = other_scheduler;
    }

    public IObservable<string> Ticker
    {
        get
        {
            return Observable
                .Interval(TimeSpan.FromSeconds(1), scheduler)
                .Select(_ => DateTime.Now.ToLongTimeString());
        }
    }
}

The test instantiates a test scheduler, which is then advanced to make deterministic assertions. The code should speak for itself:

[TestMethod]
public void BackgroundTickerTest()
{
    (new TestScheduler()).With(scheduler =>
    {
        var ticker = new BackgroundTicker(scheduler);

        int count = 0;
        ticker.Ticker.Subscribe(_ => count++);
        count.Should().Be(0);

        scheduler.AdvanceByMs(1000);
        count.Should().Be(1);

        scheduler.AdvanceByMs(2000);
        count.Should().Be(3);
    });
}

Summary

passedtest

Code: https://github.com/d-led/reactiveexamples

Previous article: The WPF + ReactiveUI Refactored Version of the Responsive UI Hello World.

See also: the c++ version.

  1. as in, not within TDD
  2. which now also targets Xamarin and Windows Phone 8 and Windows Store Apps
  3. I originally intended to use SpecFlow, but the specs refused to flow frictionlessly
  4. see Intro to Rx