Day 1: A simple Unittest Framework

As I did rant about the lack of a useful build and test ecosystem in C++ earlier, I obviously needed to write my own for my development.

The first step for this is the Test Framework.

For every Unit-Test framework, one rule is more important than any other: The more difficult it is for a developer, to write and perform unit tests, the less he/she will write them. Therefore any unit-test framework that aims to be really used, needs to be as simple as possible.

For the development of my game engine, I will accomplish that, by utilising Python in the build itself, to automatically find any test class written, and perform the tests while building.

The Framework architecture

The C++ part of this test framework is composed from seven classes, from which one UnitTest class needs to be overwritten for any unit test. This can happen completely inside a header file, neglecting the need, to write a second implementation file for every unit-test header in use.

The framework itself looks like this:Class diagram for the UnitTest framework

The developer overwrites the UnitTest class, for every test case, he wants to write. Any TestCase class is pre-initialised with a LogRoll instance, the developer can use, to write log statements to, during the execution of the test.

All the UnitTest implementations will finally be aggregated to a single TestSuite instance and run from there. The TestSuite itself is responsible to set the right stage for the tests and perform any reporting, derived from the test execution. It will store the final report in an instance of TestReport that is returned from the TestSuites runTests method.

The user can now choose one of the provided implementations of the TestReportPresentation class, to store the test report in a certain format. As of now, the only availible implementation PlainTextReportPresentation lets you store the report in a simple plain-text format, but I will most likely include different report presentation classes for pretty-html reporting and whatever I need in the future.

Use it

To use this test framework in a pure C++ enviroment (without python as a processor, which I will add later), One would need to write some test classes (obviously) and write a single main method somewhere. For example a main.cpp file.

This main should look a bit like this:

#include <iostream>
#include <UnitTestFramework.h>
#include "TestFoo.h"
#include "TestBar.h"
#include "TestOther.h"

int main(int argc, char **argv) {
	Testing::TestSuite suite;
	suite.addTest(new TestFoo());
	suite.addTest(new TestBar());
	suite.addTest(new TestOther());

	Testing::TestReport report = suite.runTests();

	std::cout << "All done.\n";

	Testing::TestReportPresentation* presentation =
			new Testing::PlainTextReportPresentation();

	std::cout << presentation->toString(&report);
	std::cout.flush();
}
  • Include iostream and the UnitTest framework.
  • Include every test class, you want to perform
  • In main: create a new TestSuite
  • add a new instance of every test class to the test suite
  • tell the TestSuite to run the tests and collect the test report, returned from the method.
  • Decide upon a TestReportPresentation implementation to use, and create a report preentation from it.
  • Decide what to do with the report. In the above case I simply outputted it to std::cout. In a real enviroment, one woud propably write that to a file or something like that.
  • Decide upon a return code, for successful or unsuccessful test execution results

Things to do

The framework as it now stands is powerful enough for most of the actual tests in my scenario, but there is still to much boilerplate code involved, to execute the tests.

Using Python, I want to write a test runner, that mimics the execution pattern of the main method described above, but auto-detects any test classes and generates a matching test suite on the fly.

Also, the python part should be able, to compile the main code (the code under test) into a library and use it and it’s headers, to perform the tests upon.

Stumbling Blocks along the Way

Some difficulties, I ran into, while programming and how I dealt with them:

The clock, tick tock…

While writing the classes of the framework, I realised, that it is very vital for a stable test framework, to be able to run without the need of different libraries, other than STL. Therefore I had a pretty hard time, implementing something, that would give me a timestamp in millisecond accuracy without using the POSIX or BOOST libraries.

I ended up with something, that isn’t precise to the actual millisecond on the clock, but is precise the the actual second on the clock and to the millisecond in the delta between to given points in time, after the program started.
For assigning timestamps to log messages and test runs, this is sufficient.

The code looks like this:

#include <time.h>

static clock_t initialclock = clock();
static time_t initialtime = time(NULL);

/**
 * returns the current time in milliseconds since 1 January 1970 01:00.
 *
 * Since the milliseconds are calculated from the clocktick value, this value
 * is only somewhat accurate, if no debugging takes place.
 */
inline long getCurrentTimeMillis(){
	clock_t now = clock();
	now = now - initialclock;

	long mymillis = (now*1000)/CLOCKS_PER_SEC;
	long t = (initialtime*1000)+mymillis;
	return t;
}

So in short, I used the time() function of the time.h standard library header, to given me a single reference point in time, assuming 000 millis to that point.
For every following request to getCurrentTimeMillis() I use the clock() function to determine the clock ticks that have past since the program did start, divide them by the CLOCKS_PER_SEC constant (multiplied by 1000) to get milliseconds and determine the millisecons timestamp from there.

Numbers and Strings

Yes, that old croc again.

C++ has some basic classes, that are usable and not all bad, and save the world from having about as much String classes, as C++ programmers, which is good, because it has not always been that way.

Anyway, for some arbitrary reason, that I never really could get behind, there is no function to this standard string class, that allows you to create a string, that represents the value of a numeric variable. In short, there is no function, that let’s you create this: “42” from this: 42.

Once again, we could use boost, and I will certainly do that in my main engine code, but for the Test Framework, I wanted to be independent from any 3rd party libs.

So here is what I did, without any further comment. 😉

/**
 * because C++ is stupid like that. 😦
 */
inline std::string numberToString(int number){
    char buff [255];
    std::sprintf(buff, "%d", number);
    return (std::string) buff;
}

Finally

Thats it for day one.

For the moment, the source for this framework can only be obtained from a sourceforge svn, here: Gamedragon Sourceforge SVN.
If you are interested in using this framework in your own projects, please let me know in the comments. I might be persuaded, to package it in its own right and set up a separate sourceforge project for the build tools I use for the Dragon Game Engine.

Next up, using Python, to work some magic and reduce boilerplate code for the test execution. After integration that into an overall build tool, I am finally good to do some actual work. 😉

So long, and thanks for all the fish.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s