Makefiles for Test Driven Development

I’m a big believer in Test Driven Development. That’s the model where you write a failing test, then you write just enough code to make the test pass, but no more. It sounds counter-intuitive,but experience has shown that I get a finished, deliverable product a lot faster, and I’m less likely to find bugs. Even more importantly, my customers are less likely to find bugs.

Setting up a Makefile to support test driven development adds some additional challenges to what I’ve shown in my earlier articles. It’s probably necessary to keep source and test is separate folders just to maintain sanity. We’ll probably also have an outside dependency on our test framework, and there’s a good chance that we’ll need to create test doubles for some of the dependencies. A hypothetical folder structure might look like this:

project
  - mock        // Test doubles
  - src
  - system      // functions which interact with hardware
  - test
  - unity       // our test framework

Each of those folders contains its own Makefile. For all but the test folder, that Makefile looks very similar to what we’ve seen in previous articles.

The Test Makefile

The good news is that everything you learned in the previous articles still applies in the test Makefile. But we have some additional requirements as well.

I wrote this Makefile based on working with the Unity testing framework. If you would like to use Unity with your own project have a read of Unity Testing Framework Setup.

New Requirements

First, we’re going to need to include headers and source from other folders. The mock and unity folders contain code that will only be used by the tests. We’ll also need to compile against the source of the units that you are testing.

Additionally successful completion of a build should require actually running and passing the tests. I prefer to make two targets: one that builds the test, and another default target which runs the test, and lists the built test as a dependency.

Including Other Folders

First let’s get the include and linking paths taken care of.

# test/Makefile

CFLAGS=-I../mock -I../src -I../unity -I../system
LDFLAGS=-L../unity -lunity
VPATH=../mock:../src

You can see from the CFLAGS variable that I need to include files from each folder that isn’t this folder. I’ll also need to include files from this folder, but that doesn’t require a special -I flag callout.

For the LDFLAGS variable, the only one of those folders which is building a library is unity. That was a choice on my part. In this case I could have easily built the mock objects into a library as well, but I wasn’t sure that I wouldn’t run into a situation where I would need to split my tests, and my mocks, across two executables to avoid naming conflicts.

As a refresher, -L../unity tells the linking stage of compilation that it should look for libraries in the ../unity folder. -lunit tells the linker that it should link with a library having the name libunity.a, which is the name I gave the unity library when I created the Makefile.

VPATH is the new kid on the block here. This is another magic variable that only applies to GNU Make. It’s a colon-separated list of folders where the compiler should look for sources to build objects referenced in this Makefile. We’ll look at it again in a minute.

Using Tests

For my tests, I want to use the same wildcard magic I demonstrated previously:

# test/Makefile

SRCS=$(wildcard *.c)
BASES=$(basename $(SRCS))
OBJS=$(addsuffix .o, $(BASES))

all: test_suite
	./test_suite

test_suite: $(OBJS)
	$(CC) $^ $@ $(LDFLAGS)

The program test_suite should return 0 if all tests past, and something other than 0 if there are failing tests. This will cause the build to fail if there are failing tests.

That will pick up all of my tests, as well as any infrastructure files that I need, such as my main function and test suite definitions.

But I also need to include the units that I’m trying to test. So I make a couple of changes.

// System Under Test
SUT=controller.o motor.o
  .
  .
  .
test_suite: $(OBJS) $(SUT)
	$(CC) $^ $@ $(LDFLAGS)

Now the units that we’re testing are going to be compiled in as well. Note that the files controller.c and motor.c don’t live in the test folder, but in the src folder. Because of the VPATH variable I defined earlier, the compiler knows where to pick up the source files.

For a lot of programs, this is probably enough. But sometimes we’ll need more.

I stole this example from an embedded program, and there are some functions I can’t call in a test, because they operate on hardware that I don’t have on my workstation, or they require me to control the outputs from a system function.

To fix that problem, I use test doubles. These are functions which have the same signature as the functions I’m calling, but instead of taking the usual action, they either return values that I set up beforehand, or they record the input given to them so that I can query it later in tests.

TESTDOUBLES=mock_motor.o mock_time.o mock.o
  .
  .
  .
test_suite: $(OBJS) $(SUT) $(TESTDOUBLES)
	$(CC) $^ -o $@ $(LDFLAGS)

So far, this is good. We have a test suite, and we can use it to test drive our code. This might be enough to satisfy your needs on a personal project.

No Builds Without Passing Tests

If your project is an ongoing product or internal system, you probably want to make sure that nobody can ship a build that doesn’t pass it’s automated tests.

We can make this happen by changing the top level Makefile.

# Makefile

all: product

product: test
	make -C src

test:
	make -C test

Now, based on behavior we defined in our tests, the build will fail if tests fail, and the product won’t be built. This prevents a bad build from being available to ship to customers or put into production.

I’ll dig into how to write tests and test doubles in later articles.

Comments

comments powered by Disqus