I'm a big believer in Test Driven Development. That's the model where you write a failing test, then you write just enough code to make the test pass, but no more. It sounds counter-intuitive,but experience has shown that I get a finished, deliverable product a lot faster, and I'm less likely to find bugs. Even more importantly, my customers are less likely to find bugs.
Setting up a Makefile to support test driven development adds some additional challenges to what I've shown in my earlier articles. It's probably necessary to keep source and test is separate folders just to maintain sanity. We'll probably also have an outside dependency on our test framework, and there's a good chance that we'll need to create test doubles for some of the dependencies. A hypothetical folder structure might look like this:
project - mock // Test doubles - src - system // functions which interact with hardware - test - unity // our test framework
Each of those folders contains its own Makefile. For all but the
folder, that Makefile looks very similar to what we've seen in previous
The Test Makefile
The good news is that everything you learned in the previous articles still applies in the test Makefile. But we have some additional requirements as well.
First, we're going to need to include headers and source from other folders. The mock and unity folders contain code that will only be used by the tests. We'll also need to compile against the source of the units that you are testing.
Additionally successful completion of a build should require actually running and passing the tests. I prefer to make two targets: one that builds the test, and another default target which runs the test, and lists the built test as a dependency.
Including Other Folders
First let's get the include and linking paths taken care of.
CFLAGS=-I../mock -I../src -I../unity -I../system LDFLAGS=-L../unity -lunity VPATH=../mock:../src
You can see from the
CFLAGS variable that I need to include files from each
folder that isn't this folder. I'll also need to include files from this
folder, but that doesn't require a special
-I flag callout.
LDFLAGS variable, the only one of those folders which is building
a library is
unity. That was a choice on my part. In this case I could
have easily built the mock objects into a library as well, but I wasn't sure
that I wouldn't run into a situation where I would need to split my tests,
and my mocks, across two executables to avoid naming conflicts.
As a refresher,
-L../unity tells the linking stage of compilation that it
should look for libraries in the
-lunit tells the
linker that it should link with a library having the name
which is the name I gave the unity library when I created the Makefile.
VPATH is the new kid on the block here. This is another magic variable
that only applies to GNU Make. It's a colon-separated list of folders where
the compiler should look for sources to build objects referenced in this
Makefile. We'll look at it again in a minute.
For my tests, I want to use the same wildcard magic I demonstrated previously:
SRCS=$(wildcard *.c) BASES=$(basename $(SRCS)) OBJS=$(addsuffix .o, $(BASES)) all: test_suite ./test_suite test_suite: $(OBJS) $(CC) $^ $@ $(LDFLAGS)
test_suite should return 0 if all tests past, and something
other than 0 if there are failing tests. This will cause the build to fail
if there are failing tests.
That will pick up all of my tests, as well as any infrastructure files that I need, such as my main function and test suite definitions.
But I also need to include the units that I'm trying to test. So I make a couple of changes.
// System Under Test SUT=controller.o motor.o . . . test_suite: $(OBJS) $(SUT) $(CC) $^ $@ $(LDFLAGS)
Now the units that we're testing are going to be compiled in as well. Note
that the files controller.c and motor.c don't live in the test folder, but
src folder. Because of the VPATH variable I defined earlier, the
compiler knows where to pick up the source files.
For a lot of programs, this is probably enough. But sometimes we'll need more.
I stole this example from an embedded program, and there are some functions I can't call in a test, because they operate on hardware that I don't have on my workstation, or they require me to control the outputs from a system function.
To fix that problem, I use test doubles. These are functions which have the same signature as the functions I'm calling, but instead of taking the usual action, they either return values that I set up beforehand, or they record the input given to them so that I can query it later in tests.
TESTDOUBLES=mock_motor.o mock_time.o mock.o . . . test_suite: $(OBJS) $(SUT) $(TESTDOUBLES) $(CC) $^ $@ $(LDFLAGS)
So far, this is good. We have a test suite, and we can use it to test drive our code. This might be enough to satisfy your needs on a personal project.
No Builds Without Passing Tests
If your project is an ongoing product or internal system, you probably want to make sure that nobody can ship a build that doesn't pass it's automated tests.
We can make this happen by changing the top level Makefile.
all: product product: test make -C src test: make -C test
Now, based on behavior we defined in our tests, the build will fail if tests fail, and the product won't be built. This prevents a bad build from being available to ship to customers or put into production.
I'll dig into how to write tests and test doubles in later articles.