Testing

I’m a big fan of automated testing, for many reasons. It provides the easiest and most reliable way to see if your code is working, aside from strong static typing. Further, in my experience, driving interface design from the perspective of making it easy to test tends to make for good interfaces.

In order to test our interpreter, it will be helpful to have a testing framework which will automate the process of running tests, and collecting and presenting test results. This isn’t strictly necessary, as we could just ad-hoc design our tests, with a println indicating success here and a resultCode indicating failure there. I think we’ll have more confidence in our results if they’re all delivered the same way, through the same testing mechanisms.

Now if you’ve ever designed a testing framework, you’ll have quickly realized that like every other piece of software, having automated tests to ensure your testing framework still works is important. But we’ve got a bit of a quandary. Testing your testing framework seems valuable, but also like we’re getting into a “turtles all the way down” situation. At some point, you have to trust that the code you’re writing is correct. In our case, I started with a dumb test to be driven by the framework, essentially testing correctness of the testing framework manually.

The first test:

public class FooTest {

    @Test
    public void firstTest() {
        assertEquals(0, new Foo().bar());
    }
}

/tst/alonzo/FooTest.java

The call to the test runner:

public static void main(String[] args) {
    new TestRunner()
        .withTest(FooTest.class)
        .run();
}

/tst/alonzo/TestingMain.java

I started with these two classes before writing any of the testing framework code. In general, I believe in the value of test-driven development. Without going too much into TDD and its positives and negatives, here’s a benefit I’ve found from practicing TDD. Driving the code from a test usually results in a better, more usable interface. So, in this instance, I wrote the test the way I wanted to write a test, and wrote the code to run the test in a way that seemed easy to run tests. Honestly, this top-down thinking is one of the hardest things for new developers to really grok and practice. It is key in choosing your abstractions well and in writing interfaces that you actually would want to use.

So far, the architecture of the testing framework is relatively simple. The TestRunner collects tests, then runs them, collecting the TestResults, and finally displaying those results. In order to run the tests, the TestRunner makes use of the RunnableTest class, which does all the heavy lifting. It searches the class for any methods labeled with the @Test annotation, and the constructor. RunnableTest implements the Supplier<TestResult> interface, by running the wrapped test and constructing a result. RunnableTest works in concert with the static methods in Assert which throw AssertionFailureException to indicate a test failure.

Now we have a testing framework with which to test the code we write for the interpreter.