Chapter 11. Review of Testability

Chapter 11. Review of Testability
	Part I. Roulette

Table of Contents

Wheel Rework
Java NonRandom Class
Python NonRandom Class

This chapter presents some design rework and implementation rework for testability purposes. While testability is very important, new programmers can be slowed to a crawl by the mechanics of building test drivers and test cases. We prefer to emphasize the basic design considerations first, and address testability as a feature to be added to a working class.

Additionally, we'll address some issues in construction of class instances and an idealized structure for the main procedure.

Overview

We have been studiously ignoring an elephant standing in the saloon. This is the problem of testing an application that includes a random number generator (RNG). There are two questions raised:

How can we develop formalized unit tests when we can't predict the random outcomes? This is a serious testability issue in randomized simulations. This question also arises when considering interactive applications, particularly for performance tests of web applications where requests are received at random intervals.
Are the numbers really random? This is a more subtle issue, and is only relevant for more serious applications. Cryptographic, statistical or actuarial applications may care about the randomness of random numbers. This is a large subject, and well beyond the scope of this book. We'll just assume that our random number generator is good enough.

We'll address this first issue by developing some scaffolding that permits controlled testing. There are three approaches to replacing the random behavior with something more controlled. One approach is to subclass Wheel to create a more testable version without the random number generator. An alternative to changing wheel is to define a subclass of java.util.Random (or Python's random) that isn't actually random. A third approach is to record the sequence of random numbers actually generated from a particular seed value and use this to define the exected test results. For more information on this third alternative, see On Random Numbers.

On Random Numbers

Random numbers aren't actually “random”. Since they are generated by an algorithm, they are sometimes called pseudo-random. The distinction is important because pseudo-random numbers are generated in a fixed sequence from a given seed value. Computing the next value in the sequence involves a simple calculation that is expected to overflow the available number of bits of precision leaving apparently random bits as the next value. This leads to results which, while predictable, are arbitrary enough that they pass rigorous statistical tests and are indistinguishable from data created by random processes.

We can make an application less predictable by choosing a very hard to predict seed value. One popular choice for the seed is the system clock. In some operating systems a special device is available for producing random seed values. In Linux this is typically /dev/random.

Conversely, we can make an application more predictable by setting a fixed seed value and noting the sequence of numbers generated. We can write a short demonstration program to see the effect of setting a fixed seed. This will also give us a set of predictable answers for unit testing.

We'll need to build and execute a program that reveals the fixed sequence of spins that are created by the non-random number generator.

Procedure 11.1. Revealing the Non-Random Spins

Create an instance of NonRandom or Random. Set the seed of the this random number generator to a fixed value, for example, 3.
Create an instance of Wheel using this non-random number generator.
Call the next method of Wheel six times, and print the winning Bin instances. This sequence will always be the result for a seed value of 3.

This discovery procedure will give you the results needed to create unit tests for Wheel and anything that uses it, for example, Game.

Good testability is achieved when there are no changes to the target software. For this reason, we don't want to have two versions of Wheel, one for testing and one for normal operations.

Instead of having two versions of Wheel, it's slightly better to have a random number generator that creates a known sequence with which we can test. To get this known sequence, we have a choice between creating a non-random subclass of java.util.Random or controlling the seed for the random number generator used by Wheel. Both will produce a known sequence of non-random values.

One consequence of either of these decisions is that we have to make the random number generator in Wheel more visible. Our favorite approach to making something more visible is to assign an object through an official interface method or possible as part of the constructor. In the case of Wheel, we'd jave our overall simulation or test assign an appropriate number generating object to the instance of Wheel rather than have Wheel privately create a generator.

When we are doing testing, we can associate a Wheel with an instance of NonRandom, or an instance of Random initialized with a known seed. For actual use, we can then associate a Wheel with an instance of java.util.Random; this will either use the RNG's default constructor to assure unpredictability.

We'll need an additional constructor for Wheel that allows us to provide an appropriately initialized generator. Our current constructor secretly creates an instance of a RNG, using the RNG's default constructor. We'll need an additional method that provides an RNG, already initialized with an appropriate value.

For Python programmers, this is handled as a second parameter with a default value.

Note that, consistent with our principle of deferred binding, we don't have to choose exactly which implementation we will use. By allowing Wheel to take either an instance of Random initialized with a given seed or an instance of a new subclass, NonRandom, we have given ourselves the flexibility to choose either implementation. We'll provide specifications for NonRandom, but using a fixed seed for Random is seen by some as simpler.

Python Subclass Design

Because of the late binding in Python, and the lack of compile-time type checking, we don't have to use strict subclass inheritance to replace the Python random number generator with our own non-random number generator. We could cheat and provide a class with just the minimal interface that our application requires.

While legal, we prefer not to bypass the inheritance tree in Python. We find that applications are much easier to maintain if reasonable care is taken to assure that the classes and interfaces are matched up as precisely as possible. In this case, we'll want to make our non-random generator a strict subclass of the Random class.


Deliverables		Questions and Answers