Methodology, Technique and Process
Prev Chapter 1. Foundations Next

Methodology, Technique and Process

We want to focus on technical skills; we won't follow any particular software development methodology too closely. We hesitate to endorse a specific methodology; doing so inevitably alienates readers who embrace a different methodology. To continue this rant, we find that almost everyone has an existing notion of the proper way to organize software development work. This leads to the common practice of customizing methodologies, in most cases without a deep background in the methodology or the changes being made.

We prefer to lift up a few techniques which have a great deal of benefit.

The exercises are presented as if we are doing a kind of iterative design with very, very small deliverables. We present the exercises like this for a number of reasons.

First, we find that beginning designers work best with immediate feedback on their design decisions. While we present the design in considerable detail, we do not present the final code. Programmers new to OO design will benefit from repeated exposure to the transformation of problem statement through design to code.

Second, for iterative or agile methodologies, this presentation parallels the way software is developed. A project manager may use larger collections of deliverables. However, the actual creation of functional source eventually decomposes into classes, fields and methods. For project managers, this exposition will help them see where and how rework can occur; giving them a chance to plan for the kind of learning that occur in most projects.

Third, for project teams using a strict waterfall methodology -- with all design work completed before any programming work -- the book can be read in a slightly different order. From each exercise chapter, read only the overview and design sections. From that information, integrate the complete design. Then proceed through the deliverables sections of each chapter, removing duplicates and building only the final form of the deliverables based on the complete design. This will show how design rework arises as part of a waterfall methodology.

We try to embrace a variety of deliverables in addition to working source code. In particular, we expect unit test classes in addition to the functional classes. Further, we expect Javadoc comments or Python docstrings in each class. Additionally, we will need to write some demonstration classes in order to determine the exact sequence of random numbers that are created from a specific seed; these are not deliverable as part of the final application, but will be necessary to construct proper unit tests.

This section addresses a number of methodology or process topics:

On Quality

Our approach to overall quality assurance is relatively simple. We feel that a focus on unit testing and documetation covers most of the generally accepted quality factors. The Software Engineering Institute (SEI) published a quality measures taxonomy. While officially "legacy", it still provides an exhaustive list of quality attributes. These are broadly grouped into five categories. Our approach covers most of those five categories reasonably well.

  • Need Satisfaction. Does the software meet the need? We start with a problem statement, define the use case, and then write software which is narrowly focused on the actor's needs. By developing our application in small increments, we can ask ourself at each step, "Does this meet the actor's needs?" It's fairly easy to keep a software development project focused when we have use cases to describe our goals.

  • Performance. We don't address this specifically in this book. However, the presence of extensive unit tests allows us to alter the implemention of classes to change the overall performance of our application. As long as the resulting class still passes the unit tests, we can develop numerous alternative implementations to optimize speed, memory use, input/output, or any other resource.

  • Maintenance. Software is something that is frequently changed. It changes when we uncover bugs. More commonly, it changes when our understanding of the problem, the actor or the use case changes. In many cases, our initial solution merely clarifies the actor's thinking, and we have to alter the software to reflect a deeper understanding of the problem.

    Maintenance is just another cycle of the iterative approach we've chosen in this book. We pick a feature, create or modify classes, and then create or modify the unit tests. In the case of bug fixing, we often add unit tests to demonstrate the bug, and then fix our classes to pass the revised unit tests.

  • Adaptation. Adaptation refers to our need to adapt our software to changes in the environment. The environment includes interfaces, the operating system or platform, even the number of users is part of the environment. When we address issues of interoperability with other software, portability to new operating systems, scalability for more users, we are addressing adaptation issues.

    We chose Python and Java to avoid having interoperability and portability issues — these platforms give admirable support for many scalability issues. Generally, a well-written piece of software can be reused. While this book doesn't focus on reuse, Java and Python are biased toward writing reusable software.

  • Organizational. There are some organizational quality factors: cost of ownership and productivity of the developers creating it. We don't address these directly. Our approach, however, of developing software incrementally often leads to good developer productivity.

Our approach (Incremental, Unit Testing, Embedded Documentation) assures high quality in four of the five quality areas. Incremental development is a way to focus on need satisfaction. Unit testing helps us optimize resource use, and do maintenance well. Our choices of tools and platforms help us address adaptation.

The organizational impact of these techniques isn't so clear. It is easy to mis-manage a team and turn incremental development into a quagmire of too much planning for too little delivered software. It is all too common to declare that the effort spent writing unit test code is "wasted".

Ultimately, this is a book on OO design. How people organize themselves to build software is beyond our scope.

On Rework

In the section called “Problem Statement”, we described the problem. In the section called “Solution Approach”, we provided an overview of the solution. The following parts will guide you through an incremental design process; a process that involves learning and exploring. This means that we will coach you to build classes and then modify those classes based on lessons learned during later steps in the design process. See our Soapbox on Rework for an opinion on the absolute necessity for design rework.

We don't simply present a completed design. We feel that it is very important follow a realistic problem-solving trajectory so that beginning designers are exposed to the decisions involved in creating a complete design. In our experience, all problems involve a considerable amount of “learn as you go”. We want to reflect this in our series of exercises. In many respects, a successful OO design is one that respects the degrees of ignorance that people have when starting to build software. We will try to present the exercises in a way that teaches the reader how to manage ignorance and still develop valuable software.

Soapbox on Rework

We consider design rework to be so important, we will summarize the idea here.

Important

The best way to learn is to make mistakes.

Rework is a consequence of learning.

All of software development can be described as various forms of knowledge capture. A project begins with many kinds of ignorance and takes steps to reduce that ignorance. Some of those steps should involve revising or consolidating previous learnings.

A project without rework is suspiciously under-engineered.

For some, the word rework has a negative connotation. If you find the word distasteful, please replace every occurance with any of the synonyms: adaptation, evolution, enhancement, mutation. We prefer the slightly negative connotation of the word rework because it helps managers realize the importance of incremental learning and how it changes the requirements, the design and the resulting software.

Since learning will involve mistakes, good management plans for the costs and risks of those mistakes. Generally, our approach is to manage our ignorance; we try to create a design such that correcting a mistake only fixes a few classes.

We often observe denial of the amount of ignorance involved in creating IT solutions. It is sometimes very difficult to make it clear that if the problem was well-understood, or the solution was well-defined there would be immediately applicable off-the-shelf or open-source solutions. The absence of a ready-to-hand solution generally means the problem is hard. It also means that there are several degrees of ignorance: ignorance of the problem, solution and technology; not to mention ignorance of the amount of ignorance involved in each of these areas.

We see a number of consequences of denying the degrees of ignorance.

Programmers. For programmers, experienced in non-OO (e.g. procedural) environments, one consequnece is that they find learning OO design is difficult and frustrating. Our advice is that since this is new, you have to make mistakes or you won't learn effectively. Allow yourself to explore and make mistakes; feel free to rework your solutions to make them better. Above all, do not attempt to design a solution that is complete and perfect the very first time. We can't emphasize enough the need to do design many times before understanding what is important and what is not important in coping with ignorance.

In one advanced programming course, we observed the following sad scenario. The instructor provided an excellent background in how to create abstract data type (ADT) definitions for the purely mathematical objects of scalar, vector and matrix. The idea was to leverage the ADTs to implement more complex operations like matrix multiplication, inversion and Gaussian elimination. The audience, primarily engineers, seemed to understand how this applied to things they did every day. The first solution, presented after a week of work, began with the following statement: “Rather than think it through from the basic definitions of matrix and vector, I looked around in my drawer and found an old FORTRAN program and rewrote that, basically transliterating the FORTRAN.” We think that a lack of experience in the process of software design makes it seem that copying an inappropriate solution is more effective than designing a good solution from scratch.

Managers. For managers, experienced in non-object implementation, the design rework appears to be contrary to a fanciful expectation of reduced development effort from OO techniques. The usual form for the complaint is the following: “I thought that OO design was supposed to be easier than non-OO design.” We're not sure where the expectation originates, but good design takes time, and learning to do good design seems to require making mistakes. Every project needs a budget for making the necessary mistakes, reworking bad ideas to make them good and searching for simplifications.

Methodology. Often, management attempts the false economy of minimizing rework by resorting to a waterfall methodology. The idea is that having the design complete before attempting to do any development prevents design rework. We don't see that this waterfall approach minimizes rework; rather, we see it shifting the rework forward in the process. There are two issues ignored by this approach: how we grow to understand the problem domain and providing an appropriate level of design detail.

We find that the initial “high-level” design can miss details of the problem domain, and this leads to rework. Forbidding rework amounts to mandating a full understanding of the problem. In most cases, our users do not fully understand their problem any more than our developers understand our users. Generally, it is very hard to understand the problem, or the solution. We find that hands-on use of preliminary versions of software can help more than endless conversations about what could be built.

In the programming arena, we find that Java and Python (and their associated libraries) are so powerful that detailed design is done at the level of individual language statements. This leads us to write the program either in English prose or UML diagrams (sometimes both) before writing the program in the final programming language. We often develop a strong commitment to the initial design, and subsequent revisions are merely transliterations with no time permitted for substantial revisions. While we have written the program two or three times over, the additional quality isn't worth doubling or tripling the workload. We feel that the original workload should be managed as planned cycles of work and rework.

On Decision-Making

Many of the chapters will include some lengthy design decisions that appear to be little more than hand-wringning over nuances. While this is true to an extent, we need to emphasize our technique for doing appropriate hand-wringing over OO design. We call it “Looking For The Big Simple”, and find that managers don't often permit the careful enumeration of all the alternatives and the itemization of the pros and cons of each choice. We have worked with managers who capriciously play their “schedule” or “budget” trump cards, stopping useful discussion of alternatives. This may stem from a fundamental discomfort with the technology, and a consequent discomfort of appearing lost in front of team members and direct reports. Our suggestion in this book can be summarized as follows:

Important

Good OO design comes from a good process for technical decision-making.

First, admit what we don't know, and then take steps to reduce our degrees of ignorance.

Which means not saying “work smarter not harder” unless we also provide the time and budget to actually get smarter. The learning process, as with all things, must be planned and managed. Our lesson learned from Blaise Pascal is that a little more time spent on design can result in considerable simplification, which will reduce development and maintenance costs.

It's also important to note that no one in the real world is omniscient. Some of the exercises include intentional dead-ends. As a practical matter, we can rarely foresee all of the consequences of a design decision.

On Reuse

While there is a great deal of commonality among the three games, the exercises do not start with an emphasis on constructing a general framework. We find that too much generalization and too much emphasis on reuse is not appropriate for beginning object designers. See Soapbox on Reuse for an opinion on reuse. Additionally, we find that projects that begin with too-lofty reuse goals often fail to deliver valuable solutions in a timely fashion. We prefer not to start out with a goal that amounts to boiling the ocean to make a pot of tea.

Soapbox on Reuse

While a promise of OO design is reuse, this needs to be tempered with some pragmatic considerations. There are two important areas of reuse: reusing a class specification to create objects with common structure and behavior, and using inheritance to reuse structure and behavior among multiple classes of objects. Beyond these two areas, reuse can create more cost than value.

The first step in reuse comes from isolating responsibilities to create classes of objects. Generally, a number of objects that have common structure and behavior is a kind of reuse. When these objects cooperate to achieve the desired results, this is sometimes called emergent behavior: no single class contains the overall functionality, it grew from the interactions among the various objects.

When the application grows and evolves, we can preserve some class declarations, reusing them in the next revision of the application. This reduces the cost and risks associated with software change. Class definitions are the most fundamental and valuable kind of reuse.

Another vehicle for OO resuse is inheritance. The simple subclass-superclass relationship yields a form of reuse; a class hierarchy with six subclasses will share the superclass code seven times over. This, by itself, has tremendous benefits.

We caution against any larger scope of reuse. Sharing classes between projects may or may not work out well. The complexity of achieving inter-project reuse can be paralyzing to first-time designers. Often, different projects reflect different points of view, and the amount of sharing is limited by these points of view. As an example, consider a product in a business context. An external customer's view of the product (shaped by sales and marketing) may be very different from the internal views of the same product. Internal views of the product (for example, finance, legal, manufacturing, shipping, support) may be very different from each other. Reconciling these views may be far more challenging than a single software development project. For that reason, we don't encourage this broader view of reuse.

On Design Patterns

These exercises will refer to several of the “Gang of Four” design patterns in Gamma95. The Design Patterns book is not a prerequisite; we use it as reference material to provide additional insight into the design patterns used here. We feel that use of common design patterns significantly expands the programmer's repertoire of techniques. We note where they are appropriate, and provide some guidance in their implementation.

In addition, we reference several other design patterns which are not as well documented. These are, in some cases, patterns of bad design more than patterns of good design.

Prev Up Next
Solution Approach Home Deliverables