About
Exploring Solution Spaces, Keith Ray's blog on Software development and other topics.

Send comments to:
keithray@mac.com

For Agile Training, eLearning, or Coaching contact:

Industrial Logic, Inc.

866-540-8336 (toll free)

510-540-8336 (Berkeley, California)

Links

xpminifaq

Résumé

“Adopting XP” Article 2002 (pdf)

“ Refactoring” Article 2006

AYE Conference

Lucien W. Dupont

Elisabeth Hendrickson

Johanna Rothman's Managing Product Development

Brian Marick's Exploration Through Example

Esther Derby's Insights You Can Use

Laurent Bossavit's Incipient(thoughts)

Dale Emery's Conversations with Dale

Martin Fowler's Bliki

Creating Passionate Users

Archives

Subscribe
RSS Exploring Solution Spaces XML

Don't Let Your Programmer Tests Turn Into Fairy Tales

Maybe you start out with:

void testLittleRedRidingHood()
{
    LittleGirl child = new LittleRedRidingHood();
    assertTrue(child.hasHood());
}

but then you add something more

void testLittleRedRidingHood()
{
    LittleGirl child = new LittleRedRidingHood();
    assertTrue(child.hasHood());
    assertTrue(child.hasBasket());
    Basket basket = child.getBasket();
    assertTrue(basket.hasGoodies()); 
        // we're kind of drifting away from LittleRedRidingHood now...
}

and more

void testLittleRedRidingHood()
{
    LittleGirl child = new LittleRedRidingHood();
    assertTrue(child.hasHood());
    assertTrue(child.hasBasket());
    Basket basket = child.getBasket();
    assertTrue(basket.hasGoodies());
    Wolf wolf = new BigBadWolf(); 
        // now we're really going astray...
    assertTrue(wolf.hasBigTeeth());
}

If your programmer test (a.k.a. "unit test") is telling a long story, it's probably is too long, not focused enough.

Step Into Your Agile Groove With Industrial Logic’s Greatest Hits

[/docs] permanent link

Spammers are using my email address

My email address used in this blog (k e i t h r a y @ m a c . c o m, not that anti-spam measures are going to help anymore) is being used as the return-address by spammers, so my inbox is filling up with automated spam rejection messages. I am not the one trying to sell you male enhancement drugs or devices, other "discounted" drugs, nor Florida real estate.

I hardly ever use that email address anymore, so I suppose I'll be shifting over to another address and closing that one down, sooner or later. Frakking spamers. :-(

[/docs] permanent link

Making a Waterfall Project Succeed

Many moths ago, I read an informative and often amusing book called It Sounded Good When We Started: A Project Manager's Guide to Working with People on Projects by Roy O'Bryan and Dwayne Phillips about (among other things) a waterfall hardware/software project whose management got disconnected from reality and the author had to take over project management to get the project back on track. Many of the techniques he used (putting the project plan on the wall as a frequently-updated information radiator, low-overhead daily updating of current and near-future planned tasks, avoiding delays between engineering activities and quality assurance activities) are also used in Agile projects.

I think of Agile development methods as including lightweight ways of incorporating feedback into the process. It turns out making a Waterfall project successful requires the same thing: frequent, timely feedback.

Step Into Your Agile Groove With Industrial Logic’s Greatest Hits

[/docs] permanent link

Jame Shore on What's the value of functional testing?

James Shore lightening talk (video) transcript quotes:

[...] it's about getting up in front of a whiteboard with your experts and saying, "let's talk about some examples of your domain. Let's look at what goes on." Whenever I've done this with customers, there's been this great "ah ha" of "oh my goodness, we didn't realize that we misunderstood each other so much." And this always happens before we start talking about Fit. This happens while we're actually at the whiteboard talking about examples. Fit is actually secondary to this process.

(Nowadays I describe Fit as providing a way of automating your examples. But you know what, you don't need Fit to do that. You get the power without using Fit. But if you use Fit without examples, it won't be that useful.)

So, to answer these questions: does it work, are we done, is it right, and minimize waste, I think we can use TDD, perhaps with exploratory testing to tell us if it works; involved customers to tell us if it's done; concrete examples to tell us if its right, and we might automate those examples.

Thus it seems to me that there's no need for [functional] testing; we can eliminate waste and keep it light.

Check it out

[/docs] permanent link

Elisabeth Hendrickson on Why Automated Acceptance Tests are Crucial

Elisabeth Hendrickson’s lightening talk (video).

"Automated Acceptance Tests are absolutely crucial to make sure that we know, we get the feedback right away, if we have violated an expectation that the customer had [...]. Furthermore, Customers don't want to have to repeat their expectations: [...] 'I have these expectations of the system, now it's the programmers problem to figure out how to automate those expectations to get that feedback.'"

[/docs] permanent link

Industrial Logic's Greatest Hits

Industrial Logic has a new home page, where you can browse and purchase eLearning products for C++, Java, and C# on the subjects of Code Smells, Refactoring, and Microtesting.

The front page also lets you Take a Tour and experience the Welcome Album, which includes blooper videos. Joshua Kerievsky, Mike Hill, and Gil Broza are the authors of this self-paced training material, as well as the instructors who use the same materials in Industrial Logic's in-person workshops.

Check it out.

[/docs] permanent link

Return of the Blog: Pairwise testing

I haven't been blogging much lately, but it's time to spill. Miscellaneous subjects follow.

Via Michael Bolton, a paper (pdf) "Pairwise Testing: A Best Practice That Isn't", by James Bach and Patrick J. Schroeder Quotes:

This article is about another apparent best practice that turns out to be less than it seems: pairwise testing. Pairwise testing can be helpful, or it can create false confidence. On the whole, we believe that this technique is over promoted and poorly understood. To apply it wisely, we think it’s important to put pairwise testing into a sensible perspective. [...]

Pairwise testing fails when you don’t select the right values to test with. [...]

Pairwise testing fails when you don’t have a good enough oracle. [...]

Pairwise testing fails when highly probable combinations get too little attention. [...]

Pairwise testing fails when you don’t know how the variables interact. [...]

The pairwise story is one that all testers would like to believe: a small cleverly constructed set of test cases can do the work of thousands or millions of test cases. As appealing as the story is, it is not going to be true in many situations. Practitioners must realize that pairwise testing does not protect them from all faults caused by the interaction of 1 or 2 fields.

[/docs] permanent link

What's the point of tests that pass all the time?

When I'm doing Test Driven Development, my tests (almost) always fail - at least once.

This is because I write a test and run it before I write the code to make the test pass. It should fail because the functionality isn't implemented yet.

I could design my code without using the tests to guide me, but then I find that I spend a lot of time debugging manually, and afterwards, I don't have tests to find regressions. I define regressions as bugs for already-implemented features which can get injected into the code during refactoring or when adding new features. So the tests in TDD are serving two purposes: guiding my design, and guarding against regressions.

However, I don't check into source-code-control when tests are failing, and I hope my team-mates do the same thing. So from the viewpoint of the team's continuous integration server, the programmer tests used for TDD should never fail.

Sometimes they do, because we make mistakes: we forget to check in something, or the tests are poorly written -- for example, a test might be accessing a live external server that isn't running all the time, or has inconsistent time-outs or other 'flaky' behavior.

The skills for TDD and writing reliable tests are more subtle than they ways of wizards, but they can be learned. [I've been watching Lord of Rings again.]

[/docs] permanent link

Const Assurance in C++

"splogs" is inspired to use 'const' more in C++ after exposure to Haskell. Quote:

[...] When a software developer is debugging a program (as s/he often is), just knowing that one "thing" will not change after creation decreases the scope of required knowledge for that session by one. If all things are immutable, that significantly reduces the probability that bugs are present in dynamic logic. Thus, by being "const-ipated" with your code (by using const wherever possible), you can kill bugs before they even appear!

An anonymous commenter responded:

I've found that striving for immutability in class design (also rubbing off from the functional versions where data types are immutable) also makes concurrency a lot easier and faster. For infrequently mutation, reconstructing parts of the object graph (to preserve immutability) can be a lot cheaper in the long run if it obviates the need for a lock in hot code paths.

Consider this bit of code:

Results aFunction(SomeClass& anObject)
{
    someval = anObject.SomeMethod();
    // later...
    someotherval = anObject.SomeOtherMethod();
    // etc.
    return results;
}

Given that anObject is not const, you'd have to assume that SomeMethod and SomeOtherMethod could be modifying the state of the object. If you changed the order of calls, or called one of those methods more than once, you could break the desired behavior. The only way to know for sure would be to look at the source-code and/or documentation for SomeClass's methods. Now with const-ness:

Results aFunction(const SomeClass& anObject)
{
    someval = anObject.SomeMethod();
    // later...
    someotherval = anObject.SomeOtherMethod();
    // etc.
    return results;
}

The simple act of declaring anObject to be const also implies that SomeMethod and SomeOtherMethod are const methods. (The compiler will complain if that's not true). This assumes the programmer for SomeClass was following the rules (that is, he's not 'faking' const-ness) and SomeClass isn't modifying some external state that we care about. In this case with a const reference and const methods, we could be pretty sure that we could change the order of method calls, and the number of times the methods are called, with no effect on the correct behavior.

Unfortunately, even with 'const' we can't always be sure the behavior is correct, so writing micro-tests helps enforce the intended behavior. A test like...

TEST(aFunctionDoesWhatWeWant)
{
    SomeClass anObject; // set up how we want
    Results fResults = aFunction(anObject);
    AssertEqual(expectedState, anObject.GetState());
    AssertEqual(expectedResults, fResults);
}

... would help us verify that the following changes are okay, if it still passes after changing the order and number of calls.

Results aFunction(const SomeClass& anObject)
{
    someval = anObject.SomeMethod();
    someotherval = anObject.SomeOtherMethod();
    // later...
    someotherval2 = anObject.SomeOtherMethod();
    someval2 = anObject.SomeMethod();
    // etc.
    return results;
}

Declaring a function parameter const also helps tell other programmers that the function isn't going to try to change the object's value. Tests will help document the function's desired behavior.

[/docs] permanent link

Some C++ Code Quality Metrics

• Purity metric: the ratio of pure virtual methods over the total number of methods of a class. Purity of 1.0 is good for an abstract base class. Purity of 0.1 might be ok for a class implementing the Template Method Design Pattern. Purity of 0.5 is likely to be bad because the class is going to be confusing. It is likely to be an abuser of "inheritance for implementation". Note that Template Method relies on subclassing from a class with a partially-concrete implementation, and might be better replaced by the something like the Strategy Design Pattern, which has a concrete class parameterized by injection of pluggable strategy objects (whose parent class is purely abstract).

• Public Interface metric: the ratio of public methods over the total number of methods of a class. A measure of 1.0 is probably good: all public methods, and no private methods (at least it's testable). A Public Interface number of 0.1 is likely to be bad - the class is consisting of mostly private or protected methods, and thus hard to test. A low number is a sign of the Iceberg Class , often indicating that the class is too large and should be broken up into several smaller, collaborating classes (after the break-up, maybe only one class is still part of the public API, but the other classes who now have public methods would be testable.)

[/docs] permanent link