About
Exploring Solution Spaces, Keith Ray's blog on Software development and other topics.

Send comments to:
keithray@mac.com

For Agile Training, eLearning, or Coaching contact:

Industrial Logic, Inc.

866-540-8336 (toll free)

510-540-8336 (Berkeley, California)

Links

xpminifaq

Résumé

“Adopting XP” Article 2002 (pdf)

“ Refactoring” Article 2006

AYE Conference

Lucien W. Dupont

Elisabeth Hendrickson

Johanna Rothman's Managing Product Development

Brian Marick's Exploration Through Example

Esther Derby's Insights You Can Use

Laurent Bossavit's Incipient(thoughts)

Dale Emery's Conversations with Dale

Martin Fowler's Bliki

Creating Passionate Users

Archives

Subscribe
RSS Exploring Solution Spaces XML

The Story of the Graphing Calculator

http://www.pacifict.com/Story/

... I used to be a contractor for Apple, working on a secret project. Unfortunately, the computer we were building never saw the light of day. The project was so plagued by politics and ego that when the engineers requested technical oversight, our manager hired a psychologist instead. In August 1993, the project was canceled. A year of my work evaporated, my contract ended, and I was unemployed.

I was frustrated by all the wasted effort, so I decided to uncancel my small part of the project. I had been paid to do a job, and I wanted to finish it. My electronic badge still opened Apple's doors, so I just kept showing up. ...

... After a demo to outside developers, one person called Apple claiming that we infringed his patent, causing a fire drill until I could show prior art. Another company, the makers of Mathematica™, simply demanded that our product be pulled. Apple very politely declined. One week we were evading security, the next week Apple is rising to our defense...

... I view the events as an experiment in subverting power structures. I had none of the traditional power over others that is inherent to the structure of corporations and bureaucracies. I had neither budget nor headcount. I answered to no one, and no one had to do anything I asked. Dozens of people collaborated spontaneously, motivated by loyalty, friendship, or the love of craftsmanship. We were hackers, creating something for the sheer joy of making it work.

[/docs] permanent link

Kent Beck Test Talk

Developer testing as driver of good design / "healthy software", and as a measure of accountability. http://www.itconversations.com/shows/detail301.html

This presentation was recorded at the Developer Testing Forum held in Palo Alto, California, November 17th, 2004. Thanks to Agitar Software and the SDForum, the producers of the event.

[/docs] permanent link

Fitnesse Python

I'm catching on reading the Fitnesse mailing list. Here's a page describing writing automated acceptance tests in PyFit: http://agiletesting.blogspot.com/2004/11/writing-fitnesse-tests-in-python.html

[/docs] permanent link

BayXP meeting Dec 15, Foster City CA

Our guest speaker will be III (pronounced "three"), an authority on Chartering. Project Chartering is one of the practices of Industrial XP. III advocates chartering for providing a good beginning for a any project. He will present examples from chartering done at HP for their XP-SIG. Check out the paper he wrote on chartering here: http://industriallogic.com/papers/Chartering.pdf

Project Chartering helps people answer questions like: Is the idea for the project worthwhile? How does the project further the organization's vision/mission? How would we know if the project is a success? Who is part of the project's Project Community?

Badging is required, so please try to arrive early.

Meeting Location:

Offices of Electronics for Imaging, Inc.
303 Velocity Way
Foster City CA 94404

Time: November 17th 7:00 PM mixing; 7:30 PM - 9:30 PM presentation

[/docs] permanent link

Reading Software

We programmers read software much more often than we write it, but do we write for readability? Do we teach programmers how to read software?

[/docs] permanent link

Learning, Doing, Articulating

Some people learn by doing. Some people (like me) learn by reading. I've been doing C++ for a long time, but I would say that I learned it by reading the C/C++ Users Journal and various books and other documentation. The parts of C++ that I learned "by doing" have to do with compiler limitations and library bugs.

While the syntax and usage of a C++ is a fairly limited set of knowledge; good object-oriented design is much less limited. No set of books can cover everything, particularly not the compromises that often occur in real-world projects. So I've been learning OO by doing as well as by reading.

The third stage is taking the implicit / tacit knowledge acquired by doing, and articulating it. Not just to teach other people, nor to prove to others that you've acquired this knowledge, but to examine and re-examine what you think you know.

When asked questions about object-oriented design, I sometimes find it difficult to articulate everything that could or should be said... not because I don't know it, but often because something is so obvious to me that I would not think that I would need to mention that aspect.

For example, many people avoid multiple-inheritance in C++, but when asked why, they just say it's "bad" or "dangerous". While there can be problems with "diamond" inheritance (where multiple parents of a class have a common parent), multiple inheritance isn't all bad. Being able to "mix-in" Observer/Observable functionality into specific "child" classes in your class hierarchy might be an appropriate use of multiple inheritance. An aspect of multiple-inheritance that I would consider too obvious to mention (but I'll mention here) is that it can't be changed at run-time... it's inflexible (and 'abusive' to the concept) if you're trying to use inheritance to implement aggregation. You're not going to decide at run-time whether to add Observable behavior to some object, but you may want to change or add other behaviors at run-time: appropriate patterns for that would be Decorator, Composite, Delegation, Strategy, etc...

Because not everyone is at the "articulating" stage of knowledge, use auditions to evaluate programmers: pair-design and pair-program with the candidate on real-world problems, for example, keeping in mind that really good insights on various problems will take more than an hour or two...

[/docs] permanent link

BayXP

The next meeting of BayXP will be Wednesday Nov. 17, 2004 (a week early to avoid Thanksgiving vacations). Jeff McKenna of www.netobjectives.com will be speaking informally on the planning game and retrospectives -- how they work together.

Badging is required, so please try to arrive early.

Time: November 17th 7:00 PM mixing; 7:30 PM - 9:30 PM presentation

Meeting Location:

Offices of Electronics for Imaging, Inc.

303 Velocity Way

Foster City CA 94404

[/docs] permanent link

Really Optimized Memory Allocation

Travis Griggs tested memory allocation and freeing in C (no garbage collection versus automatic garbage collection) and Smalltalk. His test generates and frees 20,000,000 small objects, with only 1,000 of them being "live" at any time. His results:

9.5 seconds: plain C allocing but never freeing [no gc]

9.5 seconds: plain C with Boehm Garbage Collector

1.0 seconds: Cincom VisualWorks Smalltalk

This goes to show that 20 years of people improving Smalltalk's performance does add up. Unfortunately for C programmers, the fast algorithms used in "Generation Scavenging"(pdf) involve periodically copying small live objects from "new space" to "old space" -- this requires "fixing up" pointers to point to the new location, or the use of "handles" (pointers to pointers). The Boehm garbage collector is not going to fix up pointers, because it must assume that some piece of memory that looks like a pointer might actually be something else. [And C programmers have been known to copy pointers into int variables, etc.] A Java or Smalltalk environment can distinguish between object pointers and other things that may have the same bit-pattern as a pointer, so fixing up object pointers is safe.

The speed of "Generation Scavenging" comes from the simplicity of allocating objects in "new space" -- just increment the pointer that marks the division between the used and un-used part of the "new space" heap (it's more like a stack that never has to be decremented.) In some systems (Sun's HotSpot Java VM?) there's one "new space" per thread to avoid having to synchronize heap access across multiple threads. When the "new space" is filled up, do garbage collection and move "live" objects to "old space" heap (fixing up pointers or handles as needed). Most objects in Smalltalk have short lifetimes, so very few objects are copied. Then reset the "new space" pointer to clear the "new space" heap. Less often, do garbage collection to free up objects in the "old space" heap.

By the way, when I worked at Apple, I improved the performance of malloc/calloc/free in the standard-C library that shipped with (and was used by) the MPW environment and the MrC and MrCpp compilers. Simplifying the code and tuning how it did allocations of small objects improved the speed of the compilers by 5% or more. The changes I made to that library are still shipping in "Classic" MacOS.

[/docs] permanent link

Debate instead of Dialog 2

More advice for dealing software people: if you offer an idea or suggestion to "debaters" (see previous blog entry), and they poke a dozen holes in it, that means that they're interested in it. Try to imagine that they are trying to help you, because many times this is their way of doing that.

To move from debate into dialog, ask them to how to resolve one of the problems they just brought up. This will (if they are interested) switch their mode to "problem-solver", which is one of their favorite modes.

[/docs] permanent link

Debate Instead of Dialog

Many people in the software biz respond to an attempt at conversation with a rebuttal. This isn't dialog, it's debate. It's one of the reasons software developers have developed a reputation for poor social skills. Do you do this? Here's a symptom: your replies to someone else often start with the word "but". Examples:

Dialogger: "It was raining this morning."

Debater: "But it's not raining now."

Dialogger: "We could have found this bug a long time ago if we had written a unit test for [some condition]."

Debater: "But unit tests can't find all bugs."

By starting with "but", you don't give credit to the other person for their idea, even if you agree with it. Try restating their ideas before responding with your ideas or disagreements.

[/docs] permanent link

Hard to be re-usable?

There is a meme going around that it is "an order of magnitude more difficult" to write a class that is "re-usable". That didn't seem right to me, so I decided to list the rules one could follow to write re-usable classes. Once I got up to about nine or ten rules, I decided that it might not be as simple as I thought. Still, compared to all the rules one must know to write safe C++ code, these rules are not that hard. The following list isn't numbered the way I originally wrote them -- it's 7 rules here, but at some point you get into the rules of good design, the rules for using C++ or Java or whatever robustly, etc. so you could go further and make the list longer.

By the way, writing code test-first tends to make code much more re-usable, because the code is "used" at least twice - one or more times in the tests, and one or more times in the production code. Note: the previous statement is an understatement. If you as the 're-user' have access to the source code, and can refactor code as needed, then you don't have to follow all these rules initially. For example Rule 6 below doesn't need to be followed unless you determine that you need to do this to make re-use work, then you can refactor to make it work.

One obstacle to re-use is class dependencies and package dependencies. Once upon a time, I wanted to use a particular Java class, but it used so many other classes in the same package, that I was forced to add the entire package to my project. But that package used classes in a bunch of other packages. In order to use that single class, I would have had to include at least six packages and a hundred classes in my project. So...

Re-use Rule 1: Write your class to avoiding depending on other classes (etc). Obviously that can't always be done. Rule 1a: Try to use only the "basic" classes that everyone uses. More rules to come also address problems that contribute to being forced to grab a lot of classes just to use one class. Rule 1b: avoid writing "too-big" classes and "too-big" packages.

Re-use Rule 2: a class should have one responsibility, delegating other responsibilities to other classes. Ditto for packages. Symptoms of having too many responsibilities are groups of methods that each only deal with isolated member variables, and having lots of private (or protected) methods. Symptoms of too-big packages are classes that don't depend on other classes within the package. Classes that are unrelated to whatever "concept" the other classes implement. Packages named "misc". Just because a bunch of classes implement an interface or subclass from a certain class, it doesn't necessarily mean that they belong together in the same package, or even belong in the package in which the interface or parent class resides.

Re-use Rule 3: a class should not depend on other concrete classes [except the most basic types like String]. By using interfaces or abstract classes as the "type" of member variables / return values / arguments to methods and constructors, we enable easier re-use (and testing): instead of the passing in the concrete types that were expected, we can pass in new concrete types useful in the new "re-use" context. Take advantage of the power of polymorphism and all that (or at least don't prevent others from doing so). Dynamic languages like Python, Smalltalk, Ruby and so on, which don't declare the types of variables, don't have this problem.

Re-use Rule 4: To implement rule 3 and to avoid circular package-dependencies (and to allow re-use without dragging in lots of packages), the interface, the class using that interface, and the concrete classes that implement that interface, probably need to reside in three different (and fairly small) packages. Package design regarding interfaces and concrete classes isn't discussed in many places, but it is discussed in Robert Martin's book "Agile Software Development".

Re-use Rule 4: to further allow substitution of concrete types, avoid using "new ConcreteClass" within your class -- particularly the constructors. If you do need to create objects of particular types (instead of having those objects passed into methods and constructors), put the "new" call into a "creation method". This allows the re-user of that class to override that method to instantiate a different kind of object. This is useful for creating Mock objects in unit-tests.

Re-use Rule 5: the user of your class may want to subclass-and-override. Make that possible by avoiding static, final, or non-virtual methods, and private methods. Private methods either need to be declared protected, or should be moved to another class and made public, with the original class using an instance of that other class.

Re-use Rule 6: You can make member variables protected, or make member variables private. If you make them private, you should implement protected (or, if absolutely necessary, public) accessor methods to get/set the member variables. A subclass can override those accessor methods to extend or modify behavior associated with getting/setting, including returning different objects. If you go this route, then no method or constructor should ever access those private variables directly except the accessor methods themselves.

Re-use Rule 7: In event that re-use of your class would involve overriding ALL of the methods, and ignoring ALL of the member variables, declare an interface and make your class implement that interface. Now instead of overriding every method in your class, the re-user can make their own implementation of your interface.

C++ makes following some of these rules difficult. For example, in a constructor, method calls to virtual functions act like the method is not declared virtual. In the constructor of class "Parent", calls to virtual accessor methods to set member variables will not invoke overriding accessor methods implemented in class "Child." In this case, you would need to do the work of setting member variables via virtual accessor methods in an "Initialize" method, which must NOT be called from the constructor. For example: "Parent* obj = new Child; obj->Initialize();". Don't forget to declare destructors virtual. C++ allows declaring argument types "by value" as well as "reference" and "pointer". "By value" doesn't allow polymorphism because it creates a new object of the declared type on the stack, possibly copying only a subset of the members from a sub-class object passed into that method call.

Following all of these rules all of the time may create ugly code. Make classes re-usable "as needed". Develop test-first so that you can later refactor classes for re-use at any time, protected from unintended bugs by suites of unit tests. There is a concept for class hierarchies of "Object-Oriented Normal Form" in Michael Feather's book "Working with Legacy Code" summed up by the rule: "No child class's methods override any parent class's concrete methods." Any time you would be tempted to do that, you instead restructure the inheritance hierarchy, adding abstract classes or interfaces so that you only override unimplemented abstract/interface methods ("pure virtual" methods in C++.)

Here's an example of following these rules in C++:

class Region   // the "not very re-usable" version of this class
{
public:
    Region();
    Region( Rect shape ); // "by value" argument  
    ~Region();  // not virtual, making subclassing dangerous
    void Clear(); // not virtual, can't be overridden.
    void Union( Rect anotherShape );
    void Intersect( Rect anotherShape );
    Bitmap ConvertToBitmap();
private:
    Bitmap myBitmap;
};

// Now the more re-usable version...

class RegionInterface
{
public:
    virtual ~RegionInterface();
    virtual void Clear();
    virtual void Union( const RectInterface& );
    virtual void Intersect( const RectInterface& );
    virtual BitmapPtr ConvertToBitmap();
};

//BitmapPtr is a smart pointer declared something like this:
//typedef std::auto_ptr < BitmapInterface > BitmapPtr;

class Region : public RegionInterface
{
public:
    Region(); 
    virtual ~Region(); // "virtual" necessary for "delete (RegionSubclass*) obj" to work right.
    virtual void Initialize( const RectInterface & ); // because we can't call virtuals in the ctor
    virtual void Clear();
    virtual void Union( const RectInterface& );
    virtual void Intersect( const RectInterface& );
    virtual BitmapPtr ConvertToBitmap();
protected:
    BitmapPtr GetBitmap() const;
    void SetBitmap( BitmapPtr );
private:
    BitmapPtr myBitmap;
};

Note that doing

    Region* reg = new Region;
    reg->Initialize( aRect );

more than once in the code-base is repeated code [a bad thing], so you would want to put those two lines into a creation-function (probably a static method in class Region). I've also left out copy constructors and assignment operators.

One last aside... look above at how much "overhead" C++ requires. In dynamic languages like Smalltalk, the only difference between the "less re-usable" class and the "more re-usable" class would be the addition of the accessor methods (which I understand Smalltalkers do by habit already.) Here's the same example in a dynamic "c-ish/pascal-ish" language I just made up:

class Region   // the "not very re-usable" version of this class
{
public:
    ctor Region();
    ctor Region( shape );
    proc Clear(); // not virtual, can't be overridden.
    proc Union( anotherShape );
    proc Intersect( anotherShape );
    func ConvertToBitmap();
private:
    var myBitmap;
};

class Region // more re-useable version 
{
public:
    ctor Region();
    ctor Region( shape );
    proc Clear(); // not virtual, can't be overridden.
    proc Union( anotherShape );
    proc Intersect( anotherShape );
    func ConvertToBitmap();
protected:
    func GetBitmap() const;
    proc SetBitmap( aBitmap );
private:
    var myBitmap;
};

If you developed test-first, you would have pretty much the same set of tests for this class in a dynamic language as you would in a static language, but much less overhead text.

[/docs] permanent link

Presidents and Stock Market

Is there a pattern to stock market activity relative to who is president? I got year-end Dow Jones numbers from this web-site (http://www.cfainstitute.org/pressroom/overview/cfa40yr_timeline.html) and graphed them. The only pattern I've noticed is that for most presidents (Nixon, Ford, Carter, Reagan, GWBush), the Dow ends lower after their first year in office. The numbers and graphs are here: http://homepage.mac.com/keithray/dowJonesPresidents.xls.pdf

[/docs] permanent link

Books To Read

I've gotten a back-log of books to read, with more arriving Real Soon Now. If you're developing software in Java, C++, C#, or some other object oriented language, you need to buy these books for your professional toolkit.

Refactoring to Patterns by Joshua Kerievsky

Working Effectively with Legacy Code by Michael Feathers

JUnit Recipes: Practical Methods for Programmer Testing by J.B. Rainsberger and Scott Stirling

Pragmatic Version Control Using CVS by Dave Thomas and Andy Hunt

Pragmatic Project Automation: How to Build, Deploy, and Monitor Java Apps by Mike Clark

Pragmatic Unit Testing in Java with JUnit by Dave Thomas and Andy Hunt

Pragmatic Unit Testing in C# with NUnit by Dave Thomas and Andy Hunt

Extreme Programming Adventures in C# by Ron Jeffries

In a completely different area, I recommend this as well: Moral Politics : How Liberals and Conservatives Think by the cognitive linguist George Lakoff.

[/docs] permanent link

Knowledge Creation Activity

Writing software to do something that hasn't been done before is a knowledge creation activity. In many respects, it is like artistic creation. And therein lies the rub. Artistic creation is a messy activity involving trying some things and throwing away the things that don't work. Inspiration does not respect age or seniority. Sparks fly from collaborative agreement and disagreement, but not from command-and-control.

Can you imagine a group of people composing music in an hierarchical chain of command: a chief composer, with middle-manager-composers, and squads of junior composers? "Sorry boss, we tried to make the eighth-note triplets work like your plan, but the rhythm's off." The boss replies: "You gotta make them work. We have a schedule, and the base line needs to be integrated with your part by the end of this week -- and we don't have time to do it over."

Of course, writing software isn't just like writing music, nor is it just like anything else that isn't writing software. In some environments, doing things that are very similar to what has been done before, a team can create or customize software predictably and without much fuss. Still, even on those projects, on the level of an individual's or pair's effort for any one hour, there will probably be some creativity, some trying things, some throwing away of ideas that don't work, some sparks and inspiration.

[/docs] permanent link

Paper on Dynamic Languages

Good paper - Dynamic Languages-ready for the next challenges, by design by David Ascher. Note that Smalltalk is one of the first dynamic languages, and lives on in multiple commercial and open-source versions. And don't forget Objective-C, which was initially commercial but now is open-sourced. [Aside: why don't the maintainers of gcc take the changes that Apple made to support their blend of Objective-C and C++? Apple made the changes available, but the gcc people didn't integrate them.]

[/docs] permanent link

Reasons to do Unit Testing / TDD / Automated acceptance tests

1. There are studies/statistics that say unit testing finds 70% of the defects.

2. A unit test provides an example of what a piece of code is supposed to do. Going through the code in the debugger only tells you what it is doing, not what it is supposed to do.

3. You can run hundreds of unit tests in the same amount of time it takes to go through one function in the debugger.

4. Unit tests as change-detectors - you've changed one line of code in a million-line system. Did you break anything? Run the automated unit and acceptance tests and find out. (If you've done strict TDD, your tests have nearly 100% code coverage.)

5. Unit tests as documentation -- multiple examples of what the code is supposed to do, and how it should handle failures.

6. Test-driven development is a thinking/designing process that happens to leave you with a bunch of tests. You can do the same thinking/designing process without TDD, but you don't end up with any tests.

7. Exploration. You can try out an unfamiliar API in a test, preserving a record of what that API does (and notifying you later if the API's behavior changes.)

8. Making code testable improves its design - making it more modular and encouraging you to make it more cohesive.

Given all these points (especially points 1 and 4), why would any company that is trying develop software to satisfy customers demands allow its programmers to NOT do unit testing and automated acceptance testing?

[/docs] permanent link

Links - Advice To Managers, Lean Development

Jerry Weinberg's Advice for Software Development Managers.

Mary Poppendieck on increasing productivity [profits], quality and speed archived web-interview and slides.

[/docs] permanent link

Project Censored

Top 25 new stories of 2003-2004 that you didn't see: Project Censored.

(#4) High Uranium Levels Found in Troops and Civilians quotes:

Uranium is a chemically and radiologically toxic element, clinically proven to be a cause of various types of cancer and congenital malformations (birth defects). Internal contamination of uranium is responsible for variety of systemic and organ system problems [...].

Most American weapons (missiles, smart bombs, dumb bombs, bullets, tank shells, cruise missiles, etc.) contain high amounts of radioactive uranium. Depleted or non-depleted, these types of weapons, on detonation, release a radioactive dust which, when inhaled, goes into the body and stays there.[...]

[...] because radioactive contaminants from uranium weapons travel through air, water, and food sources, the effects of U.S. deployment in Afghanistan will be felt in Iran, Pakistan, Turkey, Turkmenistan, Uzbekistan, Russia, Georgia, Azerbaijan, Kazakhstan, China and India. Countries affected by the use of uranium weapons in Iraq include Saudi Arabia, Syria, Lebanon, Palestine, Israel, Turkey, and Iran.

Uranium is preferred over all other "ballistic" metals [...] because it offers a set of unique metallurgical properties: it is extremely dense yet ductile metal[...] uranium dust burns spontaneously at room temperature [...]. Uranium metal has a very unusual property not available in any other metal; it is "self-sharpening", meaning that when it hits a target at high velocities (1 km/sec) it erodes and breaks in such a way as to continuously re-sharpen its point[...].

In 2003 scientists from the Uranium Medical Research Center (UMRC) studied urine samples of Afghan civilians and found that 100% of the samples taken had levels of non-depleted uranium (NDU) 400% to 2000% higher than normal levels. The UMRC research team studied six sites, two in Kabul and others in the Jalalabad area. The civilians were tested four months after the attacks in Afghanistan by the United States and its allies.

NDU is more radioactive than depleted uranium (DU), which itself is charged with causing many cancers and severe birth defects in the Iraqi population–especially children–over the past ten years. Four million pounds of radioactive uranium was dropped on Iraq in 2003 alone. Uranium dust will be in the bodies of our returning armed forces. Nine soldiers from the 442nd Military Police serving in Iraq were tested for DU contamination in December 2003. [...] the test found that four of the nine men were contaminated with high levels of DU, likely caused by inhaling dust from depleted uranium shells fired by U.S. troops. [...].

Sgt. Hector Vega, Sgt. Ray Ramos, Sgt. Agustin Matos and Cpl. Anthony Yonnone from New York's 442nd Guard Unit—they are the first confirmed cases of inhaled uranium oxide exposure from the current Iraq conflict. Dr. Asaf Durokovic, professor of Nuclear Medicine at the Uranium Medical Research Centre http://www.umrc.net/ conducted the diagnostic tests. The story was released April 3, 2004 in the New York Daily News. There is no treatment and there is no cure. [...]

[...] Today, more than 240,000 Gulf War veterans are on permanent medical disability and more than 11,000 are dead. They have been denied testing, medical care, and compensation for depleted uranium exposure and related illnesses since 1991.

[...] Even worse, they brought it home in their bodies. In some families, the children born before the Gulf War are the only healthy members. Wives and female partners of Gulf War veterans [...] are now internally contaminated from depleted uranium carried in the semen of exposed veterans. Many are reporting reproductive illnesses such as endometriosis. In a U.S. government study, conducted by the Department of Veterans Affairs on post-Gulf War babies, 67% were found to have serious birth defects or serious illnesses. They were born without eyes (anophthalmos), ears, had missing organs, missing legs and arms, fused fingers, thyroid or other organ malformations.

[/docs] permanent link

Jigsaw Puzzle Exercise

To illustrate to managers/executives the differences between waterfall, code-n-fix, and iterative/incremental development...

Prepare for the demonstration by buying three small jigsaw puzzles. Assemble two of them, and turn them over (while still assembled.) For the puzzles that represent waterfall and iterative/incremental development, find groups of three connected puzzle pieces and label them as follows: "requirements 1", "code 1", and "tests 1" for the first set of three. "Requirements 2", "code 2", and "tests 2", and so on for an adjacent set of three pieces, and so on. Make sure that not all of "requirements" pieces are directly connected to other "requirements" pieces, etc. Now it should be easier to put together these puzzle again... but that's not all.

To dissemble the puzzles... For the "waterfall" puzzle: put all of the "requirements" pieces in one bag, all of the "code" pieces in a second bag, and all of the "test" pieces in the the third bag. For the "iterative/incremental development puzzle" you'll need lots of little bags, one for the "1" pieces, one for the "2" pieces, and so on. Label each of the bags (or use little little tiny ziplock bags that you can see through).

Now for the exercise with the managers/executives. Divide them into three groups. Their assignments are to put together a puzzle within a time limit (say 20 minutes). The group that gets done first is the winner, or the group that has the most contiguous pieces assembled.

For the "waterfall group", give them all of the "requirements pieces" immediately, wait 5 minutes, and then give them the "code" pieces, wait another 5 minutes, and then give them all the "test" pieces.

For the "iterative/incremental development group," give them the first bag immediately, give them the second bag as soon as they've assembled the pieces from the first bag, and so on.

For the "code-n-fix" group, give them the puzzle that you haven't marked up... maybe withholding a few random pieces to make it a little more challenging. ;-)

If you need strict scoring conditions, each connected piece counts as 1 point. Each missing piece from within a group of connected pieces subtracts 1 point. If there are more than one group of connected pieces, subtract 1 for each group besides the first.

Being JustaProgrammer, (and being poor at jigsaw puzzles) I haven't (yet) done such an exercise. I offer this exercise on a non-exclusive license to anyone who wants to try it, with the requirement that you send me a description of how this worked out in practice.

I apologize for not blogging lately. Things are busy at work. My laptop was having problems and I spent some time transferring files from the old one to the new one. My dog ate my homework. :-) I'll be back to the test-first "From Model to Model-View-Controller" paper real soon.

[/docs] permanent link

typo correction

correction for string-list

typedef std::list<std::string> StringList;

[/docs] permanent link

Part 1 - Test Driven Development from Model to Model-View-Controller

This is an example of test-driven development. We're going to implement a simple calculator object test-first and hook it up to a user-interface. First thing we need to do is #include the test framework. I'm using my version of Michael Feather's CppUnit.

#include "TestDrivenDesign.h"
#include <string>
#include <list>

Now define our first test. I like to put these into a namespace.

namespace SimpleCalcTests
{
    DECLARE_TEST(CalcDisplayStartsEmpty);
    DECLARE_TEST(CalcHistoryStartsEmpty);

    DEFINE_TEST(CalcDisplayStartsEmpty)
    {
        SimpleCalc aCalculator;
        assertStringsEqual( "", aCalculator.GetDisplay() );
    }

    DEFINE_TEST(CalcHistoryStartsEmpty)
    {
        SimpleCalc aCalculator;
        StringList astringList = aCalculator.GetHistory();
        assertLongsEqual( 0, astringList.size() );
    }

    TestSuite* Suite()
    {
        TestSuite* result = new TestSuite("SimpleCalcTests");
        result->addTest( new SimpleCalcTests::CalcDisplayStartsEmpty );
        result->addTest( new SimpleCalcTests::CalcHistoryStartsEmpty );

        return result;
    }
}

So I've decided that a calculator has a display representable by a string, and a history that is a string-list. In order to get this to compile, I better define the minimal classes required. Normally you only implement one test at a time, but these are two very simple tests. Slow C++ compile-link-run turn-around-time tends make you want to batch things up. Be careful to keep the batches small.

typedef std::list StringList;

class SimpleCalc
{
public:
    SimpleCalc()
    {
    }

    virtual ~SimpleCalc()
    {
    }

    virtual std::string GetDisplay() const
    {
        return "display";
    }

    virtual StringList GetHistory() const
    {
        StringList result;
        result.push_back( "history" );
        return result;
    }
};

Note that the initial implementations of GetDisplay() and GetHistory() are going to make the test fail. That is intentional; I want the tests to inially fail in order to test the tests . I also initially wrote a single test, but changed it to two tests, so that each test only tests one thing. It's too easy to get false negatives or false positives if a test is testing multiple things. If you can't do the "right thing" when things are simple, what's going to make you do the right thing when thigns are more difficult.

And why did I make the destructor and other methods virtual? First, it's a habit I've developed - if there is inheritance and the destructor and other methods are not virtual, you can get certain bugs that can be hard to debug. Second, I'm doing Object Oriented Programming. Polymorphism is a big part of that, and C++ does run-time polymorphism by declaring things virtual. Third, NOT declaring a destructor or a method virtual is a form of optimization... and optimizing code before you have hard evidence that it needs it is "premature optimization". Just don't do it.

I also need to declare a test-runner and main function...

int main (int argc, char * const argv[]) 
{
    TestRunner calcTests("CalcTests");
    calcTests.addTest( SimpleCalcTests::Suite() );
    calcTests.runAllTestsOnce();

    return 0;
}

Here's the output from this code so far:

TestSuite start CalcTests
    TestSuite start SimpleCalcTests
        TestCase CalcDisplayStartsEmpty...  AssertionFailedError 
            'expected: '' but was: 'display'', line 77, 
            file 'simplecalc/main.cpp'
            AssertionFailedError 
            FAIL 0 seconds
        TestCase CalcHistoryStartsEmpty...  AssertionFailedError 
            'expected: 0 but was: 1', line 85, 
            file 'simplecalc/main.cpp'
            AssertionFailedError 
            FAIL 0 seconds
    TestSuite end SimpleCalcTests
TestSuite end CalcTests

!!!FAILURES!!!
Test Results:
Run:  2   Failures: 2   Errors: 0
There were 2 failures: 
    1) line: 77 simplecalc/main.cpp "expected: '' but was: 'display'"
    2) line: 85 simplecalc/main.cpp "expected: 0 but was: 1"

This output is a little verbose because at various times using this test-suite I have needed to see where crashes were occurring (thus output about which suite and test are beginning to run and finishing running) and where slow code/tests were. Yes, I do care about code-speed, but I measure before I optimize.

Now that I have the tests failing, time to make them pass. In this case, by permitting the display string to get empty and the history list to be empty. I'll demonstrate the use of a test-suite. (TestRunner is a sub-class of TestSuite, by the way.)

class SimpleCalc
{
public:
    [other code unchanged ...]

    virtual std::string GetDisplay() const
    {
        return "";
    }

    virtual StringList GetHistory() const
    {
        StringList result;
        return result;
    }
};

Here's the output of the passing tests. It's much more succinct.

    TestSuite start CalcTests
        TestSuite start SimpleCalcTests
            TestCase CalcDisplayStartsEmpty...  0 seconds
			TestCase CalcHistoryStartsEmpty...  0 seconds
		TestSuite end SimpleCalcTests
	TestSuite end CalcTests
	OK( 2 tests)

Part two will get into the meatier part of the calculator.

[/docs] permanent link

tips for venting "computer rage"

If your robot or computer is giving you trouble (quote from Douglas Adam's Sirius Cybernetics Corporation song: And when it breaks down
Or starts to annoy
Or grinds when it moves
And gives you no joy
Cos it's eaten your hat
Or had sex with your cat
Bled oil on your floor
Or ripped off your door
), just pick some of Dr. Smith's robot insults to describe the robot or computer to your satisfaction: http://www.geocities.com/SunsetStrip/Towers/3332/smith.html. Here are a few good ones:

Arrogant Automaton
Automated Oaf
Blundering Bag of Bolts
Computerized Clod
Cybernetic Simpleton
Digitised Dunce
Hulking Mass of Mechanical Ignorance
Mechanized Misguided Moron
Obsolete Piece of Scrap Metal
Sorry Specimen of Computerhood
Traitorous Tin-Plated Fugitive from a Junkyard
Traitorous Transistorized Toad
Worthless Electronic Scrapheap
Tyrannus Thesaurus

[/docs] permanent link

Engineering

I would say that most people think "engineering" as designing using mathematical principles -- a [1] structural engineer who uses equations from handbooks to verify that the building the architect designed won't fall down. [2] Or a chemical engineer who uses equations from handbooks to verify that X amount of heat will be generated from a chemical process that takes in W and Z amounts of various chemicals that will be used in a factory to create certain plastics. [3] The "Army Corp of Engineers" is mostly seen as building dikes and bridges, not as designing solutions.

To argue that software development is engineering will result in arguing whether any mathematical principles apply to the design, to what extent, and also bring in confusion about manufacturing and construction, and reinforce waterfall notions of design, then implement, then test. While there certainly are some mathematical principles involved in computer science, that isn't the problem we have in developing software.

To instead argue that software development is a cooperative "game" in creating and deploying "knowledge" and various people-oriented practices help make that work (which allows talking about getting customers, testers, and developers talking instead of writing documents, and inverting design-implement-test waterfall to XP-style test, then implement, then design) gets the conversation away from arguing about definitions on "engineering" that neither participant really understands (and analogies to construction that really don't apply).

In my examples 1,2, and 3, the "engineer" doesn't need to talk to "customer" much at all. In [1], the architect talks to the customer so the structural engineer doesn't have to. In [2], there probably is some direct communication, but possibly not if a "plant process designer" has just delegated some work to the chemical engineer. In [3] the conversation is limited to "stop this river from flooding this subdivision" or "get those tanks across that river" -- the amount of "knowledge" to be generated is very limited compared to the literally millions of decisions necessary to have a team write a million lines of source code.

[/docs] permanent link

Burn Charts

Alistair Cockburn's chapter on project-tracking using burn-up charts, burn-down charts, "iceberg lists", and earned-value: http://alistair.cockburn.us/crystal/articles/evabc/earnedvalueandburncharts.htm

[/docs] permanent link

Bad APIs 2

An alternative implementation of the function alloc(size_t size) could be defined to return a pointer to a block of memory that has the size at the front of it. In fact, this is how many implementations of these library functions are implemented -- but it wasn't standardized, and it wasn't made typesafe -- on those implementations where this was done, alloc would return a pointer to the data portion of the block, and you would have to "back up" and typecast the pointer to get the length portion of the block. A semi-typesafe example:

struct MemoryBlock
{
    size_t         block_size;
    unsigned char  block_data[];   // unbounded array
};

MemoryBlock* Alloc( size_t sizeIn );

void Free( MemoryBlock* blockIn );

While this has the benefit of less space on the "client" side of the function (retaining one pointer instead of a struct with two fields) the danger of this style is that heap corruption is more likely - bad code writing past either end of the data field can erase those block_size fields.

Consider all the debugging tools we have today - would we need Purify, SmartHeap, and all those other tools as much if library functions like alloc and free had been implemented in a more object-oriented fashion?

I'm currently trying to debug some code (not written by me) that mixed C and C++ memory-management: it used "free" on memory allocated by "new char[]" and "delete []" on memory allocated by "alloc". I think these functions were smart enough not to work with memory that the other set of functions allocated, so there were memory leaks. In some implementations, these functions might be safely interchangeable for non-object allocations, in other implementations you can expect crashes.

So I fixed the code to be consistent, but I'm also getting crashes because some C++ objects are being deleted more than once [messing up the heap]. Apple's DebugMalloc provides some help in determining when the heap is messed up, but the irony is that DebugMalloc's leak detection feature works by implementing a form of garbage collection - it reports a leak when a memory allocation is no longer being pointed to. If I had real garbage collection, (or if the original coder had used boost::shared_ptr) I wouldn't have to worry about double-deletes.

[/docs] permanent link

BAd APIs

Bad APIs start at the very beginning. For example, the C function alloc(size_t size) - a function to allocate memory in the heap - creates a block of memory; but what does it return? Not a data-structure describing the block, but half of that information, just the pointer. Imagine what could have happened if that function has been defined correctly:

struct MemoryBlock
{
    char*  block_start;
    size_t block_size;
};

MemoryBlock Alloc( size_t sizeIn );

void Free( MemoryBlock * blockIn );

If these two pieces -- pointer and length -- were always kept together, then other data-types implemented in C, like strings could have been safer...

void StringCopy( MemoryBlock* destString, MemoryBlock* sourceString );

[/docs] permanent link

sustainable improvements

Eventually, every programmer has to learn to make sustainable improvements in code that maintain or improve the goodness of its design, rather than unsustainable "patching" of code, which degrades the goodness of its design.

Someone complains that members of his team are patching code instead of improving it, and therefore Extreme Programming isn't "scalable". The fact is, this lesson about the inappropriate use of "patching" is something that needs to be learned even if the team isn't doing XP. If it is doing XP, they are going to learn this lesson more quickly and more obviously than they would in a non-XP environment: their velocity is going to slow down over time until they can't create any new user-value within a single iteration. They will be forced to start refactoring ("refactoring" means improving the design of the code, not "rewriting" as many people seem to think). It seems that many if not most first-time XP teams have to learn the importance of refactoring from experience.

[/docs] permanent link

Flu

How do people know when a chicken or some other bird has avian flu? Does it get a runny beak? Can chickens sneeze?

[/docs] permanent link

Notes From the WWDC Keynote (seen on video)

WWDC 2004 - 3500 developers attending - 17% increase over last year

Apple Stores - started 3 years - now have 80 stores

20 million visitors per year

250 million $ sales of third-party software

ginza store in tokyo

itunes music store - 70% market share for legal downloads, europe 62% of sales, 700,000 songs

ipod = 50% market share by units sold

airport express & airtunes - rendezvous for your home stereo - lossless compression and encryption - itunes streaming to target base station

BMW - 3 series, minicoopers, etc., dealer-installed connection for ipod (in glove box) and car stereo/stereo-controls.

all powermacs are dual processor now.

1.25 gigahertz bus - faster than pcs

ibm = 2.0 to 2.5 gigahertz - 25.% increase

intel = 3.2 to 3.6 gigahertz - 12% increase

LCD displays - competitors buy the panels that apple rejects

20-inch, 23-inch aluminum, connect to pcs, macs, powerbooks, and 30-inch aluminum (g5 only with special video card using two dual-dvi connections). G5 can drive two 30-inch displays - 8 million pixels (single display) 2500x1600 4.1 million pixels 77% more than the 23-inch, $3299 ; all have dual firewire and usb ports built in - single cable that splits, dvi connection.

panther - pixlet? - 12 million users - 50% of installed base - no other "new OS" has this many users, - 12,000 applications "transition is now over"

industry has had only three major OS transitions so far

apple II to MacOS 1984

DOS to Win95 - 1995

MacOS to MacOS X - 2000

Win95 +etc to Longhorn - 2007

microsoft office 2004 - "better than windows version"

borland - java tools

quark

oracle - 10g database

peoplesoft

sun - java tools

alias - bob bennet - maya complete - 25% of maya sales are for mac - alias sketchbook pro - 70% of downloads are for mac - maya unlimited "cloth, hair, fluid" effects used in games and movies - Lord of the Rings

MYST IV - Revelation - simultaneous release on Mac and PC - music by peter gabriel

guitar rig - low-latency audio - daniel haver - garage band plugin (and stand-alone) - foot controller to step through pre-sets, custom tones, sounds like analog instruments/amps/etc. studio and live-stage performances

orbit - aran anderson - satellite simulations - 650 actual unclassified sats - computations to locate these used to take super-computer, developed in 3 months using xcode (one developer) - free freefall screen saver to ship

macos x 10.4 tiger - ship 2005 "first half of year" - apple is again innovator and others will be following ('other people still trying to copy panther') - 150+ new features - OS X is #1 "unix" os by unit shipments - new feature 64-bit for any process, 64-bit system library, can also run 32-bit processes, LP64 GCC support, better SMP fine-grain locking, XGRID built-in, access control lists

improved SMB performance, home directories, Kerberos and NTLMv2 authentication, HTML email, word tables in TextEdit

search - itunes search is fast, now do the same thing in the rest of OS - "spotlight" - standard files formats, extensible, works with current apps, spotlight integrated into address book, mail, system preferences, and finder, search copyright metadata, search fields built into finder window, search by "color space" to find CMYK images, save queries as "smart folders"

spotlight addressbook - save a query as a "smart group", search birthday "within next seven days"

spotlight mail - search 50,000 messages instantly, save a query as a "smart mailbox"

spotlight prefers - removed "favorites" bar - search for prefs related to "keyboard" or "desktop" including synonyms for microsoft terms ilke "wallpaper". ("wifi" = "airport" = "802.11")

search for "everything abuot bertrand" across multiple apps - system-wide search in menu bar - shows pop-down list for images, movies, mail messages, address book contacts, pdf documents (like "half dome" in a PDF map), can show in window for complete lists, sort by kind/date/people

spotlight sdk "today"

H.264 - quicktime video - standard codec from MPEG group for high-def DVDs, scalable from high-def down to "3G" cell phone - open industry standard, frank casanova - 250 million copies of QuickTime shipped, can display high-def image as same data-rate as today's std-def DVD rates, will be built into DVDs soon.

safari web browser - RSS and Atom support - "personal clipping service" - searching "tons of rss feeds" and save queries as bookmarks, sort by date or "this week", time, source of article, change length of articles to display in list format, i-tunes store has RSS feed of "top ten songs", group feeds together in 'tabs'

"core audio" already shipping - "core image" & "core video" to ship in tiger - uses "GPU" to do graphics processing in floating-point. allows real-time filters, 100+ filters built-in. extensions via "image units" and "video units"

phil schiller - "fun-house" demo app - over a billion floating-point calcs per second - "stack many effects" on top of each other - built on top of quartz - effects can be applied to text, non-destructive, live, active - any of the stacked filters can be tweaked independently, apply filters to quicktime video, implemented in one person-week in xcode

".mac" - half a million subscribers - sync engine built-into tiger, .mac preference panel, sync email, calendars, address boosk, etc., across .mac and multiple macs - sync sdk for other apps to use.

dashboard - "expose for widgets" - built on webkit - javascript - little accessory apps - calculator, address book, itunes, stickie, stock tracker, web-cam viewers, and others. one button to bring up the widget collection, single-key-press to show/hide all the widgets - sdk available today

automator - visual scripting - interactive scripts, 100 actions built-in, share and email scripts, iLife, email and access to other apps - sal soghoian - script to grab images from web (.mac home pages) - icons for various popular 'categories', list of actions for each category - drag action to "work-flow area" - automatically link actions together, click various settings for each action [radio buttons, pop-up menus, etc.], looks like this 'settings' GUI may be generated from applescript 'dictionary' of various apps. save "work-flow" scripts "into" various applications like Safari - makes scripting about as easy as hypercard, but different UI - SDK shipping today

ichat - personal video conferences that "just worked" - now using H.264 - multiple users - 10 chatters for audio + 3 with video/audio - the 3 video feeds are joined into a single image/window - the 2nd and 3rd video feeds have a "perspective" look to them. [actually 4 videos - yourself is shown in small image]

developer copy of Tiger today

[] permanent link

Metrics and Manager as Coach

Brad Appleton wrote a very good post on the XP mailing list about his experience with a Little-League baseball coach who collected a lot of metrics, using them to improve the kids' game, but not using them as rewards or incentives (or blaming/dis-incentives.) A few quotes:

I used to play little-league baseball. Overzealous parents and coaches were too often the norm, and made it be so much about winning-vs-losing that it sucked a lot of the fun and learning/improving right out of it.

Then one year I played for a team that had a manager that was different. He abhorred all that other stuff. He was big on strategy and using data. [...] He used a lot of that to help him come up with the batting line-up, which people were best at fielding which positions, pinch-hitting and relief-pitching.

He shared the info with the assistant coaches. The info was not shared with us kids. For starters, we were possibly a bit young to try and make sense of it, but he also didn't want to see it used to try and rank people as better/worse than one another.

[...]

Anyway, with all that data-gathering going on, and the way it was shared, it was NEVER used as reward/incentive. We all knew what our win/loss record was. After each game (win or lose) the very next practice the manager led a discussion about things that we did well, and then areas for improvement - which usually became our "drills" for the rest of that practice session.

Whenever we played a particularly significant game, at the very beginning, the manager would tell us we were going out for ice-cream afterward REGARDLESS of whether we won or lost. Other teams and managers never did that. They only went out to the ice-cream parlor if they won. [...]

I was never before on a little-league team that was so motivated, and enjoyed playing so much, and that so actively encouraged other team members (especially the weaker players, who needed it much more than our "all-stars"). All in all it was a great season, and the best I recall in my own little-league "career". We lost only one game all season. [...]

I have a feeling we wouldn't have done as well if rewards like "ice-cream" had been conditional upon winning. I know we certainly wouldn't have enjoyed it as much. [...]

[/docs] permanent link

DDD and Multiple Databases

In Eric Evans's Domain Driven Design, he describes various strategies to use when there are MultipleModels. Two or more teams may each have a BoundedContext for their unique models, and a SharedKernel for the model they have in common. In other situations, where one or more teams are forced to use an application or a database whose model is undesirable, the teams can use an AnticorruptionLayer to convert to/from two models. (In the ConformistStrategy, the team adapts to an existing model, even if it isn't the most desirable, and in the SeparateWays strategy, the teams don't have a shared model or even a shared kernel.)

I bring this up to link it with Martin Fowler's DatabaseStyles: he uses the term IntegrationDatabase for a database intended to be shared by multiple applications, and ApplicationDatabase to describe a db intended to be used by only one application.

It seems that the strategies for SharedKernel, AnticorruptionLayer and ConformistStrategy will tend to encourage the creation or use of an IntegrationDatabase, and the strategies for BoundedContext and SeparateWays will tend to encourage the creation of ApplicationDatabases. And that's OK, as long as everyone involved understands why these choices were made.

[/docs] permanent link

An Agile Success Story at Sabre

Marco Dorantes Martinez points us to a Computer World article describing Sabre's rebuilding of its air-travel reservation system: "...four-year, $100 million-plus project...C++ and Java running on 17 HP NonStop database machines and 45 Linux servers...it's all working.". The article doesn't go into details, but a participant on that project has posted on the XP mailing list: they use XP practices including pair programming, open work-spaces, TDD and refactoring, and stand-up-meetings, have one or more XP coaches; the project is using multiple XP teams that grow and shrink in numbers as demand shifts. And they must be using Frequent Releases:

Small steps also make it possible to respond to changes in technology. That's why Linux, which didn't look like a viable option in 2001, could be brought in later in the game.

And small steps make it possible to go live with each iteration of the system before moving forward -- to make sure the technology works and to make sure the system is what users need. If it's not, it can be changed.

[/docs] permanent link

Speed Over Quality

On Artimage.com/weblogs, Robert Martin writes "...then why would our company want to pay for crap that slows us down? Wouldn't they rather have good clean code that keeps us running fast? ... We often blame managers for schedule pressure. We often complain that our companies set unreasonable deadlines and have unrealistic expectations. This might be true, but it's only half the problem. The other half, the most important half, is within us. We consider our worth as programmers to be more associated with speed than with quality. And that's a tragedy; because it leads us to create messes. It leads us down the slower path. By rushing we ruin our chance to truly go fast." This generated a lot of comments, and one of my comments was the following:

Why do some managers managers value speed over quality? I can think of two reasons: they believe that buggy software is normal, or they get rewarded for delivering buggy software early.

The two ideas are often combined. If buggy software is normal, let's set an "aggressive" date for it to be "finished" (we're rewarded for meeting that date, even though nothing is really working well), and then spend twice that amount of time removing the bugs that were put in (and blame those programmers or those testers for any delays).

How did the idea of buggy software come to be "normal"? Because so few programmers are taught bug-reduction techniques like code reviewing, pair-programming, unit testing or test-driven-development in college, and relatively many programmers [none of whom read blogs like this or even books on these subjects] learn these techniques after college. And managers (with MBAs or otherwise), haven't gotten training on the value of these practices either.

[and check out these books - they're very timeless and enlightening: http://www.geraldmweinberg.com/books.html ]

Some commenters said that sometimes meeting the deadline is more important than quality, but that is actually rather rare. Most of the time, considerable effort is spent on bug-fixing. If quality was not important, why are we fixing the bugs? As Johanna Rothman points out, some companies are spending more than 75% of their time fixing bugs (rework).

[/docs] permanent link

Energy Prices

David Pimentel, Cornell professor of ecology, says: "The next time you're pumping gas or paying the heating bill, ponder this: As high as fuel prices are in this country, they would be even higher without government subsidies to prop up the industry. Instead of paying at the pump, every American family is paying about $410 in taxes each year for subsidies that keep gasoline prices and other energy product prices artificially low." [That's billions of dollars in taxes subsidizing the energy industry.]

Unfortunately, no politician is going to get elected by promising to raise gas prices, even if doing so decreases our dependance on foreign oil, decreases corporate welfare payments (and thus lowers our taxes), and pays for more efficient energy use.

[/docs] permanent link

Feedback

Matt Heusser writes on StickyMinds:

In less-healthy organizations, feedback may be viewed as criticism or personal attack. When this happens, the organization cuts off its own ability to react and respond to change. ...[feedback] must be positioned so the recipient does not feel insulted or drawn into a spitting contest.... Another way to foul up the works is to point out problems without offering solutions. ... try asking simple, open-ended questions. ... "What are the issues I'm forgetting?" "What does success mean?" ... Even if your feedback is well received, and you find or fix what would otherwise have become a major fly in the ointment, you run the risk of being perceived as the "Savior" who will "put the project back on track." There is another name for a Savior whose project fails: scapegoat.

[/docs] permanent link

Snips

On unit tests / programmer tests:

Michael Feathers: "Recently when people have asked me what a unit test is I tell them it is a test that runs in under 1/10th of a second." (XP mailing list)

"If the computer gets wet, would compressing its files help dry it out?" (said in a dream I had.)

George Lakoff on conservatives' use of language to dominate politics:

They have put a huge amount of money into creating the language for their worldview and getting it out there....

Language always comes with what is called "framing." Every word is defined relative to a conceptual framework.... "revolt," that implies a population that is being ruled unfairly... If you then add the word "voter" in front of "revolt," you get a metaphorical meaning saying that the voters are the oppressed people....

In Arnold Schwarzenegger's acceptance speech, he said, "When the people win, politics as usual loses." ... what he has done is frame himself and also Republican politicians as the people, while framing Democratic politicians as politics as usual....

The conservative worldview, the strict father model, assumes that the world is dangerous and difficult and that children are born bad and must be made good. ... The only way to do that is through painful discipline ... good people are the disciplined people. ... to the right wing, the good citizens are the disciplined ones ... Wealth is a measure of discipline.

... With Schwarzenegger, it's in his movies: most of the characters that he plays exemplify that moral system. ... He just had to stand up there, and he represents Mr. Discipline.

The phrase "Tax relief" ... got picked up by the newspapers as if it were a neutral term, which it is not. First, you have the frame for "relief." For there to be relief, there has to be an affliction, an afflicted party, somebody who administers the relief, and an act in which you are relieved of the affliction. The reliever is the hero, and anybody who tries to stop them is the bad guy intent on keeping the affliction going. So, add "tax" to "relief" and you get a metaphor that taxation is an affliction, and anybody against relieving this affliction is a villain.

So what should they be calling it [taxes]?

... Taxes are what you pay to be an American, to live in a civilized society that is democratic and offers opportunity, and where there's an infrastructure that has been paid for by previous taxpayers. ... The highway system, the Internet, the TV system, the public education system, the power grid, the system for training scientists ... which has to be maintained and paid for. Taxes are your dues — you pay your dues to be an American. In addition, the wealthiest Americans use that infrastructure more than anyone else, and they use parts of it that other people don't. The federal justice system, for example, is nine-tenths devoted to corporate law. The Securities and Exchange Commission and all the apparatus of the Commerce Department are mainly used by the wealthy. And we're all paying for it.

People vote their identity, they don't just vote on the issues, and Democrats don't understand that. Look at Schwarzenegger, who says nothing about the issues. The Democrats ask, How could anyone vote for this guy? They did because he put forth an identity.

[/docs] permanent link

WikiWikiHyperCard

Ward writes:

[HyperCard] Stacks were already easy to edit. The fields were automatically WYSIWYG editors. But linking was a pain that involved moving between both cards involved. My Links field abandoned regular stack links and used search on demand instead. Operationally, one would just type links they might follow in the Links field. Each line had a button nearby that would take you to the card if it existed or beep otherwise. If you held the button down, it would relent and go make the card for you.

How did it work? It worked great. Lots of people came by to see HyperCard work. I'd show them a few graphic things and then pull out my stack. We'd get to poking through people and projects and so on when my guests would invariably say, "that's not exactly right." So we'd fix it right then and there. And we'd add a few missing links and go fix them too. The stack was captivating. We were often late for lunch.

One of the most impressive HyperCard stacks I ever used was one to teach the language Esperanto. It was a very well-structured piece of educational software. Particularly cool were sections that had prose in Esperanto, and if you clicked on one of the words, it would be replaced by the english equivalent.

HyperCard is orphaned, now, but there are some open-source and commercial replacements in development... see here for direct and indirect links. PythonCard looks interesting. (Is there no Smalltalk-based xCard?) Ideally, there could be a web-enabled HyperCard that's user-editable like a wiki, but more structured like HyperCard.

[/docs] permanent link

Source Code Control

I'm still using CVS, but Subversion looks very good - check out this article by Mike Mason: http://osdir.com/Article203.phtml. There a book on-line as well.

[/docs] permanent link

Tweaking Stuff Until It Works

A comment by Adrian Howard on Ralph Johnson's blog:

I've found TDD especially effective with new programmers. They no longer make that classic beginners mistake of typing a bunch of code then tweaking stuff at random until it works. TDD forces them to move in small increments and they get continual positive feedback - absolutely fantastic for learning.

Unfortunately I've seen "experienced" (non-XP) programmers who work the same way - typing a bunch of code then tweaking stuff at random until it works. Even worse, whole projects where the tweaking is done in the "testing phase". Those kinds of projects tend use a process I call "unconscious waterfall": there's a coding phase, and a testing-and-fixing phase, (and very rarely a design phase at the beginning). The testing-and-fixing phase tends to take the longest amount of time. See also CodeAndFix.

Those unconscious waterfall projects tend to give responsibility for "quality" to the testing staff, when the true responsibility for testers is to report on the state of the project. Writing quality code is ultimately the developer's responsibility, and there are many tools to help: TDD or unit testing, pair programming or code reviews, etc. QA's role is to provide some feedback, but it usually comes too late in unconscious waterfall projects.

Check out a paper by Joahanna Rothman for testers on non-agile projects: What do they pay you to do? and this one on testing in an agile project: Testers Shine on Agile Projects. What a difference!

[/docs] permanent link

TDD - Refactor Sooner To Avoid Throwing Away Tests

Jim Menard in Unit Testing with OCUnit:

1. Write a large chunk of the application.
2. Write test suites that exercise everything written so far.
3. Fix the bugs found by the tests.
4. Write a few more tests; fix some more bugs.
5. Add new features.
6. Write tests for the new features.
7. GOTO 5.

Which he explains by:

This process lets me experiment and make a few design mistakes during step 1, perhaps starting from scratch a few times. Since I wait before writing the initial set of test cases, I don't have to write and rewrite tests that are exercising code that I'm probably going to throw away.

I'm thinking he hasn't quite "gotten" Test Driven Development yet. Then again, if you have no idea on how to do something, a spike (without unit tests) lets you figure it out. The XP practice would then be to throw the spiked code away, and now that you know how to do it, implement it test-first. This way you spend very little time in the debugger, and almost no time fixing bugs.

However, his desire to avoid "exercising code" that he's going to throw away may be counter-productive. He may be writing too-big tests for too-big features, and probably writing too many tests before refactoring. My experience with TDD is that tests rarely have to change if they test for expected results rather than testing "how" something is done.

When I'm doing TDD well, I write a small test that does very little, and then a small piece of code to pass the test, repeating with additional small tests until the code does everything I think it needs to do. Along the way, I refactor to eliminate duplication... I am very alert to writing a test that looks very similar to an existing test, and I'm on the alert for chunks of code under test that also seem to be duplicates. These are telling me that it's time to refactor to create an abstraction. I don't want to have 15 almost-identical tests that can be replaced by a single test after a refactoring; but replacing a second or third duplicate test by creating an abstraction is not painful, because the tests are small and I catch the duplication as soon as I see it.

When I work this way, I rarely need to throw away and rewrite tests as the design evolves. I may need to change a name here or there, or some other small refactoring, but the tests are mostly immune to internal changes to the code under test.

When I have problems doing TDD, it usually comes from not knowing how to use an external API. I take a stab at it. Find it doesn't work at all. Have to look up [more] documentation, take another stab. Search for example code, take another stab. And so on. (Also let my partner drive, if I'm pair programming). It's particularly irritating if the external API I'm trying to use is poorly documented, buggy, or behaves differently on different versions of the platform I'm working on. (This gets discovered when the test fails when I build and run on that other platform.) The test just sits there until I figure it out. Sometimes I take an entirely different path, and that test has to be eliminated -- though that's not a big deal, since that test is probably only three or four lines of code anyway.

Back to the looking for duplication to be refactored... Jim has the following code in his test-case:

    STAssertEquals(123L, [entry amountInPennies],
                   @"bad amount; 123 != %ld",  // format string
                   [entry amountInPennies]);
    STAssertEquals(1.23, [entry amount],
                   @"bad amount; 1.23 != %f",
                   [entry amount]);

I look at the format-string argument to STAssertEquals and see duplicated code. I'm not familiar with OCUnit, but the xUnit frameworks I'm familiar with will tell me the actual and expected values if the test fails -- I don't have to write a format string to do get that output. If OCUnit doesn't do this, I would rewrite it. Since in Objective-C we need to get the type correct for non-object types, and since we don't have function/method-name overloading, I'd write "equals" functions for the various types, so my tests would look like this:

    STAssertLongsEqual( 123, [entry amountInPennies] );
    STAssertDoublesEqual( 1.23, [entry amount] );

It's by making small refactorings like this, as soon as you spot the duplication that you avoid having to do big refactorings later.

Jim's article also talks about putting the data for the tests in an XML file, which might be useful for acceptance-testing, but is just a waste of time in test-driven development. You're not eliminating any duplication, you're just moving the data from the tests, where they're convenient, to an external file, which creates an indirection and dependency. The indirection makes it harder to understand why a test has failed, and the dependency means that refactoring is going to be harder -- changing the tests may also require changing that XML file, or vice versa, as well as more ways to fail: the test might not find the xml file, or find the wrong xml file, or the xml file might have the wrong data, having bugs in the xml-handling code, etc.

[/docs] permanent link

Smokers and Unit-Testers

What inappropriate metaphor can I write about today, which will annoy the most people? How about smokers and non-smokers being analogous to undisciplined developers (the majority) and the disciplined developers (like those who participate in successful XP projects).

A smoker may say that his smoking doesn't bother anyone else, and a typical developer who never writes unit tests may say his lack of discipline doesn't bother anyone else, but they're both wrong. If I'm in the same room as a habitual smoker, or on the same project as a undisciplined developer, I'm going to suffer to some extent -- from the physical odor of the former, or the code-smells of the latter.

The larger context also sees problems: higher health-care costs for the smoker [and thus higher expenses by insurers and governments]; longer project times (for debugging) and lower-quality products from the undisciplined developers.

Some strategies for trying to get stop people from smoking get you a deserved "anti-smoking zealot" label, and trying to promote XP and test-driven-development by using unflattering analogies [like I did here] can get you an "agile zealot" label.

Preaching the good things that happen to people to quit smoking doesn't seem to affect the nicotine addicts -- even when that preaching comes from the converted. Larry Niven finds himself writing about food in his more recent novels, because now his taste-buds are operating much better than they were when he was smoking. To people who expect all software projects to have bugs, an XPer saying how his XP projects that had fewer bugs seems to have no impact. Either the XP project is judged to be full of exceptional developers, or the projects are judged as being not "as hard" as others.

It's a quality of life thing. A non-smoker can climb stairs easily and taste his food; a disciplined XP developer can deliver quality software with less suffering. A crisis, or a particularly bad example, can sometimes move a non-smoker to stop smoking. [Best fictional example: Kenneth Branagh offered a cigarette by someone who smokes through the hole in his throat in Dead Again. His response: "I just quit."]

If there is no personal crisis - late, buggy software and heroic bug-fixing are considered normal, and the company isn't failing and people are still getting raises - then software developers are going to continue with their usual habits.

Fortunately, there does seem to a be trend to adopting test-driven development or at least unit testing... unit test frameworks are freely available for most platforms, and the frameworks themselves are not too difficult to use. For some programmers, the benefits of higher quality, more-modular, code is worth the effort, even if their co-workers don't see the point in it.

[/docs] permanent link

XP on a Large Project

A Paper[pdf] by Amr Elssamadis of ThoughtWorks. Quotes:

...35 developers, 15 business analysts, and about 10 QA...

Unit tests and integrated builds - are ABSOLUTELY MANDATORY... As the application gets larger and larger it becomes almost impossible to add new code or refactor existing code without going through tests.

We originally had longer iterations - one-month ... had problems estimating [large stories] ... We are now doing two-week iterations ... estimations are closer to target....

Have an iteration planing meeting at the beginning of each iteration where the customer and developers split up in groups all day to discuss the latest story cards and estimate them. At the end of the day regroup and present your estimations and findings... This will keep the whole team in-the-know ... without burderning everyone with an extremely tedious and long meeting.

Frequent design meeting (lunch...) are very helpful....

It will always be tempting not to refactor and to just patch a solution, but if it is patched too much the team will be forced to make major refactorings later on.

...a group of individuals are needed to be the customer....

[/docs] permanent link

Red Dwarf

I was watching my tapes of Red Dwarf yesterday. It's an excellent British science-fiction comedy second only to the radio version of The Hitchhiker's Guide to the Galaxy in quality. No connection to "Agile" software development, except that you could say the show is about team-work. Extremely reluctant (and thus funny) team-work in the early seasons.

Seasons 6 and 7 were the least funny, perhaps because the team was working most efficiently then. However, the split of the original creative team of Grant and Naylor, and no longer filming in front a live audience, probably had a lot to do with the decline in quality in these seasons. Season 8 returned to hilarity levels that seemed normal in seasons 2 through 5.

One of the best bits of season 6 is this part of "Legion":

RIMMER: Go to blue alert.
LISTER: What for?  There's no-one to alert - we're all here.
RIMMER: I would just feel more comfortable if I know that we're all on
  our toes 'cos everyone's aware it's a blue-alert situation.
LISTER: We all are on our toes.
RIMMER: May I remind you all of Space Core Directive 34124?
KRYTEN: 34124.  "No officer with false teeth should attempt oral sex in
  zero gravity".
RIMMER: Damn you both, all the way to Hades!  I want to go to Blue Alert!
LISTER: Ok, ok.

LISTER presses a button.  The "Alert" box on the wall starts to flash
blue.

RIMMER: Thank you.  A bit of professionalism.
KRYTEN: Wait!  I've got something - I'm punching it up.

5 Model Shot.

We see a view of an orange, comet-like thing speeding through space

6 Int. Cockpit.

LISTER: Too small for a vessel... maybe some kind of missile.
KRYTEN: It's impossible to tell at this range.  Whatever it is, they
  clearly have a technology way in advance of our own!
LISTER: So do the Albanian State Washing Machine Company.
RIMMER: Step up to red alert!
KRYTEN: Sir, are you absolutely sure?  It does mean changing the bulb.
RIMMER: There's always some excuse, isn't there?

[/docs] permanent link

How to write Acceptance Tests for GUI Applications

John Roth sums it up in the XP mailing list:

The way we usually handle this is with a layered architecture. There are three layers: the Graphical UI, the Logical UI, and the rest of the application. (The rest of the application is probably also layered, but it's not particularly interesting at the moment, so I'm ignoring it.)[...]

The GUI is a thin facade on top of the LUI. If there is a button on the GUI, there is a method in the LUI that gets called when the button is pressed. If a field is to be displayed on the GUI, there is a method in the LUI that is queried for the contents of the field. If the GUI needs to be notified of something so the screen can be updated, there is a method in the LUI that will accept a callback. And so forth.

The acceptance test deals with the Logical UI. Since the Logical UI is isomorphic to what appears on the screen, we don't lose anything except the hassle of trying to test the application through the GUI.

When using Cocoa/NextStep, you specify the LUI methods to call in InterfaceBuilder [for example, if I recall correctly, a button in a dialog calls any method you want on the class you designate as the "target", instead of "sending an event"], so in that environment you actually do very little "GUI" programming - in the sense of writing code.

My strategy for acceptance testing GUI apps has been to make the App scriptable. Support AppleScript access to your domain and GUI objects on MacOS. On Windows, we made appropriate objects COM accessible, and thus could write acceptance tests in JScript of VBScript (or almost any other scripting language on Windows.)

[/docs] permanent link

Moving to a New Environment

Sean Malloy's comments on trying to get into a new development environment: here

I was thinking about how some people won't use a language unless they can do it from emacs... and how I don't like environments that try to confine me to a single open window: Borland's JBuilder, for example, or the default modes of the windows of VC++ (though at least VC++ isn't too hard to change.). Any little thing, or lots of little things, can made moving to a new environment just a little too uncomfortable to actually do.

[/docs] permanent link

Designing for Inheritance

What's so hard about Designing for Inheritance? Not being able to predict the future. It may be impossible to know what every possible subclass is going to want to do with your base class, but it isn't hard to make a class that handles the majority of subclassing needs. (And if you make the source code available, the person writing the subclass has all the information that they need, including being able to refactor your class.)

But the real need isn't subclassing per se, but substitutability. Your class doesn't do exactly what we want, and we want to substitute our own class that does exactly what we want. In languages that require subclassing (or interface implementation) to get substitutability/polymorphism, that means we need to subclass or implement an interface. In more dynamic languages like Smalltalk, Ruby, Python, etc., we can just implement the methods in a new class, and pass in an object of that class where the other class object was expected. As long as the program isn't explicitly checking the name/identifier of the class of the object ("o.getClass().equals(expectedClass)" or "o instanceof expectedClass") we can usually get what we want.

One simple way to implement substitutability in languages like Java or C++ is to declare interfaces or abstract base classes that reflect the public API of your concrete classes. Always declare variables and parameters using the interface/ABC type everywhere - never use the concrete type. There's a rule of (less-dynamic) OO design: abstract types should only depend on other abstract types, and concrete types should depend only on abstract types. Of course, no one follows that rule to the letter. It's more of a guideline. But something to think about, perhaps with some exceptions for "common" types like String.

Here are some guidelines for writing a class that is easy to subclass: Always declare all public methods "virtual" or non-final. Declare member variables private, but also declare protected accessor methods for all the member variables and use those everywhere. Every place that you create an new object, put that "new" statement in a protected factory method that could be overridden. Break down large methods into ones that call a sequence of smaller, protected methods. If you have private methods, consider that a sign that those methods belong to another class (probably one that hasn't been written yet.) It's a little easier in Java than C++, because method-calls-on-this in a constructor in Java are "virtual" and are not virtual in C++.

[/docs] permanent link

Ant Rethinking

The inventor of Ant writes:

In retrospect, and many years later, XML probably wasn't as good a choice as it seemed at the time. [...]

[...] when I fused XML and task reflection in Ant, I put together something that is 70-80% of a scripting environment. I just didn't recognize it at the time [....]

If I knew then what I know now, I would have tried using a real scripting language, such as JavaScript via the Rhino component or Python via JPython, with bindings to Java objects which implemented the functionality expressed in todays tasks. Then, there would be a first class way to express logic and we wouldn't be stuck with XML as a format that is too bulky for the way that people really want to use the tool.

[/docs] permanent link

Measured Throughput

Agile Hiring: see Esther Derby's article on Hiring for a Collaborative Team.

Johanna Rothman writes about an organization where the number of requirements implemented are always significantly less than the number of requirements requested. They've been blaming the testers (!) and not seeing that their throughput through the whole system is limited. Given the numbers that she posts, I compute that the through-put is about 3.4 completed requirements per week for project 1, 3.7 per week for project 2, and 2.3 per week for project 3.

If that company accepted that they're only going to get about 3 requirements per week implemented, no matter how long the project is, they could avoid waste in the "requirements/design" phase by not specifying more requirements than their measured through-put allows.

They could also use that figure to move to incremental development as well, and maybe learn practices that would improve throughput as well. If they used XP or another appropriate set of agile practices, they could maximize information-flow and minimize delays between requirements, implementation, and testing.

To find the bottle-neck, we need to know who's waiting on whom in this organization. Is there a queue of finished-but-not-tested work piling up in front of the testers? Then it could be that the testers are the bottleneck. One way to deal with a bottleneck is to change to a "pull system". In this case, a programmer only does implementation when tester is about to be ready to test their work. That leaves the programmers idle, but then they could use that "idle" time to improve their development process (code reviews, unit testing, etc.)

In this system, the bottleneck is the programmers. The "pull system" can extend to the requirements: don't specify a requirement until a programmer about to be ready to implement it and a tester is about ready to test it. To move out of phasist approach, work on testing (creating automated tests) and implementation can overlap -- or (the XP way) the programming can actually follow the creation of tests and use the tests as an executable version of the specification.

[/docs] permanent link

Palindrome Day

Today is 04-01-04 (sorta). Writer at work, do not disturb. The writer is disturbed enough already. Don't be fooled by that Google "gMail" announcement - the clue to the joke is the free 1 gigabyte per user feature. San Jose Mercury News reported it as straight news. Suckers.

[/docs] permanent link

Doctor, It Hurts

The old joke: "Doctor, it hurts when I do this (raising arm over head)." The Doctor responds "So don't do that." I heard a complaint about an XP project: the acceptance testing for many stories were not being completed until the next iteration. This company was doing three-week iterations, where the stories themselves took three weeks to do (not including the testing time). So don't do that. My advice: smaller stories that aren't considered "done" until the acceptance testing for them is done. If a two-day story ends up taking four days, it still can be finished (including testing) before the iteration ends.

Estimates for a story should include testing time. I also recommended that they look into "Fit" and "Fitnesse" to see if they can ease the creation of their automated acceptance tests, and even get the tests done before the code to make them pass is done.

[/docs] permanent link

Specification by Example and Surprises

If you view XP's acceptance/customer tests and "unit"/programmer tests as specification by example, then the difficulties some game developers have experienced trying to use XP make more sense. "Game Play" is a subjective experience that is very difficult to quantify -- I'm talking about fast-paced first-person-shooter video-games, not solitaire. Part of a good game-playing experience would be the player's anticipation of what comes next, but being pleasantly surprised when it happens differently than expected.

This lesson -- "the anticipation of what comes next, but being pleasantly surprised when it happens differently than expected" -- is one I'm learning at the writer's workshop I'm attending.

[/docs] permanent link

Writing

My first day attending a workshop on writing is about to start. (Oh, I did take a creative writing class in college that was a sort of workshoppy.) Other than knowing this workshop is at least partly based on the work of Virginia Satir (google and amazon [The New Peoplemaking] her yourself - no time for links today), and knowing some of the people involved, I have no idea what's going to go on this week.

Last week, I did a little work on two science fiction novels, but did very little writing. I also attempted to write using IBM's Via Voice, and that experience had a lot to do with not writing -- all those misinterpreted words brought out the editor in me instead of the writer. And, making an introvert speak his thoughts out loud goes entirely against nature.

[/docs] permanent link

Changing Artistry

Hans Nowak blogs:

Taoism is about being in harmony with nature. As the Wikipedia page says, "Do not try to force things, for nature is overpowering. In particular one must act in accordance with how things are, not how one wants things to be."

Now, people excited about XP or other things often ask "How can we get [force] them to change?" The answer lies more in finding ways to get "them" to want to change, rather than trying to force them to change. If they don't want to change, they're not going to make the effort. If they perceive the need to change, then maybe they'll want to change. The problem is that sometimes, some people never see the need to change.

Art Kleiner has been involved in a lot of Change Artistry: his book-credits include The Dance of Change, The Fifth Discipline [uncredited co-author], The Fifth Discipline Fieldbook, The Age of Heretics, Oil Change: Perspectives on Corporate Transformation, and Car Launch: The Human Side of Managing Change.

Kleiner's most recent book seems to be a reaction to repeatedly seeing change efforts fail because the "core" members of a company don't really want the change to happen. The book is Who Really Matters: The Core Group Theory of Power, Privilege, and Success. Quoting the Publisher's Weekly review:

The old saw "the customer comes first" is a flat-out lie, argues Kleiner, a contributing editor at strategy+business magazine and the author of several business books, in this fresh look at the structure and politics of business. He contends that "a depressing number of business corporations have evolved into organizations with one primary purpose: To extract wealth from all constitutions (not just the shareholders, but the employees, customers, and neighbors as well) and give it essentially to the children and grandchildren of some of its senior executives." Such corporate selfishness works because the key decisions in are being made by the "Core Group"-executives or employees whose needs and desires determine company behavior. Others within an organization immediately sense who is in the Core Group and adjust their behavior accordingly; "Day after day, in all the small decisions we made, all the employees contributed to keeping these individuals more or less at the center of the Core Group."

Maybe this behavior - core group getting what it wants, the non-core group helping them get it - is a basic primate behavior that we've been saddled with from our primate ancestors.

Perhaps the weirdest thing I've seen that could be attributed to our primate ancestry was a comment someone made to me during the recall-election for California governor. He said "I like [some candidate who was third- or fourth-place in the polls] but I'll vote for Schwarzenegger because he's going to win." Excuse me? If he's going to win, you don't need to vote for him. If you and everyone who thinks like you actually voted for your favorite candidate, Schwarzenegger might actually be defeated.

I think wanting to vote for the winner (or otherwise helping the core group) is like an ape wanting to show support for the most likely replacement of the alpha male. Primates that didn't show support for the most likely alpha male were more likely to be killed or ostracized when the new alpha male took power (assuming they 'bet' right) -- we're descendants of those who "went along with the winners" and were thus more likely to reproduce. Today, our elections have secret ballots, so no one would ever know you didn't support a "winner", but we're still primates.

[/docs] permanent link

CMM

Dean W. Schulze on agilemanagement mailing list refers us to http://www.cio.com/archive/030104/cmm.html:

SEI officials say they are not in the business of controlling what companies say about their assessments. Nor will they reveal to the public which companies have been assessed or what the assessments consisted of. "We weren't chartered to be policemen—we're a research and development group," Hayes says.

Indeed, CIOs who look to CMM for guarantees won't find them, says Rick Harris, director of application development for OnStar, a division of GM that provides communications inside the company's vehicles. He recalls confronting a manager from one of his CMM Level 5 offshore outsourcing companies who did not know how to do a testing plan for software. "My people had to train him to do it," he says. On another occasion, Harris's staff discovered that the offshore provider had fallen far behind schedule in one of its projects but had not told him. "You'd think a Level 5 company would have told me months before that the schedule was slipping and we needed to do something," he says.

"Having a higher maturity level significantly reduces the risk over hiring a [company with a lower level], but it does not guarantee anything," says Jay Douglass, director of business development at the SEI. "You can be a Level 5 organization that produces software that might be garbage."

That assessment is borne out by a recent survey of 89 different software applications by Reasoning, an automated software inspection company, which on average found no difference in the number of code defects in software from companies that identified themselves on one of the CMM levels and those that did not. In fact, the study found that Level 5 companies on average had higher defect rates than anyone else. But Reasoning did see a difference when it sent the code back to the developers for repairs and then tested it again. The second time around, the code from CMM companies improved, while the code from the non-CMM companies showed no improvement.

[/docs] permanent link

Encapsulation

Quoting from http://www.netobjectives.com/ezines/ezine_2004_01.pdf:

Gang of Four: "Consider what should be variable in your design. Focus on encapsulating the concept that varies."

Alan Shalloway: "This mandate was a bit confusing to me at first. To me, encapsulation meant 'data-hiding'. But to the Gang Of Four, it means something else: Encapsulating the concept that varies is encapsulation of type.

I'll go into this more some other day... see also Net Objectives� other ezines: http://www.netobjectives.com/ezines/ezine_2004_02.pdf.

[/docs] permanent link

One is Not Enough

A pattern gleaned from Artima Weblogs. Knowledge is not enough to write software well. Discipline is not enough. Practices are not enough. And so on. No single thing by itself is enough. Knowledge + Practice + Feedback + Discipline (and maybe a few other things) may be enough.

[/docs] permanent link

Individual Ownership Destroys Encapsulation

Jeff De Luca, quoted by David J. Anderson:

If you do not use class ownership, when you assign a feature to someone then they must touch four classes (for example) in the system. They must have knowledge of four classes in the system.[...] Encapsulation is one of the founding principles of objects and is about a class being able to hide[...]

And how hard it is for a member of jelled team to have (or get) an understanding of the major classes of the system? In fact, how hard is it to understand any class of a system, if it's got intention-revealing names, "simple code", unit tests that reveal if your changes have broken anything, and so on? What I've seen happen many times in individual-class-ownership environment is not that the classes are intrinsically hand to understand, but that people defend their turf by discouraging other people from understanding their classes.

Also, because everyone is afraid to modify other's classes, their own classes become kitchen-sinks full of stuff that really belongs elsewhere -- this does make classes harder to understand, because they end up being badly designed and violating encapsulation "in reverse" -- the big fat individually-owned-classes have many "internal" pseudo-objects they depend tightly on, when they should be having looser connections with external classes.

[/docs] permanent link

WAGNI

Robert Martin: "Why do people get so upset when we say YAGNI [You Aren't Going to Need It], but not when we say KISS."[Keep It Simple, Stupid]

Adrian Howard: "I find some people take it as a personal criticism. Saying "You aren't going to need it" seems to enter some people's heads as "Bah - you are a complete idiot for doing this".

Maybe it should be WAGNI: We Aren't Going to Need It.

[/docs] permanent link

Efficient

Cats are one of the most efficient predators, and they spend most of their time sleeping. CPUs are more powerful than ever and they spend most of their time idle or sleeping. Does this apply to humans? Apparently not. American Farmers are (making up a number) 100 times more efficient than they were 100 years ago, but instead of resting more, they farm 100 times more land or animals (and there are fewer farmers).

Programmers can develop more powerful applications than they could 30 years ago, but more of them are spending longer hours than ever. Other American workers are also spending more hours at work. According to business magazines, most CEOs spend 60 hours a week or more at work. Compare that to our hunter-gatherer ancestors who only needed to hunt/gather 15 hours a week. Is this progress? (OK yes, we have medicine, Beethoven's 9th symphony on CD and mpg, TV, and books...)

Eventually we or our descendants will be living in a world in which the only things we won't be able to manufacture for nearly-free would be (new) entertainments and some services. What will we do then? Will a few workers and companies have money, and everyone else subsists, or will we live in a post-capitalist society? Look for deflation in everything except real-estate and entertainment. Maybe computers count as entertainment.

[/docs] permanent link

Consequences of Decision-Makers

I was thinking "how do Gore & Associates do projects?" Since there are no managers, but only "associates", some of whom are "sponsors", I imagine that new product development is organized by (and around) product champions, like 3M and Toyota. Quoting the Poppendieks: "Every new product program has a strong leader, at 3M it is the ‘champion’ and at Toyota it is the shusa or chief engineer.  In both cases, these people ‘own’ the product – their name is associated with it.  We have Art Fry’s Post-it notes and Kosaku Yamada’s Lexus. These leaders are the ‘voice of the customer’ to the team and they assure the proper amount of design goes into it before detailed development proceeds. [...] The concept of a project manager is foreign to both companies, as is the idea of monitoring tasks" [instead, they monitor results].

Do these companies start selling a new product before they create it? That's a sure way to put make time-pressure the priority over everything else, including quality. I've heard that the Post-it note project was "cancelled" three times before it was completed, so I'm guessing that 3M doesn't sell products before they're "hatched". However, many software projects are are sold (with scope and time-frame fixed) before they're planned -- sometimes before they have any estimates. And they are not re-planned when detailed estimates (or reality) show that the scope and time-frame are unrealistic.

In a problem-solving workshop that I participated in, 30 people in one room simulated a company, and the 'salesmen' were selling something we didn't know how to make. And I was part of that. I was also one of the designers for a product we didn't yet know how to make. It was hilarious. Everyone was in one room, and yet we did not communicate, estimate, and make plans based on the reality of the product development process -- a process that we hadn't yet learned how to do. This happens even more so in companies where the salesmen and the developers never meet; where the consequences are not aligned with the promises.

A typical rant about the way most software projects are done says "Late, over budget, buggy systems is the norm." and blames managers for not understanding the difficulties of software development: "So when I now go to a manager and attempt to explain that the coupling in the system is going to crush us in a year or so unless we take steps to address it now, their world view doesn't include a perspective that let's them say 'Oh right. I remember making this mistake.' [...] They can see the value of features and functions, but decoupling the system is not something you really value until you've felt the pain of not actively doing it."

The manager described above doesn't feel the pain from the consequences of his decisions (So why is he in the position to make those decisions?). Jared Diamond says this is one of the root causes for societies to collapse: "A theme that emerges from Norse Greenland as well as from other places, is insulation of the decision making elite from the consequences of their actions. That is to say, in societies where the elites do not suffer from the consequences of their decisions, but can insulate themselves, the elite are more likely to pursue their short-term interests, even though that may be bad for the long-term interests of the society, including the children of the elite themselves."

Extreme Programming and other agile methods try to get the domain expert / product champion / "Customer" closely involved in the project, but at the same time doesn't permit the Customer to make the technical decisions that developers should be making. The Customer isn't allowed to prevent the programmers from writing unit tests and keeping the code clean. However, the programmer's aren't allowed to implement features that the Customer doesn't need or want. The Customer controls scope, but does NOT make up the estimates. If the scope doesn't fit the time-frame, XP makes that obvious pretty early in the project, and it becomes the Customer's responsibility to change either the scope or the time-frame. The Customer isn't a member of an elite, insulated from the developers and the consequences of his decisions.

[/docs] permanent link

The Detective Story Denouement Revisited

The master detective (MD) tossed the dagger, in its evidence bag, to the suspect, who caught it in his left hand, transfered it to his right hand, and tossed it back to the detective, who failed to catch it. Clank.

MD leaned back to his partner and whispered, keeping his eyes on the suspect: "What do we do now? We know from the crime scene that the killer is left-handed. Is this guy right-handed or left-handed?"

Watson replied, "Maybe he's ambisexual."

MD's eyes crossed. "Ambidextrous."

"That too."

[/docs] permanent link

The Point of TDD

Tim Bacon wrote:

I would much rather aim for 100% test-driven development than aim for 100% test coverage. It seems odd to want to use coverage tools to determine after-the-fact that the development process isn't working as intended, rather than to work constantly with development teams at the source of the problem. High path and branch coverage is a byproduct of the effective use of TDD, not the aim of it.

Scott Hanselman commented:

But I fear he's missing the point.  If I'm only testing 1% of my code paths, 100% of the time, what am I accomplishing? Squat.

As my boss has said, adding a non-intrusive code coverage tool to an already successful unit-testing strategy can absolutely find gaps in coverage that staring at the screen just won't find.

But juxtaposing 100% TDD with a 100% coverage goal doesn't make sense.  They are complimentary, parallel even, but not opposing goals.

My reply: By aiming for 100% TDD, one can get 100% statement-coverage and branch-coverage as a side-effect. It's not that coverage isn't valued, but that getting coverage via TDD is easier than trying to get coverage by after-the-fact unit testing. (And nowhere did Tim Bacon say that he was only aiming for 1% of the code being written by TDD. Read what he wrote!)

TDD's rule is "write no line of code without a test to force its existence." So that pretty much guarantees 100% statement-level coverage. The rule also applies to "if" statements, etc., so you don't write an "if" statement unless you have a test to force it to exist - pretty much guaranteeing 100% branch-coverage.

The key misunderstanding is that Test Driven Development isn't primarily a testing technique, it's a development technique - a way to drive the design of working code that happens to leave a suite of "programmer tests" as a highly valuable side-effect. We use the term "programmer tests" to try to avoid misunderstandings based on the term "unit test". XP has "customer tests" / acceptance tests (driven by the customer) as a whole second level of tests -- but the point is to get "just enough" tests to insure high quality -- which is a LOT more testing than most software projects get.

I think there used to be a page in Ward's Wiki with a title like TheUnitInUnitTestIsntTheUnitYouAreThinkingOf but it appears to be gone now. See http://c2.com/cgi/wiki?search=unit and http://c2.com/cgi/wiki?search=test for whole lot of info about XP, TDD, and testing.

[/docs] permanent link

World Views

I find it amusing to notice the world views from various professions. To a tester, everything is something to test. To lawyer, everything is an argument. To a programmer, everything is a program. To a geologist, everything involves silt, lava, uplift, or plat tectonics. To an accountant, debts to be paid, money to track.

[/docs] permanent link

Cheap Products Lead to Economic Hardships

Retailers blamed for rise in slave labor"

Big-brand fashion and food retailers are accused of contributing to appalling employment conditions around the world in a new study by Oxfam [....]

[...] price pressure has led to hourly production targets that are almost impossible to reach [...]

[...] several factory managers admitted to researchers to using an array of tools to pass inspections even though they were violating the codes set by retailers. Factories supplying Wal-Mart, Toys R Us and Tommy Hilfiger were found to have false documents on hours and wages and to coach workers on how to answer inspectors' questions.

See also The Wal-Mart You Don't Know and Toy makers sending fewer hot toys to Wal-Mart.

[/docs] permanent link

Top-Down or Bottom-Up

"How do I make them do ...?" versus "How can we do ...?"

The first question I hear being asked by anti-Agile zealots and by people who favor hierarchical organizations (where decisions and information tend to flow one way -- down). The second question: I'd like to hear it being asked more often.

Laurent Bossavit:

There is a school of thought that if conditions are right, teams actually work better when there isn't a boss; they're called "self-directed work teams", or self-organized teams in the Agile jargon. With teams where "someone has to be the boss", or the single brain that directs all the activity, you also have to suppress the motivations of the individuals, or at least provide enough incentives to ensure that they completely align with the bosses' motivations, and not work against them. Robert Austin's book "Measuring and Managing Performance in Organizations" does a great job of illustrating the problems with that strategy.

I've heard of one large company that only has one job title: "associate". Here's an article about W.L. Gore & Associates. Quotes:

FEW COMPANIES [...] can say they have surpassed $1.3 billion in sales while operating a company that has no managers or employees. [...]

When Gore first started the company in 1958 [...], he became interested in Douglas McGregor's management book The Human Side of Enterprise (1960, McGraw-Hill), which espoused Theory Y management, a practice similar to what is now known as empowerment, and the lattice organization, which is characterized by an absence of assigned or assumed authority. The lattice organization has sponsors rather than bosses or managers; associates rather than employees; natural leadership defined by followership; person-to-person communication; objectives set by those who must make them happen; and tasks and functions organized through commitments. [...]

Any highly innovative company needs to have a free flow of knowledge," says John E. Sawyer, associate professor in the University of Delaware's Dept. of Business Administration. "The structure [at] W.L. Gore stimulates that form of communication." [...]

New associates at the firm are required to participate in a week-long orientation session emphasizing cultural integration. Three of those days are spent on direct-communication skills, an important element of a lattice environment. [...]

[/docs] permanent link

Bill on TDD

Bill de hÓra wrote good stuff, here's a snippet:

One aspect of TDD is that it persists knowledge about the system, executable knowledge, which was previously transient, unavailable or simply thrown away. Without TTD or test-first, this 50% output/effort was probably never captured. It was spent interpreting print lines and on hours poured over the debugger. Time spent in a debugger is not recyclable, not repeatable, not resuable by others[...].

Without TDD probably less than 20% of your time is in writing code. It's in eyeballing print lines and debugger output, reading an ill-formed spec, being in more and more meetings because things are falling behind, doing everything except adding functionality. Sure, the first month you cranked it and it felt good - we've all been there. But without the continual investment in tests, the ability to keep going and sustain pace falls off - dramatically. [...]

[/docs] permanent link

Quotes3

A blind man will not thank you for a looking-glass -- (English proverb)

It is a bad plan that admits of no modifications. -- Publius Syrus (ca. 42 BCE)

[/docs] permanent link

Fixing What's Broken

Dan Gilmor: "One lesson is deceptively simple. When you remove the barriers to changing things, you also remove the barriers to fixing what's broken." This is in an article about wiki-webs, but also applies to Team Code Ownership, my name for the XP practice of not having individual code ownership.

Lasse Koskela: "Although everybody knows that the project management should take corrective action, customize the process, or whatever is within its power in order to bring the project back on track, we implicitly accept 'it was the process' as a valid explanation for our failures even though the people with the power did nothing"...

Perhaps the difference in cooperative improvements in wikis and in businesses doing projects, is that the wikis can have both anonymity (no blaming) and explicit permission to change things. To quote from Gillmor's column:

The Wikipedia articles tend to be neutral in tone, and when the topic is controversial, they will explain the varying viewpoints in addition to offering the basic facts. When anyone can edit what you've just posted, such fairness becomes essential.

``The only way you can write something that survives is that someone who's your diametrical opposite can agree with it,'' says Jimmy Wales, a founder of Wikipedia.

David J. Anderson wrote of "Manager as Permission Giver": "A line manager can act as this tipping person to change both the behavior of his staff but also the performance of his organization. The manager can change the culture by giving permission to change it.[...] One big caveat to this 'manager as permission giver', is the idea that the line manager must have gained the right to give permission from more senior management. The more senior managers may not be 'permission givers' themselves and hence, in order to change behavior, the line manager may need to pre-emptively up-manage the situation."

Team Code Ownership gives permission to the team to fix problems anywhere in the code, but it is restrained by practices of (1) requiring all unit tests to be passing before checking in the changes, (2) review/consent by the pair partner, and (3) of course, the acceptance tests must not stop passing as well, though that may be detected as quickly as a failing unit test. With Daily Standup Meetings and Open Workspace to encourage the team to communicate problems with the code, the entire team can be involved in the decision to fix problems and whether to do any big refactorings of the code.

[/docs] permanent link

Low Defect Rates

Martin Fowler writes about a few XP projects he's aware of that have very low defects rates: one bug report per month, or even one bug report per year. Whenever I've mentioned such a thing to people in the past (there's been a few other projects reporting significant reductions of bug-rates when they started doing XP - the ones Fowler mentions are new to me), they can't believe if. Their immediate reaction to reply with a comment like: "yeah, because they have no users or no software", or suggest that the people reporting these bug-rates are lying.

As I've mentioned before, Weinberg's Laws of consulting: ""In spite of what your client may tell you, there is always a problem.... Never promise more than a ten percent improvement. (If it were possible to achieve more than a ten percent improvement, there must have been a problem, but there isn't a problem, so...)"

[/docs] permanent link

Low-Carb Programming

Nothing to say... just wanted to use the phrase "Low-Carb Programming" before it becomes trademarked.

[/docs] permanent link

Cognitive Overload and TDD.

Eric Benson's quicklinks are almost always very interesting. Today he linked to this paper on Cognitive Overload. One footnote says:

Multi-tasking is a term drawn from computer science to refer to systems which handle many tasks seemingly at once. An operating system multi-tasks by rapidly switching between tasks and placing in an intermediate store all state knowledge required by each task. It therefore interrupts each task at a stable moment, and then later swaps back the state it was in before the interruption and carries on from where it was. Humans need to be able to stabilize their state knowledge when they are interrupted or they risk losing their place when they attempt to pick up the task later.

One thing that Kent Beck mentioned in Test Driven Development was that if he's not finished test-driving a chunk of code, but has to leave it for a while, he'll leave one test failing. When he gets back to pick up where he left off, he runs the tests, sees the failing one, and by making that test pass, gets back into the mindset needed for that design problem.

Of course, when working in a team, checking in a failing test is the height of bad manners.

[/docs] permanent link

Two Writers

Jerry Weinberg interview:

Are you working on any book[s] at present?

Yes. (I suppose you'd like to know which, but I've learned it's not a good idea to discuss your books before they're hatched. But I have about 30 books in process, so they cover a lot of ground.

If you had a webcam on your shoulder throughout a "typical" workday, can you give us an outline of what we would see?

I don't have a typical workday. On writing days, you would see a computer screen with a cursor moving rapidly across spewing words, perhaps for 8 or 10 or 16 hours. On a consulting workday, you would see and hear lots of people talking and demonstrating their work to me, with a very small amount of time with people listening to me. On a training workday, you would be watching lots of people eagerly doing problem-solving exercises, laughing, scratching their heads, and undergoing various sorts of emotional turmoil. Some of the time you would see and hear them discussing what they just went through. One thing you would never see is PowerPoint slides.

Another writer is a women I knew socially (circa 1992): Roxanne Longstreet Conrad. Quoting her timeline:

1989: After another brief hallucinatory stint as a professional musician, which ends when she is required to dress in Liederhosen and play a fundraiser for the Republican Party, Roxanne's boyfriend suggests she attend a science fiction convention to meet people who might be equally deranged (other writers)....

1992: Roxanne hangs up her musicial ambition to pursue her writing.  She sells her vampire novel, The Undead, shocking not only her boyfriend but her roommate as well (luckily in a good way).  The boyfriend leaves, but Roxanne meets and marries Cat Conrad, long-haired artist and all-around great human being.

Good to see that she's still writing. I enjoyed her vampire novels, and I'm ordering one of her latest books (seems she's using a pseudonym for this.)

[/docs] permanent link

Type Checking

You might think it's obvious that compile-time type-checking is valuable for preventing mistakes. For example, David Warnock wrote "obvious I think, if the language cannot tell that x is a string or an integer then lots of easy mistakes are not caught early". Now I've done most of my programming in compile-time type-checked languages (C, C++, Java, Pascal, Object Pascal), and I'm comfortable with them, though sometimes I find them overly verbose and tightly-coupled. A problem with the tight coupling is when you change something from an 'int' to a 'long int' deep inside a class somewhere, and the change ripples across from header file to header and source files until you've had to change half a dozen files.

Unfortunately for we who are using C and C++, static type checking is broken for certain basic types, in order to be compatible with older C source code. For example, C/C++ can't tell the difference between a string or an integer in many cases, because 'char *' converts to 'int' silently. That's actually a problem with a string represented as a character-array, but some string classes (thankfully, not std::string) have a 'conversion operator' that returns a 'char *' and the same silent conversion problem can happen with those string classes. 'bool' also converts to/from 'int' silently.

Now the dynamic-typing people say that, in practice, it's very rare to pass a string into a function that expects an integer, or vice versa. (I have rarely experienced such a problem in C++ when trying to do manual refactoring, but that was quickly detected via crashes when running tests and the application.) However, there is a big loop-hole of type-checking in any language that has polymorphism, whether dynamically-typed or staticly-typed: your functions may require arguments whose type implement a certain interface (or be a subclass of a certain class), but there is nothing that forces the objects being passed into those functions to implement anything meaningful. In practice, this isn't a big problem - in fact, it happens so rarely that static type-checking advocates don't even think of this problem.

By the way, Eiffel allows you to implement inherited assertions (invariants and pre- and post- conditions); useful if you're very serious about program correctness. There are arguments that extensive unit tests (and test-driven development) eliminate the need for the Eiffel-style assertions. I'm not entirely convinced. I find that assertions are good for documenting 'negative' facts like "argument x must not be null" and unit tests are good for documenting 'positive' facts like "this function returns 12 if the arguments are 4 and 2." There have been several occasions where a failing "precondition" assertion alerted me to problems in my usage of a class in test-driven development.

Try out Objective-C: it has optional type-checking. To turn off type-checking, declare variables as type 'id', and/or declare only classes (and their member data) and class-methods in your header files, and implement the instance methods in the source file without declaring them in the header. Since the instance methods aren't declared in the header file, they won't be checked where you call them. (Some programmers may be doing this as a way of keeping "private" methods a little more private.) If you encounter an error because of the lack of compile-time type-checking, then declare the variables with the appropriate class pointer types instead of 'id', and put instance method declarations in the header files.

To sum up: arguments for compile-time type-checking ignore some realities: some popular languages, like C/C++ have weak type-checking for certain types, but in practice, there are few problems with those types, and the problems are usually detected when the code is tested. Most languages that have polymorphism also weaken compile-type type-checking, because they only guarantee that methods exist and have the right interface/base-class types of their arguments, but do not guarantee the methods do the right thing; again only testing confirms that you really have the right types. C/C++, Java, and some other languages also weaken compile-time type-checking by allowing type-casting, and the compiler in those languages can't guarantee that an object (pointer) variable isn't null / null-pointer - a null pointer is never the right "type" if you're trying to call methods on it. (Don't you just hate crashes and NullPointerExceptions? Is your favorite language preventing those?)

[/docs] permanent link

Supercomputer Video

Video on Virginia Tech building a 10+ teraflops supercomputer from 1000+ Mac G5's in three weeks: http://www.apple.com/hardware/video/virginiatech/ "For a price of 5.2 million dollars, practically anyone could build a supercomputer...". Compare to the 30% faster super-computer (at Los Alamos Labs) that cost 215 million dollars.

[/docs] permanent link

Immutable Objects

Immutable Objects are simply objects that can't be changed. Because they can't be changed, you never have to spend time debugging unintentional changes. Mutable objects, if access is not carefully controlled, can be as bad as global variables when it comes to figuring out how an object got changed.

In Eric Evan's book, Domain-Driven Design, he divides domain objects into Entities and Values, where Values are immutable objects. One of his examples is a Person as an Entity, and the person's Address or ContactInfo as a Value.

I will demonstrate... To keep the example short, I will leave out all of contact info's data except for phone number, and not worry about the format of the phone number. (Nor will I worry about persistence, databases, etc.) We'll have a Person Entity object and a ContactInfo Value object.

class ContactInfo
{
    public ContactInfo( String phoneIn )
    {
        phone = phoneIn;
    }

    public void PrintOut()
    {
        System.out.println( "ContactInfo = {" );
        System.out.println( "  phone = " + phone );
        System.out.println( "}" );
    }

    public boolean matchesPhone( String phoneIn )
    {
        return ( phone.equals( phoneIn ) );
    }
    private String phone;
}
 
class Person
{
    public Person( String nameIn )
    {
        name = nameIn;
    }

    void setContactInfo( ContactInfo ciIn )
    {
        myContactInfo = ciIn;
    }

    public boolean matchesPhone( String phoneIn )
    {
        return myContactInfo.matchesPhone( phoneIn );
    }

    public void PrintOut()
    {
        System.out.println( "Person = {" );
        System.out.println( "  name = " + name );
        System.out.print( "myContactInfo = " );
        myContactInfo.PrintOut();
        System.out.println( "}" );
    }

    private String name;
    private ContactInfo myContactInfo;
};


public class ImmutableObjectDemo 
{
    public static void main (String args[]) 
    {
        Person aPerson = new Person( "Stan" );
        ContactInfo stanInfo = new ContactInfo( "555-STAN" );
        aPerson.setContactInfo( stanInfo );

        aPerson.PrintOut();

        System.out.println( "aPerson matchesPhone 555-STAN = " 
            + aPerson.matchesPhone( "555-STAN" ) );

        ContactInfo newStanInfo = new ContactInfo( "555-1234" );
        aPerson.setContactInfo( newStanInfo );

        aPerson.PrintOut();

        System.out.println( "aPerson matchesPhone 555-STAN = " 
            + aPerson.matchesPhone( "555-STAN" ) );

        System.out.println( "aPerson matchesPhone 555-1234 = " 
            + aPerson.matchesPhone( "555-1234" ) );
    }
}

Running the above provides this output: (indentation cleaned up)

Person = {
    name = Stan
    myContactInfo = ContactInfo = {
        phone = 555-STAN
    }
}
aPerson matchesPhone 555-STAN = true
Person = {
    name = Stan
    myContactInfo = ContactInfo = {
        phone = 555-1234
    }
}
aPerson matchesPhone 555-STAN = false
aPerson matchesPhone 555-1234 = true

ContactInfo is immutable simply by keeping its data members private, and not providing any member functions that modify its data. If I had implemented an accessor method to return the phone number, that would be safe, because the I used a String to hold the phone number, which is also immutable. Person is not immutable because "setContactInfo" changes its member data.

Now let's say we need to provide an "update phone number" method on Person. This is one implementation:

 
class Person
{
....
    public void updatePhoneNumber( String newNumber )
    {
        myContactInfo = new ContactInfo( newNumber );
    }
...
}

However, that can get rather wordy (imagine all the other data that I'm leaving out:

 
class Person
{
...
    public void updatePhoneNumber( String newNumber )
    {
        myContactInfo = new ContactInfo( newNumber,
            myContactInfo.getStreetAddress(),
            myContactInfo.getCity(),
            myContactInfo.getState(), 
            myContactInfo.getZip() );
    }
    public void updateCity( String newCity )
    {
        myContactInfo = new ContactInfo( 
            myContactInfo.getPhoneNumber(),
            myContactInfo.getStreetAddress(),
            newCity,
            myContactInfo.getState(), 
            myContactInfo.getZip() );
    }
...
}

So it would be better for ContactInfo to provide us with a new copy of itself, with the updated data. This will provide better cohesion as well -- if we added a new field (like 'company name') only methods in ContactInfo would have to change; the existing methods in Person will not have to change.

class Person
{
...
    public void updatePhoneNumber( String newNumber )
    {
        myContactInfo = myContactInfo.newPhoneNumber( newNumber );
    }
...
}

class ContactInfo
{
...
    ContactInfo newPhoneNumber( String phoneIn )
    {
        return new ContactInfo( phoneIn /* and address, city, 
                    state, zip, and other members... */ );
    }
...
}

Given that ContactInfo is immutable, we have an opportunity to maintain an list of previous ContactInfo objects that we would not have had if we had just been modifying the data of a ContactInfo object in-place. Suppose we talk to our customer about this, and he says that yes, we want to be able to find a Person given an old phone number. So we can make the following changes (see lines with /*new*/ added:

import java.util.*;

class ContactInfo
{
    public ContactInfo( String phoneIn )
    {
        phone = phoneIn;
    }

    public void PrintOut()
    {
        System.out.println( "ContactInfo = {" );
        System.out.println( "  phone = " + phone );
        System.out.println( "}" );
    }

    public boolean matchesPhone( String phoneIn )
    {
        return ( phone.equals( phoneIn ) );
    }
    
    ContactInfo newPhoneNumber( String phoneIn )
    {
        return new ContactInfo( phoneIn );
    }
    private String phone;
}

class Person
{
    public Person( String nameIn )
    {
        name = nameIn;
        oldContactInfo = new Stack();
    }

    void setContactInfo( ContactInfo ciIn )
    {
        myContactInfo = ciIn;
    }

    public boolean matchesPhone( String phoneIn )
    {
        if ( myContactInfo.matchesPhone( phoneIn ) )         /*new*/
        {
            return true;                                     /*new*/
        }
        
        ListIterator i = oldContactInfo.listIterator();      /*new*/
        while ( i.hasNext() )                                /*new*/
        {
            ContactInfo oldContact = (ContactInfo) i.next(); /*new*/
                
            if ( oldContact.matchesPhone( phoneIn ) )        /*new*/
            {
                return true;                                 /*new*/
            }
        }
        return false;                                        /*new*/
    }

    public void PrintOut()
    {
        System.out.println( "Person = {" );
        System.out.println( "  name = " + name );
        System.out.print( "myContactInfo = " );
        myContactInfo.PrintOut();
        System.out.println( "  oldContactInfo = {" );
        ListIterator i = oldContactInfo.listIterator();      /*new*/
        while ( i.hasNext() )                                /*new*/
        {
            ContactInfo oldContact = (ContactInfo) i.next(); /*new*/
            oldContact.PrintOut();                           /*new*/
        }
        System.out.println( "}" );                           /*new*/
        System.out.println( "}" );
    }
    
    public void updatePhoneNumber( String newNumber )
    {
        oldContactInfo.push( myContactInfo );               /*new*/
        myContactInfo = myContactInfo.newPhoneNumber( newNumber );
    }

    private String name;
    private ContactInfo myContactInfo;
    private Stack oldContactInfo;                           /*new*/
};

public class ImmutableObjectDemo 
{
    public static void main (String args[]) 
    {
        Person aPerson = new Person( "Stan" );
        ContactInfo stanInfo = new ContactInfo( "555-STAN" );
        aPerson.setContactInfo( stanInfo );

        aPerson.PrintOut();

        System.out.println( "aPerson matchesPhone 555-STAN = " 
            + aPerson.matchesPhone( "555-STAN" ) );

        aPerson.updatePhoneNumber( "555-1234" );

        aPerson.PrintOut();

        System.out.println( "aPerson matchesPhone 555-STAN = " 
            + aPerson.matchesPhone( "555-STAN" ) );

        System.out.println( "aPerson matchesPhone 555-1234 = " 
            + aPerson.matchesPhone( "555-1234" ) );
    }
}

With this output (indentation cleaned up):

Person = {
    name = Stan
    myContactInfo = ContactInfo = {
        phone = 555-STAN
    }
    oldContactInfo = {
    }
}
aPerson matchesPhone 555-STAN = true
Person = {
    name = Stan
    myContactInfo = ContactInfo = {
        phone = 555-1234
    }
    oldContactInfo = {
        ContactInfo = {
            phone = 555-STAN
        }
    }
}
aPerson matchesPhone 555-STAN = true
aPerson matchesPhone 555-1234 = true

Now, we may actually want our intentions to be more clearly revealed after this last change. For example, we could have two methods named "matchesCurrentPhone" and "matchesPreviousPhones" to break down that slightly-too-long "matchesPhone" function. We might even rename the "matchesPhone" function to something like "matchesCurrentAndPreviousPhones".

Immutable objects imply Side-Effect-Free Functions, which Eric Evans says is one of the idioms that contribute to "Supple Design". The other idioms listed in his book are Assertions, Intention-Revealing Interfaces, Standalone Classes, and Conceptual Contours. Check it out.

[/docs] permanent link

Political Humor

Spent an hour in my car, in a parking lot, listening to an interview with Steve Bhaerman a.k.a. Swami Beyondananda on a local public radio station. Made me smile many times. (Also lots of puns, though not ones that make you groan - they make you grin, instead.) He does live performances, and he's written a book, Duck Soup for the Soul and has a web page: http://www.wakeuplaughing.com/.

Based on what I've heard, I'll look for his books at the bookstore, and keep an eye out for his next performance in the Bay Area. He wrote another book titled Driving Your Own Karma: Swami Beyondananda's Tour Guide to Enlightenment and apparently another one that's out of print: When You See a Sacred Cow... Milk It for All It's Worth

He mentioned that there is a documentary called "Bringing Down a Dictator" about how a group a young people "used resourcefulness, theater, nonviolence and yes -- comedy -- to put an end to Slobodon Milosevich's tyrranical regime in Serbia without a shot being fired." First I've heard of this, but then I haven't made much of an effort to get non-USA-filtered news. It's available in video (probably VHS-NTSC videotape) at Bhaerman's products page.

He say's he's not anti-Republican, but that our current situation of a strong right-wing and a weak left-wing, means that we can't fly straight. He's heartened that the "silenced majority" is beginning to find its voice, and it's one that can laugh.

[] permanent link

Mars

Martian to other Martian (watching astronaut exiting a lander): "There goes the neighborhood!" I'm visualizing a old cartoon where an American Indian says that to another, as they watch Pilgrims disembarking from the Mayflower.

So why is George Bush talking about sending humans to Mars and the moon? Election-year politics? Probably. Trying to do everything his father tried to do? Maybe. I haven't heard anyone mention lately that China is planning to send manned missions to Mars and the Moon. Their success at putting a man in orbit was page-four news in the USA, but front-page news elsewhere. The revival of the Space Race is a way for the USA to say that we are still a power in technology and science, not to be eclipsed by China, even though China manufacturers almost everything that Walmart and other retails are selling in the USA today.

Mars-nay-sayers say they we should be solving problems at home, before "sending" people elsewhere. Is that what the proto-American-Indians said before they crossed the Bering Strait into the Americas? Or the Vikings before they sailed west? Or the Spaniards and English and other Europeans? (Or the slaves?) People, as long as they remain human, will always have problems. No sense in delaying space exploration (and colonization) forever until we "solve" humanity.

I want humans on the moon and Mars, and elsewhere, so we have a "backup" for our species if life on Earth gets destroyed by extinction-event-sized asteroid. I'm confident that even if we do send people into space, we'll still be working on the problems at home as well. Of course, we need to clean up our environment - it's becoming unsafe to eat fish more than once a week from mercury and other pollutants in and around our coastlines. Let's hope that if/when we terra-form and colonize Mars, that we treat its environment better than we've treated our own so far. But we gotta make those first steps.

[/docs] permanent link

Pluggable Components

Responding to James A. Robertson's assertion that buying and putting together components - at the user level - never works...

First, there is a whole industry of Photoshop plugins and other plugins for various host applications (video editing). If you don't work in those fields, you don't hear about them, but these kinds of components are a successful business for many companies.

Also, there was a brief time (some of like a Camelot) where Apple's OpenDoc seemed to be on the road to success, in spite of being written in C++ (which means that any component crashing would crash your whole host application.) Most of OpenDoc activity was on MacOS, though IBM built an implemention on OS/2 and someone (Apple?) built an implementation on Windows. While the WikiPedia says that the timing wasn't right (the rise of Java), I'd say that Apple's faltering finances at the time actually doomed it.

The most successful use of OpenDoc was CyberDog which had components to handle each of the internet protocols (http, ftp, mail, etc.) and mime types (html, images, etc.). There were at least two commercial word processors that were OpenDoc hosts, and we were eagerly awaiting ClarisWorks (now AppleWorks) reimplemented as OpenDoc components and host application (it was was demoed but never shipped). Web-sites (and maybe even OpenDoc components themselves tied to websites) to help you find and buy OpenDoc components were springing up. There were some companies offering free readers components and non-free editor components so you could distribute documents authored by commercial components and have people be able to read them. The true strength of OpenDoc over JavaBeans was that the "Bento" file format allowed creating a single document from multiple OpenDoc components.

When Apple had financial difficulties and accepted a large sum of money ("investment") from Microsoft, Microsoft required them to kill off OpenDoc because it competed with OLE/Com/ActiveX and kill off CyberDog because it competed with Internet Explorer. [This same agreement allowed Microsoft to retroactively license patents from Apple that it was violating.] Once OpenDoc was killed off, Microsoft's "ActiveX on MacOS" effort evaporated, leaving third-party "ActiveX on Mac" developers high and dry.

I regarded OpenDoc as overly complex, but it was technically workable. Java Beans were too simplistic - they couldn't be used without having a programmer put them together, and didn't have a document format like Bento. OpenDoc was actually easy for a user to use, but hard for a developer. If not for Apple's finances, and Microsoft's monopolistic threats, I think it would have succeeded. I think it would have been more easily successful if OpenDoc had been implemented on a robust OS (like MacOS X), maybe with each component running in a separate process space (so that one component crashing would not crash your entire app), and if the saved data included a rendering (an image/text format like PDF, so you didn't have to distribute "reader" OpenDoc components with your document), or if the OpenDoc components could have been embedded in your document (like fonts can be embedded in a PDF).

A re-implementation of OpenDoc (and the Bento file-format) using Cocoa/Objective-C would be very powerful (and easier for programmers) because it would be able to avoid a lot of C++ rigidity and overhead. Implementing in a almost crash-proof language like Smalltalk or Python would also be a win. If the actual components ran in separate address spaces, using some "distributed object" protocol, then it wouldn't matter what language your wrote the component in.

[/docs] permanent link

Wireless

Quoting Bruce Eckel out of context "if I were wireless and could drift around the house while connected to the net, that could seem like an excessive luxury but it would be helpful enough to make it worthwhile." Once you go wireless, you find uses that you wouldn't consider when tied down at a desk. For example, rather than look in the newspaper's TV guide, I look up http://tvlistings2.zap2it.com/ on my laptop, while sitting in my lazy-boy recliner. More information, more up-to-date.

[/docs] permanent link

Theory of Constraints Animated

Any system can produce only as much as its critically constrained resource; if it were not so, the output of a system would tend towards infinity or zero. A very cool animation of this theory is here: http://www.tocca.com.au/Services/demoprojectM.htm.

[/docs] permanent link

Iterative Design of Skis

Quoting "A Bad Skier's Revenge" of American Heritage of Invention & Technology

In 1947 [... Howard Head ] blamed “those long, clumsy hickory skis.” Unlike most first-time skiers, however, he had an idea about how to improve them. “If wood were the best material,” he said, “they’d still be making airplanes out of wood.”

[...] he took a hickory ski and conducted his own stress test on it. He was amazed to discover that hickory’s strength was two times what was stated in the engineering handbook. He had designed his ski so it was too weak! He replaced the fragile honeycomb with a marine-fir plywood core. During the next two years he and his crew built 40 different prototypes. Many years later he recalled, “If I had known then that it would take 40 versions before the ski was any good, I might have given it up. But, fortunately, you get trapped into thinking the next design will be it.” [...]

[...] From the time Head offered his Standards until the time he sold his company, ski manufacturing enjoyed 15 percent growth every year.

[/docs] permanent link

More Python Multiple-Returns

John Roth, via email, tells me that "multiple-return" like this:

 quotient, remainder = divmod(divisor, dividend) 

is actually a syntactic short-cut for this:

result = divmod(divisor, dividend)
quotient = result[0]
remainder = result[1]

The divmod function returns a tuple (an immutable list) and the language automatically unpacks it for you. When I wrote my earlier blog entry, I actually did think of the this "divmod" function... except I couldn't recall the exact name (almost remembered the Forth equivalent, useless in this context), and I was, of course, too lazy to look it up in the Python docs.

Once on comp.lang.smalltalk, there was a discussion on how hard it would be to add increment-assignment to Squeak's Smalltalk compiler. Someone posted the actual code, and it was only a few short modified methods. I expect that automatic-unpacking, ala Python "Multiple-Returns", would also be a small modification to the compiler, that most Smalltalk environments could support each in their own way.

[/docs] permanent link

Data Without Meaning
Patrick Logan's weblog has the following:

Jay Han responds to my item on using databases for coordination...

We know that databases provide concurrency control and transaction management. These features let applications share data -- you can call this coordination at low level. But what about coordination at high level? How can they exchange semantics of data? e.g. "9999-12-31 in date field means now? never? forever in the future?" [...]

As long as databases only store data, and don't store whole objects (data+algorithms), they will always risk disassociating meaning and values.

[/docs] permanent link

Returning Multiple Values

One of the neato things about Python is returning a small number of multiple values from a method is easy. For example, finding the elements that are shared by two collections/lists [hope I got the syntax right]:

   beforeNotAfter, afterNotBefore = foo.findNonOverlappedElements( before, after )
   # do stuff using beforeNotAfter
   # do stuff using afterNotBefore

Alan Knight says the Smalltalk "moral equivalent" to this uses a block (my paraphrasing here):

    foo findNonOverlappedElementsOf: before and: after 
        doing: [ : beforeNotAfter : afterNotBefore
            "do stuff to beforeNotAfter".
            "do stuff to afterNotBefore" ].

Coming from a background of C/Object-Pascal/Objective-C/Java/C++, it's hard for me to think of reasons to do a Python-style multiple-return; instead of one method that returned two values, I'd write two methods, or single method that returns a data-structure or has output-parameters. And, unfortunately, Objective C, Java, etc., don't make "blocks" easy. Though there is a dialect of Objective-C that has blocks, it's not the dialect supported by gcc and Apple.

The other aspect of the Smalltalk solution here is "tell, don't ask". We tell 'foo' to find the non-overlapped elements, and we tell foo what to do with the resulting lists of elements (execute our block). This forces a certain cohesion to our code. The Python example is violating the spirit of the Law of Demeter, if not the actual Law (1) because we ask and then work the results of what we asked.

footnote 1: Law of Demeter (LoD) is really just a guideline that is particularly questionable when collections are involved.

[/docs] permanent link

Carnegie Hall

"How do I get to Carnegie Hall?" "Practice, man, practice!"

People have been complaining about bad software for a long time. If you want to get good at something, you have to practice it (with feedback). We want people to get good at design, and the best way is to design all the time, or as often as possible, with feedback from trying to use and modify the design. Ditto for "architecture", maintainable /readable code, and so on. (And people and project management skills? Practice can't hurt.)

[/docs] permanent link

Ward Cunningham

A January 5, 2004 Interview with Ward Cunningham, via Bill de hÓra on flattening the cost of change curve:

Traditionally, the cost of change curve said that if we detect the need for a change early, it costs less to make the change than if we detect the need late. [...]

Our feeling was that the limiting factor on any change was not when it was done, but how much thinking was required. If we made a change during week one, and it took us two days to understand what was really required, it took two days to make the change. If we made a change during week 21, and it took us two days to understand what was really required, it took us two days to make the change.

In week one, we might have had to write 20 statements. In week 21, we might have had to write 20 statements and change four. But if you practice making changes, the time it takes to change four statements is not that great.[...]

Many people are afraid of changes because although they understood the code when they wrote it, that understanding has disappeared. [...] people must be able to look at what they're going to change and understand it.[...]

[...] I hate it when a new requirement comes in that doesn't fit nicely, as if the program were designed to make the requirement hard. In that case, we have a lot of work to do. But the nature of the work is first changing the program so the new requirement is an easy fit, and then doing the easy work to incorporate the requirement. [...] The patch approach means that the next guy who comes along will have to understand both the system that wasn't made to do the new requirement, and the patch that tried to overcome that system without changing it. [...]

Now somebody might say, "Why don't we look forward, look at all the work we have to do? Why don't we design a system that makes all work easy from the beginning?" And if you can pull that off, that's great. It's just that, over and over, people try to design systems that make tomorrow's work easy. But when tomorrow comes it turns out they didn't quite understand tomorrow's work, and they actually made it harder.

I like the notion of working the program, like an artist works a lump of clay. [...] the more she handles the clay, the more the clay tends to do what she wants. [...]

A development team works on a piece of code over several months. Initially, they make a piece of code, and it's a little stiff. It's small, but it's still stiff. Then they move the code, and it gets a little easier to move. In a project I mentioned earlier [in Part II], we added a schema evolution capability to our database. It softened up the program. It was much easier to change. Every time we did a schema change and we got better at it. The programmers and the code—as a unit—softened up. We worked the program, and we kept it flexible.

That bit about schema evolution shows the difference between "all design up-front" versus incremental/iterative design. The "all up-front" approach would try to get the schema right, once, and not necessarily allow for changing the scheme. And of course, at some point, changing the schema is very likely to be necessary - and it will be hard work because it will be done late, with the code assuming that it will never change.

In the incremental/iterative approach, the schema will first be created for the requirements of the current iteration (perhaps in the first two-week iteration), and then be altered in successive iterations to meet the requirements of the stories scheduled for those iterations. To be able to handle those changing schemas, the team will have to develop code that can handle schema evolution. Every time something breaks because of a schema change, the team fixes the code (and adds tests) in such a way to prevent breakage of that kind from happening again. By the time the end of the project is reached, the code is "soft" and flexible, easy to change, easy to understand, because the team had to make it that way just to be able to work on it during the project. This becomes a much more valuable asset to a company than inflexible code that is hard (and expensive) to maintain and enhance.

[/docs] permanent link

A History of Smalltalk

http://www.mojowire.com/TravelsWithSmalltalk/DaveThomas-TravelsWithSmalltalk.htm

[/docs] permanent link

Quotes2

"Concentrate on clear objects and you will eventually produce reusable ones; concentrate on reusable objects and you will produce muddled ones." -- Chamond Liu in the book Smalltalk, Objects, and Design

"The reason a lot of people do not recognize opportunity is because it usually goes around looking like hard work" - Thomas Alva Edison

"I couldn't wait for success... so I went ahead without it" - Jonathan Winters

"Mac OS X. Because making Unix user-friendly is easier than debugging Windows." - unknown

"I am Homer of Borg, prepare to...mmm, donuts." - unknown. Probably not said on the Simpsons

"The fact that you know more today, and are more capable today, is good news about today, not bad news about yesterday." -- Ron Jeffries

"The Only Consistent Feature of All Your Dissatisfying Relationships is You." - http://www.despair.com

[/docs] permanent link

Blinker Chess

I've had an idea of a computer-mediated chess game with a twist: some of the pieces, some of the time, are invisible to one's opponent. I call the idea "Blinker Chess". Each player starts the game with approximately one-third of your chess pieces (randomly selected) invisible to his opponent. Each turn, one of that player's pieces (again randomly selected) changes state: either becomes invisible or visible. You know which of your pieces are not visible to your opponent (they show on your screen differently).

If you attempt to move your piece to some square, but an invisible piece would get in the way, that other piece becomes visible, but you forfeit that move. If you move a piece onto a square that is already occupied by your opponent, whether by a visible or invisible piece, you capture it (and you see it, briefly, if it had been invisible.) Pawns, of course, capture on the diagonal as per chess rules, but if their forward movement is blocked, the blocking piece becomes visible.

Your King is never invisible. You may not know when your King is in check, because the threatening pieces may be invisible to you. The game ends in capture of the King more often than by checkmate. The "tournament chess" rule about not castling when under threat doesn't apply to Blinker Chess.

If you move an invisible piece, your opponent doesn't see it. You can also "pass" -- not move a piece -- to give your opponent the illusion that you moved an invisible piece; your opponent will not know that you passed instead of moving.

If you've played a chess variation like this, send me an email. If you know of an open-source chess game viewer, for peer-to-peer chess playing, please let me know. My email address is keithray@mac.com.

[/docs] permanent link

Unconscious Choices

Back in Novemenber, Laurent Bossavit wrote:

On occasion, I've asked colleagues why they choose to organize their software work as "projects", as opposed to other forms of organization. The response was : "What other forms of organization ?" Like Moliere's gentleman who had been talking prose all his life without even knowing it, many of us, including project managers, never question why software development should take the form of "projects".

Projects are defined as having a beginning, middle, and end, probably because we, as human beings, like to have stories have a beginning, middle, and end. (And that preference become reflected in accounting practices, law, hiring practices, and so on.) While I enjoy reading/watching fiction and story-telling greatly, I've always felt that real-life doesn't have precise beginning and endings (other than births and deaths). This may be the root cause of my not writing much fiction - yet.

I tend to think of software development as continuous, because I'm usually involved in developing, maintaining, and shipping multiple versions of a software product, not just one "project". After a product version ships, I'm still working on it to fix bugs, and I use it as a base for making the next version. Before one version ships, I may be involved in defining the requirements of the next version, so one "project story" overlaps the next in time. When I worked at Apple, we used the "train" metaphor for one of the products I worked on (the ETO [Essential Tools and Objects] subscription): the train leaves the station three times a year, and whatever is ready to ship gets on the train.

I've sometimes used the term "unconscious waterfall" to describe the serial process used on many projects that I've seen or heard of: a series of phases that start with requirements gathering, a design phase (this seems to be optional), has a long coding phase, and a testing (and fixing) phase crammed in at the end (and often the testing phase is too short). Quite often, the phases had beginnings and ends at prescribed calendar times, rather than when the products of each phase were really "complete". Almost all of the people involved in these projects were unaware of the possibility of iterative, incremental, software development.

How many other unconscious choices are we making?

[/docs] permanent link

Four Strategies for Iteration

Say we a have a Disk object that contains a Vector of File objects, and some other code wants to find the largest file...

Stategy 1: export the container. The least "safe" example (I'm using java-style pseudo-code) just makes the member container public, or returns a reference to the member container. Safer variations return a copy of the member container (can be inefficient) or return an immutable container of immutable objects.

 class Disk
 {
    private Vector mFiles;
    // etc...
    public Vector getFiles() 
    { 
       return mFiles;
    };
 }
 class ExampleOne
 {
    public File largest( Disk d )
    {
        Vector files = d.getFiles();
        File largestFile = null;
        int largestSize = -1;

        for ( int i = 0; i < d.size(); i++ )
        {
            File f = files.elementAt( i );
            if ( f.getSize() > largestSize )
            {
                largestSize = f;
                largestFile = f;
            }
        }
        return largestFile;
    }
 };

Multiply this strategy by a lot of classes, a lot of containers, and a lot of code doing these kinds of query, and that for-loop gets duplicated everywhere... nasty.

Strategy 2: export iterators instead of containers. Slightly better than the previous as we can hide exactly what kind of container Disk has as a member. This also allows Disk to not have a container at all, and the Iterator it returns could makes operating-system calls instead of getting items from a container. If I remember correctly, hiding the container-type / iterator-type is a bit difficult to do in C++ if you're using STL container classes, because the iterators in that framework don't have a common base class declaring virtual methods for iteration; you'd have to declare your own iterator base class and subclasses -- all because STL does "compile-time polymorphism" instead of "run-time polymorphism" for maximum efficiency.

 interface Iterator
 {
     bool hasNext();
     Object getNext();
 };
 class Disk
 {
    public Iterator getFilesIterator() 
    { 
       ... return a new iterator -- not necessarily one referencing 
           a container.
    };
 }
class ExampleTwo
 {
    public File largest( Disk d )
    {
        Iterator filesIt = d.getFilesIterator();
        File largestFile = null;
        int largestSize = -1;

        while ( filesIt.hasNext() )
        {
            File file = filesIt.getNext();
            if ( file.getSize() > largestSize )
            {
                largestSize = file.getSize();
                largestFile = file;
            }
        }
    return largestFile;
    }
 };

Strategy 3: implement "doForEach" methods in the container-owner class, and pass in "blocks" of code. [See Jeff Langr's Java Style for 'do', 'collect', 'select/reject', 'detect', 'inject into', etc...]

 public Interface OneArgBlock
 {
    public void execute( Object item );
 }
 class Disk
 {
    private Vector mFiles;

    public void doForEachFile( OneArgBlock block )
    {
        for ( int i = 0; i < mFiles.size(); i++ )
        {
            block.execute( mFiles.elementAt( i ) );
        }
        // or use an Iterator internal to this method...
    }
 }
 public class FindLargestFile implements OneArgBlock
 {
    File myLargestFile = null;
    int myLargestSize = -1;

    public void execute( Object item )
    {
        File file = (File) item;
        if ( file.getSize() > mLargestSize )
        {
            mLargestSize = file.getSize();
            mLargestFile = file;
        }
    }
 }

 class ExampleThree
 {
    public File largest( Disk d )
    {
        FindLargestFile finder = new FindLargestFile();
        d.doForEachFile( finder );
        return finder.mLargestFile;
    }
 };

This example takes up more code, but if used everywhere, this can be a powerful idiom and reduces the amount of code you have to write. Note that Java permits using an inline anonymous implementation of OneArgBlock, but C++ will require a named subclass. Smalltalk, Python, and Ruby are very concise because this idiom [anonymous instances of something like OneArgBlock] are built into these languages. This can even be done in C using function-pointers.

strategy 4: inline the functionality

 class Disk
 {
    private Vector mFiles;

    public File getLargestFile()
    {
        File largestFile = null;
        int largestSize = -1;
        File f = mFiles.elementAt( i );
        if ( f.getSize() > largestSize )
        {
            largestSize = f;
            largestFile = f;
        }
        return largestFile;
    }
 }
 class ExampleFour
 {
    public File largest( Disk d )
    {
        d.getLargestFile();
    }
 }

As you can see, this is the least invasive -- no one but Disk gets access to its File Vector -- and this example is the most compact. However, this can result in a class having too many methods, each of which are variations of each other... like getLargestFile(), getSmallestFile(), getOldestFile(), getYoungestFile(), getBluestFile() etc. If this becomes the case, you should consider using variations of strategy 3.

[/docs] permanent link

SF and Science

Science Fiction: MRI and a virus used to create willing, monomaniacal slaves: A Deepness in the Sky by Vernor Vinge. (Very good book, get A Fire Upon the Deep as well.

Science: A certain kind of MRI seems to benefit people with Bipolar Disorder... "We just happened to use this set of magnetic gradients, which we think somehow matches the natural firing rhythm of brain cells" reported in Science Daily

[/docs] permanent link

Object Oriented Robustness

Allen Holub has an article on Why extends is evil. The main points are two: that declaring variables and arguments to be of concrete class types makes changes more difficult, and that changing the internal behavior of a base class can cause problems for subclasses that relied on the internal behavior.

That second problem, Holub calls "the fragile base class problem". The "fragile base class problem" as described by Holub is NOT the "fragile base class" problem experienced in C++ where code that is compiled (at different times) against different versions of the same class can have incorrect offsets for the vtable (method pointers) and data members, sometimes resulting in mysterious bugs -- and stepping through the debugger shows the instruction pointer jumping around in ways that don't correspond to the visible source code.

Holub's "fragile base class problem" is just a specific instance of unexpected coupling between two classes (which are connected via inheritance in his example, but could be coupled by a "uses" relationship). This problem often immediately detectable if you have a thorough suite of unit tests (as practiced by TDD), and you run them all after making changes. If you have modified the base class so it doesn't perform the way the subclass expects, the subclass's tests should fail, alerting you to undo your change, or modify the class or its subclass.

His recommendation to use more 'interface' inheritance rather than 'implements' inheritance is a good one - not all problems can be seen in the tests, and the flexibility of interface inheritance allows using mock objects to get better tests ("what happens to Y if one of its calls on X fails?").

The first problem - being tied to too-specific type declarations goes away in languages that don't have type declarations. Smalltalk, Python, Ruby, Javascript, and so on. (And some new ones like Scarlet and Groovy, though Groovy has optional type declarations, like Objective C.

The "fragile base class" problem still exists in any language that has "extends"-style inheritance. Scarlet doesn't have inheritance; I'd like to hear how people are using it, but I don't know if it has any people programming in it other than its creator.

[/docs] permanent link

Equalizing IP Salaries

Quoting Business Week Online:

Six months ago, I could find high-level programmers in India willing work for $15 an hour, vs. the $100-plus an hour I was paying Americans for the same work. In only six months, that rate has climbed to $25 an hour in India, while my domestic rates have dropped to around $35-$50. On the last project I bid out, two proposals from India came in higher than domestic contractors. Admittedly, I'm in a very small sector of the larger market, and it's too soon to tell even here whether the trend will last, but I've heard similar reports from other businesses (see BW Online, 12/2/03, "U.S. Programmers at Overseas Salaries").

[/docs] permanent link