About
Exploring Solution Spaces, Keith Ray's blog on Software development and other topics.
Send comments to:
keithray@mac.com
| For Agile Training, eLearning, or Coaching contact: |
| Industrial Logic, Inc. |
| 866-540-8336 (toll free) |
| 510-540-8336 (Berkeley, California) |
| Links |
| xpminifaq |
| Résumé |
| “Adopting XP” Article 2002 (pdf) |
| “ Refactoring” Article 2006 |
| AYE Conference |
| Lucien W. Dupont |
| Elisabeth Hendrickson |
| Johanna Rothman's Managing Product Development |
| Brian Marick's Exploration Through Example |
| Esther Derby's Insights You Can Use |
| Laurent Bossavit's Incipient(thoughts) |
| Dale Emery's Conversations with Dale |
| Martin Fowler's Bliki |
| Creating Passionate Users |
Archives
Subscribe
RSS Exploring Solution Spaces XML
2006.Dec.26 Tue
Ed Gibbs writes of some code and unit tests he reviewed:
2006.Dec.24 Sun
- Clear test names.
- Only 2-4 lines of code for most of the tests.
- Thorough testing of negative conditions like testDuplicateRefundsOnlyCountOnce().
- 99% unit test coverage according to Clover.
Donate to Solar Cookers International
Refugees from Darfur face dangers daily, when they leave their camps to gather wood to cook their food. Donate to Solar Cookers International and make a concrete improvement in their daily lives.
Solar cookers are a win-win technology in sun-rich, fuel-scarce areas: they reduce smoke and lung diseases, pasteurize unsafe drinking water, and spare women and children the burdens and hazards of collecting ever-scarcer firewood for cooking.
You can also order Solar Cookers, or plans to build one for yourself, as a summer-time project for you and your children, or just for fun.
2006.Dec.19 TueThere's a new book about why some ideas survive and others die, by Chip and Dan Heath. I've been reading the excerpts... pretty interesting. Did you know that no children were ever harmed by tampered-with Halloween candy, or razor blades in apples, etc.? There were two incidents involving Halloween candy where children were harmed, but they harmed by their own parents, not by strangers. Yet this urban legend is changing how Halloween as celebrated in USA, and caused laws about tampering to be passed. (Hmm... did this start before or after the Tylenol scare of 1982? Apparently before.)
2006.Dec.09 SatSix Principles of Sticky Ideas
... unexpected danger in a common activity...
Both stories called for simple action...
Both made use of vivid, concrete images...
... tapped into emotion: fear ... disgust...
PRINCIPLE 1: SIMPLICITY...
PRINCIPLE 2: UNEXPECTEDNESS...
PRINCIPLE 3: CONCRETENESS...
PRINCIPLE 4: CREDIBILITY...
PRINCIPLE 5: EMOTIONS... it's difficult to get teenagers to quit smoking by instilling in them a fear of the consequences, but it's easier to get them to quit by tapping into their resentment of the duplicity of Big Tobacco.
PRINCIPLE 6: STORIES...
Some Web server statistics summarized from the entertaining "interesting stuff" web site Boing Boing:
| Browser operating systems | |
|---|---|
| Windows | 68.4% |
| Macintosh | 22.3% |
| unknown | 4.9% |
| Linux/Unix/Other | 4.1% |
| Browsers | |
|---|---|
| Firefox | 50.7% |
| MS IE | 26.2% |
| Safari | 12.1% |
| others | 10.5% |
"Inc. on Responding to Change"
In the article "Go Ahead, Make A Mess" in the December 2006 issue of Inc. magazine, we read:
Studies suggest that strategic planning is not only a waste of time, it's actually at least likely to hurt as help. [...]
Strategic planning essentially requires casting a blind eye to certain key facts: The world is an unpredictable place [...], markets are complex, competitors will counter your every move, and most managers fail to recognize the ways in which their own organizations deviate from their neat conceptions. [...]
According to the source cited in the article, companies that emphasize strategic planning did not perform better, on the average, than companies that do less planning.
The article also suggests that keeping things tidy has a cost: "people who claim to have 'very neat' desks report spending 36 percent more time looking for things than people who say they have 'fairly messy' desks."
2006.Dec.05 TueI'm happy to announce that I have joined Industrial Logic, a training and coaching company headed by Joshua Kerievsky. Please contact Industrial Logic if you want training or coaching on Design Patterns, Refactoring, Test-Driven Development, and/or agile software development with Extreme Programming or I.L.'s superset of XP, IXP.
Check out I.L's Smells to Refactoring Cheat Sheet (pdf) and the demo of one of I.L's eLearning products "Refactoring to Patterns Interactive".
Dave W. Smith has posted some symptoms that indicate your team isn't refactoring enough to pay down accumulating design debt. Check it out.
2006.Nov.27 MonEstimation Spread
Jockeying for Stories
Depending on a Drawing
Puzzling Tests
How do you know it's done? (Speaking of some amount of code) That's a good question to ask, but perhaps the better one is "How can you show that it's working correctly, and that it is still working correctly after someone has changed something when you are not present?"
BayXP Meeting on jBehave and rSpec
ShaneDuan will be speaking on jBehave, rSpec, and Test Driven Development, 6:00 PM, November 29, 2006, at Digital Chocolate, 1855 South Grant Street, San Mateo, CA 94402
2006.Nov.20 MonAbstract: As TDD becoming more accepted development practices and we gaining good experience on what it makes a good TDD, some have taken a second look at what we can do to improve the framework that we are using, so that it fits into our style of programming, rather than the other way around. Two good examples are jBehave, and rSpec. Here we are going to take a brief look at how they are trying to achieve this, and share our ideas of TDD without the restriction of the tools.
Comparison of Web App Frameworks
I thought this was interesting: Dave Pollak on Java, Smalltalk, Objective-C, and Ruby web-app frameworks.
It's too long for me to summarize, but he does mention some weaknesses in Ruby on Rails that I haven't seen elsewhere: security, stability, lack of multiprocessor support, etc. (And slowness and lack of unicode support, which I've already heard about.)
This guy says good things about NextStep (too bad he hasn't tried WebObjects), and mentions both strengths and weaknesses in Seaside on Squeak Smalltalk, as well as describing some features of Seaside that I haven't heard of before.
2006.Nov.19 Sun[...] even though haven't gotten to my "hate" point with Seaside (I've gotten to that point with most technologies, although with NextStep and Java, I came back to loving them... with Ruby, the jury's still out) I think it's got the right pieces parts to make it a quantum shift in web development. I think the kind of apps that could be commonplace with Seaside are totally not doable with any other web technology.
Jane's Rule for Loading Dishwashers
I've been married for nearly 14 years, and I've only recently come to understand my wife's rule for loading silverware into our dishwasher. (Married women reading this blog may roll their eyes now.) It's only recently that she explained her rule and after trying it, I came to understand why it makes the washing-and-putting away process faster.
In order to try it, of course, I had to get over the idea that what I was doing was "the right way" and open myself to the idea that there might be a better way.
As you can see in the picture above, there is a sub-divided bin for silverware in the dishwasher's lower rack. Jane's rule for putting silverware into a dishwasher is to put all the forks in one sub-bin, all the spoons in a different sub-bin, all the steak knives in one sub-bin, and so on. This doesn't happen all at once, but only as silverware gets used and then put into the dishwasher for cleaning. Or taken from the sink where it's been soaking.
The true benefit of this comes not in the "loading the dishwasher" time, since this is slightly more effort than just sticking silverware into random sub-bins. The benefit comes when it is time to unload the dishwasher, putting the clean silverware into the drawer. Now I can just grab bunches of forks from the fork sub-bin, and put them into the fork-compartment of the silverware drawer. And I can grab bunches of spoons from the spoon sub-bin, and drop them into the spoon-compartment of the drawer. Unloading the dishwasher is much more faster than it would be if I had to take individual forks out of random sub-bins.
Compare this to test-driven development. There may be a little more effort when writing code, because you are writing programmer-tests to drive writing that code. It's an "unnatural" process, like sorting silverware into sub-bins when loading the dishwasher. But the true benefit comes later in the project (possibly just minutes or hours later), when you can rely on those programmer-tests to make refactoring safer. And since you fixed bugs during TDD, you have much less work fixing bugs later when it comes time to ship your product.
I heard in an Agile Toolkit podcast of a speech by Tim Lister, that a "process" or "method" is something other than the natural way we do things. After learning and practice, it can feel natural, but it isn't the way an untrained person would do something. The speaker's example was the process of swimming. Untrained persons, thrown into deep water, either drown or dog-paddle. They try to keep their head above the water, and their feet end up deep under the water. Dog-paddling is not an efficient way to swim, but it is natural. Olympic swimmers, on the other hand, have learned an unnatural way of swimming that is very efficient. Their heads are in the water, not above it, sipping air now and then from the side, and their feet are near the surface. In the "freestyle" event, the competitors can swim in any way they like, but they all swim in very similar way because that's the most efficient way currently known. Software development and management methods like Extreme Programming or Scrum are also "unnatural" processes, but can be very efficient.
2006.Nov.09 ThuWhat is Agile Testing? The question came up on the Agile Testing mailing list, and among others, Lisa Crispin attempted to define it:
When I use the term 'agile testing' I'm talking about a set of good practices which has helped me and my teams deliver higher quality software, whether or not we were part of an agile team. We know that people, not tools or methodologies, are what makes projects succeed. Anything we can do to allow ourselves to do our best work fits my definition of 'agile'.
'Agile testing' practices come about by applying agile values and principles to testing. For example, communication is essential, and by collaborating with our customers up front to write test cases, we can improve communication. By driving coding with customer-facing tests, we're more likely to deliver what the customer wanted.
Some other examples: Automating our regression tests allows us time to do exploratory testing, so we strive for 100% regression test automation (I've never achieved anything close to this on a team who isn't automating unit tests, but we can still automate some testing). Exploratory testing is inherently agile so we work at getting better at that. Collaborating helps communication, so testers should get together with programmers and with customers from the start of a project, as well as pairing with each other.
Simple design is important in designing and automating tests. Refactoring applies to any type of coding including test scripts. Retrospectives are the single most important practice I can think of for any team, because you can't improve without identifying and focusing on what you need to improve. Dividing work into small chunks, having a stable build at all times (because we have continuous integration and collective code ownership), releasing business value frequently, all that allows us to do a better job of testing.
None of this is really new, and a lot of it I was doing years before I heard of XP, Scrum, agile, etc.
I haven't come up with a good elevator speech for agile testing so I have to resort to examples. I've heard people come up with some great one-liners but then I can't remember them. I think the Agile Manifesto works from a testing point of view as well as from a coding point of view.
Michael Bolton came up with a one-floor elevator speech:
Testers assist the developers and the rest of the project community in identifying what is desired, what we have, and the differences between them, with the four points of the Agile Manifesto as guiding principles.
Michael's slides (pdf) on User Acceptance Testing are also interesting. Check them out.
2006.Nov.01 Wed
A Leadership Warning by Dick Richards
Beware leadership that requires an enemy in order to hold power. Such leadership must create enemies. [...]
Beware leadership that requires fear in order to hold power. Such leadership must extinguish courage. [...]
It's short but meaningful. Check it out
2006.Oct.31 TueMock Objects used in Test-Driven-Development are "fake" objects that return specific values, and assert that calls are made with correct arguments and in the correct order. Avoid them. Mocks make your tests more fragile and more tightly-coupled. And at the same time, they reduce the integration-test aspects of TDD. They make your tests larger and more complicated and less readable.
Mocks reduce your tests robustness. If you change an object's API, a mock object's API may not change, making your tests out-of-date with respect to your non-test code. This depends on whether you are using a Mock-generating framework, or not, and how your mock object is implemented or how the Mock-generating framework is implemented.
In TDD, you normally write a test, write some code in a target class/method that passes the test. The third step is refactoring. That's when you can do "Extract Method"/"Extract Class" Refactorings to start creating other classes that cooperate with your target class (those extract classes may be "stub" classes at first, see below). Test-Driving with Mocks inverts that order: you create your target class and a mock class up-front, and plan on how they interact, instead of evolving that interaction in TDD's refactoring steps. You pre-judge the class design rather than evolve it.
Mocks are tool for decoupling, and I do sometimes use them. I limit my use of Mocks to those situations where the real object will not reliably do what I want. Examples: simulating errors in writing a file; simulating connections with a remote server; simulating errors from remote server. Often with libraries that were not designed with TDD in mind, such as the TWAIN libraries used for controlling scanners and implementing scanner-drivers.
Stubs are objects with hard-coded return values, but without the assertion-behavior that mocks have. Sometimes I do create stub objects during TDD, and later I focus on test-driving the stub-object class into being a "real" class. The existing tests (for code that is using those stub objects) help insure that I don't break anything that depends on that stub-becoming-real class. If a Mock generated from an interface was used instead of Stub, there would be a disconnect. My "real" class's return values (and implied behavior) could diverge from the mocked return values and nothing but code inspection or failing acceptance tests would tell me. I prefer to have failing tests tell me when behavior is changing. Remember change-detectors?
Mocks are tool, and a tool can be over-used or misused. I think for many people new to TDD, mocks are their new hammer, and every problem is treated like a nail, even when a saw or a screwdriver would be more appropriate for the situation.
2006.Oct.29 Sun
Commonly Heard, Uncommonly Thought
A commonly heard Myth: "A project with good developers will succeed no matter what their process."
So, no matter what their process, success is caused by "good developers" and failures are caused by "poor developers"? That isn't the case. A project with a bad process, will likely fail, no matter how good the developers are, unless the developers circumvent the process. The irony is that a "good developer" breaks the rules in this circumstance, while the "bad developer" follows the process rules. Too often developers are classified into "bad developers" and "good developers" because of systemic failures outside of their skills as programmers.
For example, if the domain is complex, and the process is "One month of analysis, followed by two months of design, followed by four months of programming, followed by three months of testing". (Dates fixed no matter how complete each of those activities is.) Well, that is only going to work if the team members actually do all of the activities (analysis/design/coding/testing) all the time. Testing during the design phase (perhaps via coding or by programmers "playing computer"). Thinking about coding, or actually coding, during the analysis phase. Revisiting analysis, design, and coding during the testing phase.
Very likely in such a 10-month "phased" project, they will find they are in trouble after the second month of "testing" (which often really means "bug-fixing"). At that time they will be forced to fix the flaws in their analysis, design and coding, with much grumbling and angst, and that "testing/bug-fixing" phase will end up lasting six months or longer instead of the planned three months. Sixteen months instead of 10 months.
Another myth is: "A project with bad developers will fail no matter how good their process is." (or "A good process won't make good developers out of bad ones.") This is also false in some situations, depending on how one defines a "bad developers". If it merely means "inexperienced", then depending on how well their process encourages detection and correction of mistakes, the simplicity of the domain, and how often and how well the developers get feedback from customer-proxies, domain experts, or end-users, the project could well succeed. If "bad" means "refuses to learn" and "refuses to cooperate", a good process will at least reveal those problems, but a bad process will let those problems stay concealed until disaster strikes.
This is why some poorly skilled developers are so against agile methods: their deficits in coding/testing skills or people-skills will be revealed by an agile method, and they would rather hide in a high-ceremony or chaotic process. Skill-deficits can usually be remedied by training, practice, and mentoring but only if people involved are motivated, and situation permits learning from mistakes.
In a chaotic "no process" situation, the advantage lies with the highly skilled/very-experienced developers who can write code with few defects, and who know enough to adopt a good informal process for their own work and for the work that's essential for the project's success. The other success factor I've seen in the "no process" situation is done by less-skilled developers: they work extra-long hours fixing mistakes that a good process would have prevented in the first place. I consider such overtime to be management failure.
"Success" can be very arbitrary, of course. In the history of software development, it's been fairly common for a horrible product to be declared a success and shipped. If that situation were uncommon, we wouldn't have license agreements that declares the software company is not liable for defects or even fitness for use.
More and more, software surrounds everything we see, touch, ride or fly in. My car has maybe a dozen CPUs running software for anti-lock brakes, digital dashboard, monitoring and controlling the engine, and so on. My "sonic" toothbrush maybe has a CPU as well. Clocks. iPods. Email. Phones. Chats and IM. Spell-check. Word-processing, image processing. Search engines. DNS. If your work is entirely digital, you're at the mercy of software all the time. Let's hope that the software we use is good enough; or we can do more than hope — we can learn good practices and good processes, seek feedback, and make it better.
2006.Oct.28 SatRichard Banks describes the benefits from having adopted Scrum in the last nine months. Check it out.
The biggest success factors were a willingness from the team to change and a structure with scrum that was focused on that change. The fact that an agile process is built around an understanding that it takes people to make something work and that we're not automatons is a great enabler for improvements. All I did was facilitate the change, lead a little bit by example to get things started and then get out of their way. If it wasn't for my great team then none of this would have happened and I'm indebted to them for the willingness and adaptability that they have shown over the last 12 months.
Compare to Richard's earlier attempt to introduce Scrum incrementally.
2006.Oct.23 MonIn a nutshell it was a disaster.
If you are planning on implementing Scrum you need to do it treat it as an all-or-nothing change. I talked with my CEO about organisational change and the mess that I made and he told me that if I wanted things to change I had to "go to war with the company. State what I wanted to do and take no prisoners in implementing the change." The only real benefit of the anarchic state things were now in was that any change would be an improvement and likely to be received well by the organisation.
Earlier this month, I became a licensed Certified ScrumMaster™. I should show up in the official rosters pretty soon; here is a screen-shot of my info in www.scrumalliance.org. (I need to correct some of that information.)
By the way, as this cartoon asserts, Scrum isn't a Silver Bullet. Neither is XP. However, they both do many of the things that Brooks recommends in his article "No Silver Bullet". (See previous posting).
So many people say "No Silver Bullet" mindlessly, without having read the article that made that phrase a cliché in the software profession. I'm speaking of the article by Frederick P. Brooks "No Silver Bullet: Essence and Accidents of Software Engineering," Computer, Vol. 20, No. 4 (April 1987). Here's a copy.
Lately, I've even seen the phrase "No Silver Bullet" used to argue against adopting agile methods. Here's what that article really says; you might find its emphasis on agile concepts surprising. Concepts like: "Working software over comprehensive documentation", "Customer collaboration over contract negotiation", and "Responding to change over following a plan". Quotes:
The familiar software project, at least as seen by the nontechnical manager, has something of this character [of the werewolf]; it is usually innocent and straightforward, but is capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet--something to make software costs drop as rapidly as computer hardware costs do.
But, as we look to the horizon of a decade hence, we see no silver bullet. [...]
He's referring to fact that hardware costs get cut regularly and why software costs does not have a similar perpetual future of cost reductions. But he also says: (Boldfacing is my emphasis)
Skepticism is not pessimism, however. Although we see no startling breakthroughs--and indeed, I believe such to be inconsistent with the nature of software--many encouraging innovations are under way. A disciplined, consistent effort to develop, propagate, and exploit these innovations should indeed yield an order-of-magnitude improvement. [...]
He goes on to describe the essential hard parts of software development (where the costs can't be compressed):
The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract in that such a conceptual construct is the same under many different representations. It is nonetheless highly precise and richly detailed.
I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems.
He then goes on to say why software specification, design, and testing can't be just once:
All successful software gets changed. Two processes are at work. First, as a software product is found to be useful, people try it in new cases at the edge of or beyond the original domain. The pressures for extended function come chiefly from users who like the basic function and invent new uses for it.
After some discourse in various areas, Brooks gets back to requirements: (Boldfacing is my emphasis)
Therefore, the most important function that the software builder performs for the client is the iterative extraction and refinement of the product requirements. [...] Complex software systems are, moreover, things that act, that move, that work. The dynamics of that action are hard to imagine. So in planning any software-design activity, it is necessary to allow for an extensive iteration between the client and the designer as part of the system definition.
I would go a step further and assert that it is really impossible for a client, even working with a software engineer, to specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product.
Sounds like he's advocating agile software development with frequent releases and close customer involvement. He mentions Rapid Application Development(RAD) and protyping. today we have experience in iteratively and incrementally producing software without throw-away-prototypes, and without some of the difficulties of RAD.
He also mentions the consequences of this style of development to morale: (Individuals and interactions over processes and tools.)
The morale effects are startling. Enthusiasm jumps when there is a running system, even a simple one. Efforts redouble when the first picture from a new graphics software system appears on the screen, even if it is only a rectangle. One always has, at every stage in the process, a working system. I find that teams can grow much more complex entities in four months than they can build.
Brooks says:
Incremental development--grow, don't build, software. I still remember the jolt I felt in 1958 when I first heard a friend talk about building a program, as opposed to writing one. In a flash he broadened my whole view of the software process.[...]
The building metaphor has outlived its usefulness. It is time to change again. If, as I believe, the conceptual structures we construct today are too complicated to be specified accurately in advance, and too complex to be built faultlessly, then we must take a radically different approach.
[...]
Some years ago Harlan Mills proposed that any software system should be grown by incremental development. That is, the system should first be made to run, even if it does nothing useful except call the proper set of dummy subprograms. Then, bit by bit, it should be fleshed out, with the subprograms in turn being developed--into actions or calls to empty stubs in the level below.
I have seen most dramatic results since I began urging this technique on the project builders in my Software Engineering Laboratory class. Nothing in the past decade has so radically changed my own practice, or its effectiveness.
He mentions the necessity of writing code top-down, which isn't entirely a necessity today. We can write programs in vertical slices, but not everything in that slice has to be "final" code. Since (as Brooks says in his article) coding isn't the hardest work of software development, we can code simple things in earlier iterations and grow those later into more complex things as needed, making time available for the customer to see some functionality early.
2006.Oct.19 Thu
Specification Documents as Tests
I was in a conversation with someone whose project is using Scrum. I was, if I remember correctly, trying to talk about the idea of how agile projects try to minimize the creation of artifacts not directly needed by the end-user or the developers.
She responded by talking about how essential the specification documents are for her team. I asked her "Who reads these documents?" and she said that the testers do. (NOTE: not primarily the programmers!) I then suggested to her that maybe in this agile development process, the testers should be writing these documents instead of reading them. And not writing them as static documents, but as executable tests in a form understandable by the Product Owner and programmers. Her face lit up with understanding.
I hope I helped. If she recognizes my attempt at recounting this conversion, I hope she will let me know if this helped her team in their adoption of agile practices.
2006.Oct.17 TueReview of a cool-looking Subversion App for MacOS X. I'm looking forward to using it. Maybe using it with code.google.com.
Lots of interesting stuff in Richard Thaler's Mental Accounting (pdf) paper:
[...] (1) The value function is defined over gains and losses relative to some reference point. The focus on changes, rather than wealth levels as in expected utility theory, reflects the piecemeal nature of mental accounting. Transactions are often evaluated one at a time, rather than in conjunction with everything else.
(2) Both the gain and loss functions display diminishing sensitivity. That is, the gain function is concave and the loss function is convex. This feature reflects the basic psychophysical principle (the Weber-Fechner law) that the difference between $10 and $20 seems bigger than the difference between $1000 and $1010, irrespective of the sign.
(3) Loss aversion. Losing $100 hurts more than gaining $100 yields pleasure: v(x)< -v(-x). The influence of loss aversion on mental accounting is enormous, as will become evident very quickly. [...]
[...] For example, a prix fixe dinner, especially an expensive multi-course meal, avoids the unsavory prospect of matching a very high price with the very small quantity of food offered in each course.* Along the same lines, many urban car owners would be financially better off selling their car and using a combination of taxis and car rentals. However, paying $10 to take a taxi to the supermarket or a movie is both salient and linked to the consumption act; it seems to raise the price of groceries and movies in a way that monthly car payments ( or even better, a paid-off car) do not.
More generally, consumers don't like the experience of 'having the meter running'. This contributes to what has been called the 'flat rate bias' in telecommunications. Most telephone customers elect a flat rate service even though paying by the call would cost them less. [...]
Lots of stuff to think about there. This "meter running" aversion is the real reason online micro-payment systems, or other web-apps that try to charge by each use, haven't really taken off yet. Shopping sites that aggregate your purchases, and then offer "free shipping" if you buy enough stuff, are taking advantage of these "mental accounting" biases. How many times have you bought a book on Amazon that you really didn't need, but which put you over the free shipping price.
2006.Oct.05 Thu
Code Smell: Long Parameter Plus + "Undocumented Language"
Wil Shipley describes the evil of Long Parameter List in some of Apple's APIS. But he is describing a variation on that code smell. It's kind of a combination of "Long Parameter List" with "Undocumented Language". I will bold-face that aspect of the problem in my quotes from that blog:
[...] There is usually one, giant, all-encompassing "setAttributes" function and one "getAttributes" function for setting and getting every value associated with a Carbon "object", which requires to you laboriously build up and then laboriously unparse huge, special parameter structures for even the simplest call. For example, see one of the newer (v6.4) QuickTime calls, for setting parameters on a FireWire video camera: VDIIDCSetFeatures(VideoDigitizerComponent ci, QTAtomContainer *container). That one call is used to set a HUGE number of parameters, which is "simple," except, oh, it takes eighty lines of code to prepare for that one call, and the parameters you can set are not actually documented in the documentation. [...]
[...] Carbon uses the same type, OSTypes [4-character constants] (eg, 'sit!'), extensively for everything: return codes, building parameter blocks (specifying what you want to do, how you want to do it), loading QuickTime components, etc., which means reading a function description doesn't actually give you enough information to know what to actually pass it. (Eg: "Pass it some combination of four-character constants. Good luck!") [...]
[...] (On a side note, sadly, the AppKit and Foundation team just announced they are moving AWAY from using enumerated types ... because they are worried that enumerated types could have unspecified sizes. But, seriously, Ali, this is totally broken -- enumerated types make reading and writing code MUCH easier, and they enforce sanity in switch() statements and if() comparisons. Please don't break this!) [...]
[...] For some reason some Carbon programmers think it's easier for them to add and remove APIs if they never commit to any particular functionality, so they use those giant anonymous set-get functions and then just have you build up a list of parameters yourself, with the idea being that it's a lot more flexible if they change the parameters out from under you than if they change the functions. [...]
[...] The fact that Apple had to write EIGHTY lines of example code to show how to set A SINGLE PARAMETER on SOME TYPES of cameras to its maximum value shows that THESE APIS ARE SERIOUSLY BROKEN. [...]
It goes on with examples from the KeyChain API. Check it out
2006.Oct.02 Mon
Test-Driven Database Development
Scott Ambler announces:
My article "Test Driven Database Development" appears in the current issue of TASSQuarterly magazine. The issue itself is available at http://www.tassq.org/quarterly/docs/tassq_magazine-0609.pdf. Just like you can take a TDD-based approach to developing application code, you can do the same thing with your database schema. I personally think that it's time that we start adopting quality practices for database development. If you work in an organization which considers data to be a corporate asset, and/or you're implementing functionality in your database via stored procedures, stored functions, or even OO code, then shouldn't you also have a regression test suite in place?
Quotes:
There are several advantages to TDD:
1. It promotes a significantly higher-level of unit testing [....]
2. It enables you to take small steps when [....] For example, assume you add some new functional code, compile, and test it. Chances are pretty good that your tests will be broken by defects that exist in the new code. It is much easier to find, and then fix, those defects if you've written two new lines of code than two thousand.[...]
3. It promotes detailed design. Bob Martin [5] says it well “The act of writing a unit test is more an act of design than of verification. It is also more an act of documentation than of verification. The act of writing a unit test closes a remarkable number of feedback loops, [...]"
A test-driven database development (TDDD) approach provides the benefits of TDD, plus a few others which are database related. First, it enables you to ensure the quality of your data. [...] Second, it enables you to validate the functionality implemented within the database (as stored procedures for example) that in the past you might have assumed “just worked”. Third, it enables data professionals to work in an evolutionary manner, just like application programmers. [...]
Part of test-driven-development is refactoring, and Scott gives a high level description of a database-refactoring in this article. There is more at www.agiledata.org. He also lists several database testing tools. Interesting magazine - it also has articles by James Bach, Rex Black, Michael Bolton, Duncan Card, Fiona Charles, Cem Kaner, Joe Larizza and Richard Bornet. Check it out.
2006.Oct.01 SunIf you are a super-programmer, incapable of error, do you need to do Test-Driven-Development (and Story-Test-Driven-Development (pdf))? Do you need programmer/unit tests and automated acceptance tests?
Maybe not for a small, solo project, but what about working in a team? What will happen if one of those other programmers on the team (maybe one who hasn't been hired yet) has to change something that your code depends on? Or even changes your code? Perhaps you've moved to another project, so you're not there to change your code perfectly.
Of course, you are not perfect. You will inject defects into your code sometimes. Your teammates will too. And then there are changing requirements, and changing interpretations of requirements, and operating system updates, and library updates... lots of opportunities to write imperfect code or introduce defects when modifying the code.
One way to find those imperfections is manual testing. But this is the 21st century - automate that testing! The first time you run well-written automated tests on "legacy code" (defined as code without automated tests, according to Michael Feathers), you are very likely to find bugs. Often just writing such a test helps me find code-defects, and then I run the test to confirm it. And they are so cheap to run (if written right) that using them to find less-frequent "regression" bugs will pay off.
Story-Test-Driven-Development starts off with the Customer (Product Manager/Domain Expert) defining the expected behavior of a feature using automated acceptance tests. Defining these tests clarify the requirements, which even a perfect programmer can benefit from. Without fully understanding the requirements, any programmer can write the wrong code: a "bug" even though code itself is flawless.
When an expert pair of programmers, familiar with TDD, decided to skip Test-Driven Development, they paid a price. Quoting James Shore:
What we should have done at this point is archive our spike and start over using proper test-driven development. But we didn't. Instead, we kept going, trying more and more ideas, [...] until we didn't have a spike any more. We had a nascent application. One without tests. In other words, we already had a fairly sizable amount of technical debt and the application was only a week or two old.
[...] It's now a month later and progress has slowed to a crawl. In the first month, we were adding significant new features every week, sometimes every day. What happened?
In two words: technical debt. To make progress so quickly, we cut a lot of corners. We didn't implement any tests. We only programmed for the best-case networking scenarios. We let bugs creep in.
[...] we took on technical debt and now we're paying the cost [...]
THAT was a six week project. They planned on paying off that technical debt after initial demo and alpha pre-release. (Which has happened. Check out CardMeeting.) Imagine how bug-ridden and fragile the source code of a 10-year-old product with few or no automated tests can be. If you work for a big company with "version 9.x" product, it's very likely you already know. It doesn't have to be that way. It doesn't have to stay that way.
2006.Sep.25 MonMy team has finished Mac QuickBooks 2007 and it is now available for pre-order on Intuit's website. (Actual availability is October 2.)
I think this is the best version we've done so far. Check out the Demos!
This is the second year we've used a Scrum/custom-Agile process. This year that process helped us make the development, and testing processes even more smooth and high-quality. No major surprises, no major fire-fighting.
2006.Sep.22 FriIn large applications, just to be able find classes and functions, and to be able to comprehend the structure "in the large" (to see the forest and the trees), you want "layers". Or "modules". Or "packages". Some division of responsibilities larger than the individual classes. In the image below, I show a hypothetical ideal application with eight modules or layers, compared to a near-worst-case legacy application.
The order of the layers/modules in this diagram is arbitrary and not meant to imply the amount of connections between layers, nor the directions of those connection. (But keeping the connections between modules going one-way, even if that requires adding additional "interface" modules, is a Good Thing.)
When you are doing iterative/incremental development, as recommended by all agile methods, how do you keep to the ideal layering and avoid that legacy code scattering of responsibilities? By paying careful attention to the Single-Responsibility Principle and other principles of good design, and by cleaning up any the code smells that you notice which indicate violations of SRP and other good design principles. Decide on which layer a class belongs to, each time you create a class. (Create new layers as needed.) Each feature that you implement will likely involve several layers, so pay attention to where those layers are.
If you don't pay attention during agile development, your application structure could look like a Tetris game gone badly. Refactor every day to keep the structure clean and well-organized.
2006.Sep.18 Mon
Barriers to Self-Organized Problem Solving
Jeffrey Phillips has identified three reasons why self-organized problem solving doesn't happen at work. Quote:
Why is it when we notice a problem, especially a hole in a process or an inefficent process, that we don't get a group together to immediately solve the problem? I guess there are three relevant reasons: 1) ownership 2) importance 3) politics.
Ownership... someone else appears to own the problem, and if you attempt to take it on you may be working outside your area of focus... Even if no one owns the problem, it is very likely that you'll receive negative feedback for working on a problem rather than encouragement, since working on the problem is probably not "your job"....
Importance... Often, everyone recognizes a problem exists yet it is small enough or happens so infrequently that it seems that the problem is not important and not worth solving. Everyone assumes someone else will solve it, or the team that "owns" the problem does not recognize how important that problem is to another function or work team....
Politics... a manager in a different group may feel angry that you've identified what you think is a problem in his area....
This is why Agile and Lean methods have the workers "owning" the process they use - so they can improve it at will. I believe Lean (as practiced by Toyota) also has the emphasis on improving any and every part of the process, no matter how "unimportant" it might be.
Patrick Lencioni's Overcoming the Five Dysfunctions of a Team and his subsequent Field Guide identifies how to "cure" the political problems by Building Trust, Mastering Conflict, Achieving Commitment, Embracing Accountability, and Focusing on Results. This is easier said than done, though I've seen some of this team-building stuff really work in Jerry Weinberg's workshops.
Via Ed Gibbs, Beyond Test Driven Development: Behaviour Driven Development by Dave Astels, author of "Test-driven Development: A Practical Guide". And, Scrum by Ken Schwaber, a founder of the Agile Alliance and the Scrum Alliance.
Check it out
2006.Sep.08 FriA person in a dream I had this morning said that whenever you have code that has loops operating on lists, your code is not "object-oriented". Now I don't necessarily agree or disagree, but this was the example I thought of (in my dream):
app tells book 'change margins'
In some implementations, the cascade of method calls to implement that might be:
book loops through its chapters, tells each chapter to 'change margins'. chapter loops through its pages, tells each page to 'change margins'.
But it could be different. What if the chapters share margin objects from the book, and the pages share margin objects from the chapters? If the data is shared, and not duplicated, then we don't need to loop everywhere changing it. (Though we probably still need to loop somewhere to inform all these objects that the data they depend on changed, and they need to re-layout themselves.)
app tells book to 'change margins' book tells margins to 'change'. margins tells all of its dependents (pages) to update themselves. (implied loop) each page tells all of its dependents (chapters) to update themselves. (implied loop) each chapter tells all of its dependents (the book) to update themselves.
[but what if pages are dependents of chapters? infinite loop? hmm.]
This dream was partly inspired by an article in MacTech magazine that I read, about parallelizing loops in Fortran-90 and C/C++ using some options available in Intel's compilers for MacOS X. With a couple of C pragmas (or the Fortran equivalent), the compiler takes care of splitting loop processing over multiple CPUs -- doing that yourself using Posix threads would take nearly a page of code, or somewhat smaller if you have threading classes to use.
(It also struck me that Fortran-90 is quite a different language than the Fortran I learned in the 1980's.)
Could you parallelize the looping message cascades of this book / margins example? Maybe. The loops that re-lays out each page and chapter needs one or two pieces of information from the previous page: the previous page number (so we can compute current page number = previous page number + 1). But in fact, there may be more pages than before, if the margins are made smaller. Or fewer pages if the margins are made larger. Maybe "page" is a just a flyweight object and we really only have "book" and "chapter" (or "section") where formatting is concerned. But a chapter still needs to know the last page number of the previous chapter, to get its own page numbering right.
2006.Aug.30 WedAndy Matuschak has documented the new features of Objective-C by examining the code that Apple checked into their gcc branch in the gnu.org repositories. Check it out. (If you find his gray text on black background annoying, read the RSS feed instead.)
2006.Aug.23 WedCheck out Jason Sankey's article on Build Enlightenment (via a blog byDaniel Ostermeier and Jason Sankey)
This article describes the properties of an effective build system, starting from the most basic requirements and working towards more advanced features. Follow the article step-by-step to drag your build system out of the darkness and attain build enlightenment!2006.Aug.12 Sat
Check out Dave Astel's paper (pdf) on TDD/BDD. Quotes:
[...] There are a score of books available on TDD, mine even won a Jolt award. So it seems that everything is rosy? Everyone who's doing TDD is fully understanding it and getting the full benefit, right?
Fat Chance!
Too few people I talk to really understand what it's really about. That means that many people who practice TDD are not getting the full benefit from it. What's wrong?
The focus on testing
Well... one thing is that people think it's about testing. That's just not the case.
Sure, there are similarities, and you end up with a nice low level regression suite... but those are coincidental or happy side effects. So why have things come to this unhappy state of affairs? Why do so many not get it?[...]
So with people thinking about testing, it's easy to come up with all sorts of negative reactions and reasons not to do it... especially when time gets short and the pressure's on.
By the way, I HATE saying TDD is "not about the testing". I have to say it now and then because of people not realizing it's about designing, rather than testing. The fact that the word "test" is in the name just helps confuse the issue. My preference for naming this design technique, is to call it "Behavior Driven Design". Or "Behavior-Spec Driven Design". With BSDD, I could say BSDD is about the behavior-specs, but mostly about driving design.
Alistair Cockburn's "Executable Example Driven Design" (XXDD) is OK, but a bit long-winded. And I don't want to have say "Dos-equis Driven Design is not about beer." :-)
The problem is mental frameworks. People reject ideas that don't fit into their mental framework. You say "test" and the listener's mental framework comes up with these and other assumptions:
- Testing can be done manually.
- I can do implementing, someone else can do testing.
- Testing looks for bugs.
- Tests need something to test.
- Testing is hard.
- We can skip testing if we are good enough programmers, or time is short.
None of those assumptions directly applies to TDD. TDD can't be separated into tests written by one person and code written by another because the TDD cycle of test-code-refactor is a design process that one person (or a pair) goes through in cycles, where each cycle takes only minutes. TDD/BDD helps me find bugs, but that's not the purpose of TDD. TDD isn't TDD is you do the "test" part manually.
If you say "Test-Driven Development is not about the tests," that usually doesn't provide the zen-like kick in the brain-pan that the speaker intends. Instead, the phrase gets rejected because it doesn't fit into the listener's mental framework for "tests". The listener probably thinks you're nuts. He or she might just flip the "bozo bit" and stop listening to you.
If instead you say "Behavior Spec", the listener's mental framework will have fewer contradictory assumptions. For example:
- Specs are written before coding.
- Specs can provide examples.
- Specs tell you what you need to implement.
- Someone else can write specs and I can write the code.
Not all of those assumptions are true for BDD, but there are fewer contradictions to this idea of BBD that you're trying to introduce into someone's mental framework. Behavior Specs are written before the code exists (though only minutes before). Behavior Specs are executable examples. Behavior Specs tell you, more or less, what you need to implement next. There is still the contradictory idea that someone can write the Spec and you can write the code, but that isn't as difficult to overcome as all the other assumptions that come up with the word "test".
2006.Aug.09 WedJohanna Rothman, author of Hiring The Best Knowledge Workers, Techies & Nerds: The Secrets & Science Of Hiring Technical People, writes about a few examples where puzzles or riddles in interviews were not providing help in appropriate hiring.
At another client, the interviewing team liked word riddles. [...] they missed bunch of people who were hired by other managers in the company [...] Several of the new hires misunderstood the logic puzzle because English wasn't their first language, and they didn't hear the puzzle correctly. Two of the new hires decided that if that's how the team evaluated potential candidates, they didn't want to work with that team. And one of the new hires had attempted to explain why there was more than one solution to the riddle, but the interviewer couldn't hear that.
Our choices of questions that are not directly related to the job (and no matter how you slice it, puzzles and riddles are only indirectly related to the job) reflect our culture. It's quite clear that the puzzles and riddles interviewers choose reflect their individual culture. And without meaning to, that culture primarily selects for people just like themselves.
Previously she wrote on why puzzles discriminate:
Using puzzles and riddles discriminate against anyone who isn't a (middle-upper class) white American suburban male.
Girls, for example, do not have access or the alone time to spend doing books of puzzles and riddles. It is socially unacceptable for even the geekiest girl you know to do this. A girl who spends time pursuing puzzles and riddles for her own pleasure runs the colossal risk of being ostracized from all the other girls. Boys tend to discover puzzles and riddles during middle school and continue to pursue them through high school. Middle and high school for girls is much more about social ability and social connections.
Pete Abilla, blogged about picking high benefit / low effort / "vital few" customer-focused features at Amazon.
He also advises to not get into a feature competition with competitors, since that takes your eyes off the customer and leads your product or service into featuritis. Too bad for users who want a good user-experience instead of featuritis that Microsoft doesn't take that advice.
Last year, Apple's Developer Conference had tongue-in-cheek posters advising Microsoft to "start your copiers". This year's keynote address, showed that in not-yet-released Vista, Microsoft did just that. Check out the keynote video to see side-by-side comparisons of Apple's email UI and Vista's email UI, and Apple's Calendar UI and Vista's Calendar UI and remember that the Apple UI came first.
Of course, Apple has been shipping its email and calander apps for a quite a while now, but Vista is still in beta... so if you want that UI today, you better get a Mac.
This WWDC year's posters refer to Microsoft/Vista as being a "copycat". (FYI: Apple has been using names of big cat species for its OS releases - the previous release was Tiger, the next one will be Leopard.)
2006.Aug.08 TueUncommon Education Necessary for Scrum?
Ron Jeffries versus the idea of "Scrum is common sense"... "Einstein said: 'Example isn't another way to teach: it is the only way to teach.'" Check it out. (Thanks to adapdev for the link.)
2006.Jul.31 MonImagine that you are developing version N+1 of a software application that has a document format that have to be upgraded to handle the new data that version N did not have. Pretend it's a relational database; you will have to add tables and columns, etc.
One way to handle this is to imagine all the possible transformations you will need to make to the database, and implement them. Here's a list partially taken from Refactoring Databases: Evolutionary Design by Scott W. Ambler and Pramod J. Sadalage.
- Add Column
- Add Table
- Add View
- Drop Column
- Drop Table
- Drop View
- Change Column Type (converting values)
- Change Column Values
- Change Column Format
- Introduce Calculated Column
- Introduce Surrogate Key
- Merge Columns
- Merge Tables
- Move Column
- Rename Column
- Rename Table
- Rename View
- Replace Column
- Replace One-to-Many With Associative Table
- Replace Surrogate Key with Natural Key
- Split Column
- Split Table
Implementing every feature in that list (and the other features I didn't mention) will take a while. It gets you a nice, complete file-upgrade library. But we're not in the business of developing libraries, we're in the application business. If you implement every feature, many of them will probably not be used by your application. Waste.
The other way to handle this is to proceed with the development of version N+1 of the application. Whenever you need to do one of the above transformations, you implement it, but only what you need. You get the application delivered sooner, because you're concentrating on customer value. You also have a file-upgrade library that does just what you need.
One of the teams that I worked did precisely this, and choose the incremental approach. It turned out we only needed a few of these features in our file-upgrade code. We did brain-storm a list of all the transformations might need, but we only implemented the ones we needed when the need arose. No waste.
2006.Jul.29 Sat"My greatest strength as a consultant is to be ignorant and ask a few questions" - Peter Drucker
2006.Jul.26 WedOn 2006 Jul 24, at 8:29 PM, Maruis Marais emailed to me:
Hi Keith,
Your posts around BDD is interesting. I'm in the .NET space and have followed the BDD movement since Dave Astels talked about BDD at XP2004 (I think).
I've developed a C# version of a BDD framework called NSpecify and thought you might find it interesting.
Here is a quick code example:
using System; using NSpecify.Framework; namespace SampleBehaviour { [ Functionality() ] public class Calculation { int one = 1; int two = 2; [ Specification() ] public void OnePlusOneMustEqualTwo() { Specify.That( one + one ).Must.Equal( two ); Specify.That( two ).Must.Be.GreaterThan( one ); Specify.That( two ).Must.Not.Equal( one ); } [ Specification(), ExpectedException(typeof(DivideByZeroException) ) ] public void OneDevidedByZeroMustThrowException() { int zero = 0; Specify.That( one / zero ).Must.Equal( one ); Specify.Failure( "The expected exception was not thrown" ); } } }I have a NUnit integration working and at the moment I'm working on creating a VS2005 test tip to integrate NSpecify into VSTS Testing Framework.
DavidChelimsky writing on Behaviour Spec Organization. Quotes:
2006.Jul.21 FriOne of the things that really interests me about the Behaviour Driven Development discussion is the effect it has on how you organize your specs. [...]
The reason that clarity is important is that specs serve as documentation for future development. If you want to understand how to use a class, look at the tests. Right?
# Disclaimer - this is also an example, not a recommendation # -- but closer to a recommendation than the example above! A stack - should add pushed item to the top of the stack - should return the top element on peek - should not remove the top element on peek - should return the top element on pop - should remove the top element on pop An empty stack - should be empty - should complain on peek - should complain on pop An almost empty stack (with one element) - should not be empty - should not be empty after receiving peek - should be empty after receiving pop [...][...] The generic "A stack" context describes the normal case for how a stack behaves, while the other contexts deal specifically with how a stack at or near a boundary behaves differently from the normal case. This is a far more clear specification [...]
I am now convinced that the word "test" in Test-Driven Development is a barrier to understanding that TDD is a design process. Next time I give a presentation on that subject, I will use "Behavior Driven Design" and "Behavior Specs" rather than "Test-Driven Development" and "Programmer Tests".
I have slides on TDD and BDD for your perusal. Compare them and see what you think. (Better, show the BDD slides to people not familiar with TDD/BDD and see what they think.)
Under Creative Commons copyrights, you may use these slides, unmodified, for non-commercial purposes.
2006.Jul.20 Thu12 Benefits of Test Driven Development
J. Timothy King blogs about 12 benefits of TDD. Check it out.
When I was programming in Pascal a long time ago, we had Functions and Procedures. Functions returned a value, Procedures did not. But there was more to it than that. Procedures had side-effects, but functions generally did not. This idea is called “Command-Query Separation” and it is useful today when writing object methods. Quoting from the Wikipedia:
Command-query separation (CQS) is a principle of object-oriented computer programming. It was devised by Bertrand Meyer a part of his pioneering work on the Eiffel programming language.
It states that every method should either be a command that performs an action, or a query that returns data to the caller, but not both. More formally, methods should return a value only if they are referentially transparent and hence possess no side effects.
On Behaviour-Driven Development
Two Quotes from http://behaviour-driven.org/GettingTheWordsRight. (It's a wiki, this page seems to be authored by Dan North and Dave Farley.)
2006.Jul.08 Sat"Behaviour-Driven Development grew out of a thought experiment based on Neuro-Linguistic Programming techniques. The idea is that the words you use influence the way you think about something."
"Concentrating on finding the right words led us to think about the process differently, for example, the fact that the words we use in BDD are very much focussed on the behaviour of the system led us to better understand the very close relationship between the stories we use to specify behaviour and the specifications we implement in BDD in place of tests. It also helped us gain further insight into some of the failure modes we have experienced in TDD projects."
Responding to Bruce Eckel's blog post Programming as Typing, where Bruce tries to get across the idea that programming is more about the programmers, rather than the act of hitting keys on a computer's keyboard. And that reasoning about programming practices is often based on fundamental, unproven (or even wrong) assumptions. Two quotes from Bruce: (emphasis is mine)
It may not even be possible to prove things logically when it comes to programming. So many of the conclusions that we draw this way appear to be wrong. This is what I like about the book "Peopleware," [by Tom Demarco or Larry Constantine?] and also "Software Conflict 2.0" [by Robert Glass] that I'm now reading. These books point out places where we operate based on what seems perfectly logical, and yet is wrong (one of my favorite studies in "Peopleware" shows that, of all forms of estimation, the most productive approach is when no estimate at all is made).
I think the essence of what the agilists are doing is a perfect analogy to the discovery of the scientific method. Instead of making stuff up -- and if you look back at all the "solutions" we've invented to solve software complexity problems, that's primarily what they are -- you do an experiment and see what happens. And if the experiment denies the arguments you've used in the past, you can't discard the results of the experiment. You have to change something about your argument.
Isaac Gouy mentioned two papers that provide statistical support for the effectiveness of some of the common practices of agile methods.
Quoting the blurb here about a study of 29 projects at 17 companies: (emphasis is mine)
Recent research from Harvard Business School professor Alan MacCormack and colleagues proves a theory about software development that has been gaining adherents for some time: The best process is an evolutionary one. Focusing on the area of Internet-software development, the researchers uncovered four practices that lead to success: an early release of the evolving product design to customers, daily incorporation of new software code and rapid feedback on design changes, a team with broad-based experience of shipping multiple projects, and major investments in the design of the product architecture.And this paper (pdf) analyzing 29 projects at HP. Quotes: (emphasis is mine)
In the final model predicting defect rate, three development process measures and one control variable (for systems software projects) were significant. Together, these measures explain over 74 percent of the variation in defect rate. The significant development process measures are the use of
- An early prototype
- Design reviews
- Integration or regression testing at check-in
The results illustrate that different software development practices are often associated with different dimensions of performance. For example, the use of integration or regression tests as code is checked in appears in our final model predicting defect rate but not in the model predicting productivity. Conversely, the use of daily builds appears in our final model predicting productivity but not in the model predicting defect rate.
Some practices that are correlated with performance when considered in isolation do not appear as significant predictors in multivariate models. For example, we initially found that having a less complete functional specification when coding begins is associated with lower productivity, lending support to the waterfall model of development. When we accounted for the variance explained by releasing early prototypes and using daily builds, however, this relationship disappeared. (A similar pattern exists for the relationship between a more complete detailed design specification and a lower defect rate.) In a sense, these other practices appear to “make up” for the potential trade-offs arising from an incomplete specification. This suggests that development models should be considered as coherent systems of practices, some of which are required to overcome the potential trade-offs arising from the use (or absence) of others. To the degree that such a process relies on a coherent system of practices, a piecemeal approach is likely to lead to disappointment.
One thing about statistical studies is that they do end up treating people as interchangeable components. Only one of the two studies linked to above mentions the possibility of experience of the developers as a contributer to project success. This is why Alistair Cockburn wrote Characterizing people as non-linear, first-order components in software development. Did these studies measure "good citizenship", "taking initiative", or the effects of dominant personalities?
2006.Jul.06 ThuOn Pair Programming vs Code Reviews vs Unit Testing
The problem with traditional code reviews is that they come too late. The best time to get feedback is as soon as you make a mistake, not two hours later (or four hours, or the next day). [...] backtracking is expensive.
Code review while pairing causes no interrupts because the reviewer is already devoted to the task. You get much more detailed review for the same reason. Nobody feels bad about fixing nitpicky stylistic issues right away [...]. Teaching and learning are more efficient; [...]
On a project I was on (medical software), where we did formal code reviews, we reviewed both the unit tests and the code. Without the reviews, the unit tests would have been mostly ineffective because the programmers just didn't think about what could go wrong with the code, and write the tests for that. (We ended up requiring a LOT of post-review changes to improve the unit tests.)
I can imagine that many projects with unit tests (written after the code, rather than test-first) but with no code review nor pair programming have poor test coverage. Adding code reviews to such a project would likely turn up many bugs that the unit tests didn't catch. This doesn't mean that unit tests per se are not effective, it just means that many programmers need to improve their unit-testing skills. (And reviews or pair-programming are two ways to improve those skills.)
Test-Driven Development (TDD) forces developers to develop their unit-testing skills, and combining TDD with pair programming or reviews will make the unit tests even more effective, as the team members offer feedback and ideas to each other.
Pair programming (PP) while doing test-driven development is the most effective combination of collaborative design, reviews, and unit testing that I've experienced. One variant of TDD+PP is "ping-pong programming" described by Dave Hoover here.
2006.Jun.28 WedWhy do companies that hire only "the best" employees, and want to provide "market-leading" products or services (again, their goal is to be the "best") choose to go with average technology in implementing these products and services? And then defend that technology choice by saying that they want a bigger pool of developers from which to hire -- more people that already know that technology. I guess it's that "herd" instinct. You don't get fired for buying from IBM (or Sun or Microsoft).
Some frameworks are 10X as productive. Seaside is one. RIFE and JMatter are two on the Java side that buck longstanding conventions to marvelous effect. If you don't believe that frameworks can be multipliers, then you haven't been paying attention.
Check it out. And check out WebObjects, too.
Old media versus New Media: candorville. Check it out.
2006.Jun.22 ThuUsually, if you can't make it faster, you keep the UI responsive by putting the work in the background. In a Cocoa application, sometimes you don't even need a separate thread. In one application that I was working on, I noticed that a particular function call was taking a few seconds to complete, and because I was calling it when I was handling the processing of a string received from the UI, the last character wasn't being shown in the UI. You type "abcd", see "abc" on the screen, and then two seconds later see "d". So I deferred the call using an NSObject method:
[self performSelectorOnMainThread: @selector(myLengthyMethod) withObject: nil waitUntilDone: NO ];This allows the UI to show the last character, and for me to start spinning the "waiting" indicator, without going to the trouble of creating another thread. This was a modal situation - the user can't proceed until the function call returns. In a non-modal situation, you might want to use a secondary thread, perhaps with a call like this:
[NSThread detachNewThreadSelector: @selector(myLengthyMethod) toTarget: self withObject: nil ];2006.Jun.18 SunGoogle Videos on Software Testing
James Bach is one of the good guys. Check out his blog and this google video where he talks about becoming a testing expert.
His brother Jonathan Bach talked on exploratory testing: video here.
English Style Assertions in ObjC
Tom White and Robert Chatley have been using org.jmock.MockObjectTestCase for sentence style assertions in java unit tests. Examples:
assertThat(a, eq("3")); // instead of assertEquals("3", a); assertThat(list, includes("peach", "pear", "plum")); assertThat(a, eq(true), otherwise("a should be true"));Objective-C and Smalltalk syntax allows a more english-like syntax with fewer contortions, except, of course, Smalltalk would require an explicit reference to some object to send the assertion messages to. Objective-C could get around of that with a macro or two. Objective-C syntax examples:
[ Assert that: anNSString equals: @"3" ]; // @"3" is an NSString literal. [ Assert that: listOrSet includes: @"peach", @"pear", @"plum", nil ]; [ Assert that: aBool equalsBool: true otherwise: @"aBool should be true" ];Since Objective-C doesn't have overloading, and the literal 'true' is not an object, the third example above has to use a method name to let the compiler know the parameters are supposed to boolean. (UNTESTED code follows):
#define Assert self assertLine: __LINE__ file: __FILE__ // thus [ Assert that: a equals: @"3" ]; // gets expanded to: // [ self assertLine: 120 file: "somefile.mm" that: a equals: @"3" ] // We use the __LINE__ and __FILE__ preprocessor symbols to identify the // source-code file and line of code that failed the assertion, because we // can't easily print nice stack-dumps like Java does. // header file #import <Foundation/Foundation.h> @interface TestBase : NSObject // or whatever... { // etc. } -(void) assertLine: (int) lineNum file: (const char*) fileName that: (id) firstArg equals: (id) secondArg; -(void) assertLine: (int) lineNum file: (const char*) fileName that: (id) listArg includes: (id) firstObj, ...; -(void) assertLine: (int) lineNum file: (const char*) fileName that: (bool) firstArg equalsBool: (bool) secondArg otherwise: (NSString*) aString; -(void) assertionFailure: (NSString*) aString; // etc. @end // implementation file #import "TestBase.h" #include@implementation TestBase -(void) assertLine: (int) lineNum file: (const char*) fileName that: (id) firstArg equals: (id) secondArg { if ( [ firstArg respondsToSelector: @selector(isEqual:) ] ) { if ( not [ firstArg equals: secondArg ] ) [ self assertionFailure: [ NSString stringWithFormat: @"assertion failed: %@ does not equal %@ at line %d; file %s", firstArg, secondArg, lineNum, fileName ] ]; } else if ( [ firstArg respondsToSelector: @selector(isEqualTo:) ] ) { if ( not [ firstArg isEqualTo: secondArg ] ) [ self assertionFailure: [ NSString stringWithFormat: @"assertion failed %@ does not equal %@ at line %d; file %s", firstArg, secondArg, lineNum, fileName ] ]; } else if ( [ firstArg respondsToSelector: @selector(compare:) ] ) { if ( NSOrderedSame != [ firstArg compare: secondArg ] ) [ self assertionFailure: [ NSString stringWithFormat: @"assertion failed %@ does not equal %@ at line %d; file %s", firstArg, secondArg, lineNum, fileName ] ]; } else // ... raise failure to find a known comparison method... } -(void) assertLine: (int) lineNum file: (const char*) fileName that: (id) listArg includes: (id) firstObj, ...; { if ( [ firstArg respondsToSelector: @selector(containsObject:) ) { va_list arglist; bool doesInclude = false; va_start(arglist, firstObj); id arg = va_arg(arglist, id); while ( arg != nil ) { if ( [ listArg containsObject: arg ] ) { doesInclude = true; break; } } va_end(arglist); if ( not doesInclude ) [ self assertionFailure: [ NSString stringWithFormat: @"assertion failed: %@ does not contain any of the expected" " values at line %d; file %s", listArg, lineNum, fileName ] ]; } // maybe try some other selectors like 'containsItem:' // ... etc. } -(void) assertLine: (int) lineNum file: (const char*) fileName that: (bool) firstArg equalsBool: (bool) secondArg otherwise: (NSString*) aString { if ( firstArg != secondArg ) [ self assertionFailure: [ NSString stringWithFormat: @"assertion failed: %@; boolean %d does not equal %d at" " line %d; file %s", aString (int) firstArg, (int) secondArg, lineNum, fileName ] ]; } -(void) assertionFailure: (NSString*) aString { // etc... } @end Writing your own test-framework, or extensions to a test-framework, is not hard. Everybody should do it at least once just for the practice.
2006.Jun.14 WedRoot Failure of Genesis Parachute
Universe Today summarizes the report on the root-causes of the failure of the Genesis spacecraft's parachute:
2006.Jun.10 SatIn the end they concluded that the G-switch sensors had accidentally been installed upside down in the spacecraft, preventing them from detecting the deceleration and starting the parachute sequence.. Furthermore, this error wasn’t caught by several oversight processes.
A centrifuge test was originally planned to actually test the G-switches, but it was canceled. Another test made sure the G-switches were working, but it didn’t check their orientation.
The report states that engineers were aware of the need to test the switch orientation, but decided to test them as part of a later phase.
The only documentation indicating that Genesis Project Management or Systems Engineering had been informed of a centrifuge test deletion was a single bullet presented at two management reviews that read, “SRC AU 3-g test approach validated; moved to unit test; separate test not required.”
There's a story about the project where Kent Beck began to formalize Extreme Programming, where the team was complaining that there wasn't enough time to do everything they had planned, and Kent Beck turned that around to "there's too much to do". There's only so much time, we all get only 86,400 seconds per day to work, eat, sleep, etc. We can't increase the amount of time we have per day. But with the idea that there's "too much to do", we can prioritize, defer, reduce scope, simplify, and so on. And thus, XP time-boxing and team velocity.
My "too much to do" involves books to read, writing I want to do, and a possible career change from being "just a programmer" towards teaching and mentoring and writing. With my full-time day job, it's hard for me to prioritize time for all those things. I've got a bunch of books half-read and unread. Other than blogging, I haven't done much writing-down-of-words so far. I like to relax by watching a movie or TV show or reading fiction or sometimes non-work-related non-fiction. I read blogs, mailing lists, and news on the internet instead of reading a daily newspaper.
I've trimmed my mailing lists, trimmed the blog-list, trimmed the number of TV shows I watch -- and we're in summer re-runs, so there's almost no TV shows to watch now. I suppose I could trim some more. If I "trim" my current day-job (the project ships in September, so I would wait until then), I could live off savings for a while and try to do the writing and teaching I've been thinking of. However, I like collaboration, so working alone isn't terribly attractive (If you want to collaborate with a co-teacher/co-author and you work in the SF Bay area, maybe we should get together?).
A few of the books in my to-read, to-reread, currently-reading, or partially-read pile:
- The Ruby Way, by Hal Fulton
- Agile Estimating and Planning, by Mike Cohn
- Refactoring Workbook, by William Wake
- Agile Web Development with Rails, by Thomas and Hansson
- Foundations of Ajax, by Asleson and Schutta
- The Five Dysfunctions of a Team, by Patrick Lencioni
- Designing from Both Sides of the Screen: How Designers and Engineers Can Collaborate to Build Cooperative Technology, by Isaacs and Walendowski
- Danny Goodman's AppleScript Handbook, MacOS X Edition
- Team of Rivals : The Political Genius of Abraham Lincoln, by Goodwin
- The Story of Human Language by John McWhorter
- Refactoring Databases : Evolutionary Database Design by Ambler and Sadalage
- And a some more books on MySQL, SQLite, and other databases, agile software development and project management, and other topics, that are sitting too far away for me to read the titles from my chair.
Perhaps it's time for me to time-box, estimate, and prioritize my to-do list.
2006.Jun.09 FriIn the Agile Testing mailing list, Bruce Rennie wrote: "When we return to code to change it there is a certain price we pay, which Robert Grady called 'knowledge recovery'. So, any new feature or bug fix we make has two components: the time it takes to recover knowledge about the code simply to get ourselves into a position to make changes, plus the time it takes to make the change itself. Anything we can do to reduce knowledge recovery time is a clear win. It seems to me that refactoring can be of great value here."
2006.Jun.04 SunNear-Future Science and Science Fiction
Charlie Stross's website linked to this article by Gregory Mone in Popular Science. I've lately been reading everything that Stross has published. He's a very good, up-and-coming, writer of science fiction.
A few quotes from the article:
[...] the Singularity. [...] The idea was conceived by Vernor Vinge, a computer scientist and science-fiction writer who’s now a professor emeritus at San Diego State University. We’re living through a period of unprecedented technological and scientific advances, Vinge says, and sometime soon the convergence of fields such as artificial intelligence and biotechnology will push humanity past a tipping point, ushering in a period of wrenching change. After that moment—the Singularity—the world will be as different from today’s world as this one is from the Stone Age.
[...] Vinge’s vision of the Singularity springs from his own field, computer science, but change is afoot throughout science and technology. Cosmology is undergoing fundamental revisions, genetics is giving researchers the tools to rejigger the building blocks of life, and nanotechnology has begun creeping from fantasy into reality. “Several lines of progress [are] converging,” says physicist Stanley Schmidt, editor of Analog magazine. “You can’t lock in on one field in isolation because you’ll miss how other fields affect it.”
A new kind of future requires a new breed of guide—someone like Stross, whose first novel, Singularity Sky, was recently nominated for a prestigious Hugo Award, or his frequent collaborator Cory Doctorow, who in 2000 won the Campbell Award for best new science-fiction writer. Both are former computer programmers. They are computer geeks and gadget freaks. They follow engineering and materials science and biotech, not to mention politics and economics. And they have latched on to the Singularity as the idea that symbolizes our era’s rush of new discoveries. Whether their stories will usher in another golden age or inspire a new generation of dreamers remains to be seen, but their focus is dead-on. “Right now is an extremely exciting time because there’s an explosion of knowledge in biology, an explosion of knowledge in technology, an explosion of knowledge in astronomy, physics, all over the place,” says David G. Hartwell, a senior editor at Tor Books. [...]”
The article compares the singularity with the invention of agriculture -- human society changed in ways that the hunter-gatherer could not have predicted. Today's explosions of knowledge seem to me to be a new renaissance.
2006.May.31 WedThere's a bible verse that says "he will open the eyes of the blind and unstop the ears of the deaf". Unfortunately, most humans don't have that capability. Here's a blog entry where Jon Kern, an agile expert, gives a talk for executives, and some of the executives storm out of the room because, as far as they are concerned, he's talking dangerous nonsense.
I saw nearly the same thing occur when Alan Shalloway gave a talk on agile methods at a former employer of mine. No one stormed out of the room, but there were questions whose unspoken assumption was "What you are saying is impossible." Probably few, if any, minds were changed in that presentation. (This inability to convince via words and slides is why the AYE conference doesn't give talks.)
It's an old story. Here's one example:
A Buddhist scholar has come to study with a Zen Master. The scholar is an expert on various Buddhist sutras and commentaries. When introducing himself to the Zen master, he began to talk about his studies, and his analyses, and so on.
The Zen master listens, says nothing, and starts making tea. When the tea was ready, he pours the tea into the scholar's cup until it spills over the edges, and he still keeps pouring the tea, spilling even more. The scholar cries "Stop! The cup is full; you can't pour more in."
The master stopped, saying: "You are like this cup; you are full of ideas about the Path. You come to me, and ask me to teach you, but your cup is full; I can't put anything in. Before I can teach you, you will have to empty your cup."
The modern equivalent is Stephen Covey's "5th" habit of successful people: "Seek first to understand, then to be understood."
If you can, involve the audience/students in a simulation (a.k.a. "games" a.k.a "experiential workshops"). This page at AYE mentions a a course that used a few simulations:
2006.May.30 Tue[an instructor writes:] The cycle of experiential workshops, working together on a single project [...], weekly reflections, and occasional guidance and coaching created an amazing learning experience. Students were yelling at each other. Teams were claiming their part was done, but the other teams parts weren't (though when I asked how they could say this if the product did not yet run, the class was silent). People were in tears over how others were treating them. I had to "fire" one of the leaders the week before the end of the project. [...] a month after the course finished, while I was meeting with the student I had "fired." [...] he said that the course has changed the way he views every interaction with people. It has even improved his relationship with his girlfriend. Now that's success.
An article I wrote, demonstrating some refactorings, has been published in Better Software magazine. You should, of course, buy the magazine. The article itself is also on-line here (updated link)
There's also some additional material in the Sticky Notes section:
2006.May.26 Fri
- Links to more info about code smells.
- What is Refactoring?
- A link to Michael Feather's book Working Effectively with Legacy Code
- A link to Joshua Kerievsky's book Refactoring To Patterns
- A link to Martin Fowler's book Refactoring
- A link to William Wake's book Refactoring Workbook
- Code for one of the classes derived from refactoring in my article, using C++'s "Resource Acquisition is Initialization" (RAII) idiom. (This appears to have lost its formatting. I'll ask 'Better Software' to fix that.)
- Further Reading
This article by Esther Derby is about real-time feedback of the human variety, but I mention it here because the strength of Test-Driven Development is also real-time feedback.
In TDD, you write a test that provides a clear, specific example of what the code under test will do. Then you write the code to do it (if this takes more than 5 minutes, you've probably bitten off more than you can chew, and you should consider commenting out the test and writing a more focused, smaller one). Then you refactor. Repeat this cycle. Eventually you have a bunch of tests that the code is passing. If you accidently mess up the code, one or more of the tests should failing and pointing out the problem within five minutes.
2006.May.25 Thu(I tried to write this without spoilers. It does assume you've seen episodes that feature "the hatch". If you don't know what that is, stop reading now.)
I think the reason that a person has to "hit the button" in the TV show LOST, is that the "geomagnetic anomaly" can only be suppressed by a thinking being. It's a "psycho-geomagnetic anomaly". Yeah, I know there's no real science behind that, but for science fiction, this isn't unusual.
2006.May.24 WedI'm renaming my blog "Exploring Solution Spaces". The old name, "MemoRanda" doesn't give a hint as to what I'm hoping to offer here. "Exploring Software Development Solution Spaces" is a bit too long.
2006.May.20 SatOn the Agile Testing mailing list, in a response to the never-ending
debatediscussion of "Manual Testing versus Automated Testing" Ron Jeffries has finally pinned down the false dichotomy at the heart of that debate:The only time not to automate a test ought to be if we are never going to want to run it again. But it isn't. The more common time we don't automate a test is because it would be "too hard" to automate it. That is, however, not really a property of the test, though we like to put it that way. It's really a property of our own test automation capability and tools.
This is also the difference between unit testing and testing via the debugger. If you or some other programmer is ever going to modify that code you just wrote and tested via the debugger, you will save time overall by writing a unit test to do that re-testing automatically. (Even better, TDD it.)
As Ron points out, our tools are too difficult. If manually testing the app created automatically re-usable tests that didn't break or get confused by minor changes in UI, that would be great, but few if any teams have that capability.
It can be very difficult to get legacy code under test. "RetroFITting" FIT tests into a legacy app can be extremely hard. (Christopher McMahon appears to have coined RetroFITting) So we forgo automated tests because of the expense, and repeatedly "run" manual tests. And in iterative development, you need to repeatedly test to make sure nothing previously shown to be working has stopped working.
XP makes creating automated tests a high priority because otherwise, in practice, automated tests don't get done at all. And then the expense of repeating tests manually will turn an agile project into a waterfall one.
2006.May.15 MonIn email, James Bullock commented on my Death March post, saying, among other things:
Interesting, succinct observation that [Death March projects] aren't the critical ones. I have tended to go at this observation from a different POV. I called it "getting serious." A "death march" project isn't properly staffed, scoped, or run to give a high probability of success - per the avowed goals. So either we're blind, or dumb, or these aren't the actual goals.
[...]
The thing is, for projects that really are important, somebody gets serious eventually. Until them, whatever is going on is mostly nonsense. So, I look for that getting serious event, and test it operationally out in the world. What are they willing to consider? To do? What skin are they willing to put in the game personally?
[...]
Thanks for the post. Once again, you have given me to think.
So Al Gore has been making multimedia speeches about global warning, and the producer Laurie David decided to make this into a movie, recruiting 'Deadwood' producer Davis Guggenheim to direct. Guggenheim persuaded Gore to open up about his personal life to provide the human connection for the film. A quote from an interesting interview about this:
Q. The marketing for the movie -- the trailer and the poster -- are completely over the top. "The scariest movie you'll ever see!" But the movie itself is quiet and methodical, and quite hopeful at the end. Did you deliberately choose those respective strategies?
A. It's a great trailer, very effective. But the people who make the trailer are completely different from the people who make the movie. I think they've done a terrific job on the movie, and I think a different group did a terrific job on the trailer.
The purpose of a trailer is very different from the purpose of a movie. I talked with Steven Spielberg, who saw the movie and loved it, and saw the trailer and loved it. And I asked him pretty much the same question you're asking me.
Also check out his recent appearance on Saturday Night Live. Funny!
2006.May.14 SunA Death March Project is one where the participants know the likelihood of success is very low from the very start. Wikipedia adds "Often, the death march will involve desperate attempts to right the course of the project by asking team members to work especially grueling hours, weekends (sometimes with a straight face), or by attempting to 'throw (enough) bodies at the problem' with varying results, often causing burnout."
I think one nugget of valuable information in the book Death March is that death march projects are never truly important projects. (The participants may be told it’s important, but really important projects are given more of what they need to succeed.) Thus death march projects are likely to be cancelled, even if someone turns around the development/technical side of things to successfully deliver working software.
I don't know if there is a word or phrase (bad management is too broad) to describe companies where death march conditions (particularly overtime) are the norm.
If you're on a death march project, remember that you have choices. You could work normal hours knowing that the doomed project would likely fail or be canceled even if you worked overtime. You could try to turn it around into a technical success, aware that a political success is very unlikely. You could look for another job or project. You could even try to convince the sponsors to cancel the project early and invest in something more likely to succeed.
2006.May.09 TueVia Christopher Petrilli, I see Geoffrey Grosenbach blogging about autotest-rails, with a movie demonstrating it in action. This helps him with developing web-apps using Ruby with Rails. Quote:
2006.May.08 MonAutotest runs continuously and watches to see what files you have edited. Any time you save a file, it will run the corresponding test within a few seconds. If a test fails, you can see the result immediately....
The revolutionary part of this is that it speeds development by helping you develop without needing to open your web browser! I find myself thinking more about the functional issues that need to be solved ...
Eric Hodel has a good post on Test-Driven Development and Creative Flow. Check it out.
2006.Apr.29 SatAndy Dent describes an under-rated method, "Solution-Based Modelling", that was eclipsed by UML, etc. Probably because the method was published in a book with the unfortunate title Developing Object-Oriented Software for the Macintosh : Analysis, Design and Programming written by Neil Goldstein and Jeff Alger. Some snips from Dent:
Goldstein & Alger developed an OO design method that was clearer than UML or Booch and better founded in theory. It was a standard in EDS but has vanished into obscurity. [...]
Notations have a long history in computer softare, starting with flowcharts and proceeding through structure charts, data flow diagrams...various ways of visualizing objects and classes. However helpful these notations have been, they are quite crude by graphic design industry standards. The authors know this first-hand. They enlisted the help of a professional design firm to help develop a notational scheme [...]
I got myself kicked off the early UML design discussion mailing list for taking Booch et al to task for their refusal to consider that they might not be the world's best graphic communicators and should consult some specialists. Sorry, UML users, I tried!
James Plamondon also discussed the book and the method here. A quote:
[...] Jeff Alger introduced a crowd of over fifty attendees to Solution-Based Modeling (SBM). SBM is the subject of a book forthcoming from Addison-Wesley entitled "Developing Object-Oriented Software for the Macintosh," and is the brainchild of Alger and his coauthor, Neal Goldstein.
SBM is an integrated, lifecycle approach to software development. It emphasizes the importance of involving marketing, management, and other diverse groups in the software development process from the outset. Where other methodologies-most notably the "waterfall" model, in which analysis "flows" into design, design into programming, and so on-look at software development as a series of separate stages, SBM takes a more incremental approach.
Despite the title of Alger and Goldstein's book, SBM is a platform and language-independent methodology. (The publisher insisted that the word "Macintosh" be in the title; their studies have shown that people prefer to buy books directed at a specific platform. This led Goldstein to suggest that the book be titled "Platform-independent Software Development for the Macintosh," [...]
Writing Pong Players in Smalltalk
Andres Valloud writes about how he won a coding contest for software that played Tennis Pong, Football Pong, Hockey Pong, Multiball Pong, and Super Pong some snips:
2006.Apr.27 Thu[...] The first phase of the contest was about implementing an autonomous player for 4 different versions of Pong. The games ran in a server, which would provide the status of the game in ascii, and accept commands via an URL with arguments.
All this stuff suggests a collaboration of multiple Smalltalk processes. [...] the strategy, which encapsulates the perception process [...] and the action process [...]
[...] the eyes, which take the latest raw status from the perception process and reify objects like balls, bats, the field, the scorecard, etc. [...]
[...] the strategy. It pulls the objects reified by the eyes and produces objectives based on the kind of game and whatever relationships it can perceive between the objects in play. The strategy then pushes objectives for the hands to meet [...]
[...] raw commands are queued by the hands at the interface by means of pushing. The action process pulls these raw commands out of the queue and pushes them to the game server.
All this is great except that the server was designed to not answer a status request until the next frame would start. This means that VW would lock until there was a new status [...]
Since the process scheduler and the processes themselves were nicely written, I changed the inner workings from 5 concurrent Smalltalk threads into 1 by reimplementing one method. Immediately, things started working much better. [...]
Since VW was quite fast even on this slow machine, I could bring up the server and fork off two players. Thus, I could see them play against themselves. I debugged the implementation by looking at the server display. Not the most time efficient manner, but it had to do.
[...]
We had 2 hours to implement this, but I had two Super Pong players competing against each other in about 30 minutes. Overall, I had created a strategy class with a few methods, an objective class with one method, and I had also pushed up a method in the strategy hierarchy. I spent quite a bit of time making sure they would not crash. And the final phase of the contest came to an end.
Cory Foy shows us an interesting job posting. Check out the code.
2006.Apr.22 SatIrritating, Dumb Servants and Smart Servants
Alex Bunardzic explains why "Clippy", "Microsoft Bob", etc., are "dumb servants" and why they are irritating.
- They are intrusive
- They are phony
- They are insincere
- They are manipulative: They would declare to know what to do, but as soon as a human user falls into the trap of engaging them in a conversation, they will turn the tables on humans and would saddle them with unasked-for responsibilities
- They are unaware of time
- They are insensitive to the context: All that Clippy can detect is a trivial fact that the email client has been activated by the user, [...] But that’s a far cry from ascertaining the real context of the situation.
- They are visual: Dumb servants, such as Clippy, suck at language.
He also writes about Smart Servants:
- Smart Servants are Aware of the Context: If the servant is smart, he will know that he needs to wait for the opportune moment [....] He knows that there is a difference between the situation when his master is lounging, and when his master is absorbed in a more important activity.
- Smart Servants Possess Very Reliable Memory: So, a dumb servant (and you cannot possible get any dumber than Microsoft Word) is easily identifiable as someone who isn’t paying any attention to his master’s preferences. In addition, this dumb servant tends to promptly and merrily forget everything his master had showed him and had instructed him to do. [Such as saving files in the "Drafts" folder instead of the "My Documents" folder.]
- Smart Servants Make Correct Assumptions: Furthermore, smart servants assume that their master prefers to be treated with respect.
- Smart Servants Never Insist that their Master is the Wizard: This ‘wizard’ was supposed to come equipped with the expertise needed to install the product [...] But then, all of a sudden, and without any forewarning [...] the ‘wizard’ was asking me how should he perform the next critical step, while at the same time warning me that a wrong decision on my part could jeopardize the entire installation.
My observation about Microsoft's "My Document" folder and related matters.
Have you noticed that the default setup on a Windows machine doesn't have a "Other Documents" folder? Where are you supposed to put all those documents that people send you, that you don't want to claim as your own? The recycle bin? I suppose Microsoft used "My Documents" in order to copy (but not copy exactly) the "Documents" folder that Apple provides its Mac users, but they didn't think beyond having a single folder.
In fact, a smarter system would recognize who sent you the document (either the email sender, or detect an "author" name within the document) and suggest filing the document under folders (automatically created as needed) with names like "Documents from Jane" or "Spreadsheets from Tim".
Using hard-links or soft-links (a.k.a. "Alias" for Mac users, "Shortcut" for Window users), smart software could also make a document accessible from several "author" folders if there were several authors, or even "project" folders (detected from email subject lines or document titles or keywords).
A year ago, there were articles about Apple's Spotlight search technology that predicted the end of folders. I haven't noticed folders going away. I just tried saving a spotlight search (which creates a so-called "smart folder") to the desktop, but it didn't save it there. I had to find it by doing another spotlight search for "search". It got saved to "~/Library/Saved Searches" and I had to move it from there to the desktop. It's called a "saved search query" (a.k.a. "smart folder") and looks like a folder with a gear image in it. You can't save a file into a these pseudo-folders, so you still have to put the files in a real folder somewhere. It is nice that these pseudo-folders are accessible from applications' File/Open dialogs, so you can open files from them.
Team-Building Stages and Agile
Mishkin Berteig write about the five stages (Forming, Storming, Norming, Performing) of team-building and how agile adoption figures into it. Check it out.
2006.Apr.21 FriPaul Thurrott writes:
Microsoft's handling of Windows Vista has been abysmal. Promises have been made and dismissed, again and again. [...] the initial code base was a teetering, technological house of cards. Windows Vista, in other words, has been an utter disaster. [....]
Microsoft has made some mind-numbing mistakes. It (illegally, [...]) artificially bundled its immature Internet Explorer (IE) Web browser so deeply into Windows in order to harm Netscape that it's still paying the price for the decision--a full decade later--in the form of regular critical security flaws [...]
[...] the Windows Division retains [...] the last vestiges of the bad, old Microsoft. This is the Microsoft that ran roughshod over competitors in order to gain market share at any cost. The Microsoft that forgot about customers in its blind zeal to harm competitors. [...]
[...] Sadly, Sinofsky came on board too late to help Windows Vista. But it will be interesting to see if he can remove the cancer that has almost destroyed the Windows Division from within.
[...] Sadly, Gates, too, is part of the Bad Microsoft [...]
Shame on you, Microsoft. Shame on you, but not just for not doing better. We expect you to copy Apple, just as Apple (and Linux) in its turn copies you. But we do not and should not expect to be promised the world, only to be given a warmed over copy of Mac OS X Tiger [...]
[...] As with the broken promises, Vista's failures are legion [...]
[...] They completely botched UAP [User Account Protection] [...] It's the most annoying feature that Microsoft has ever added to any software product, and yes, that includes that ridiculous Clippy character [...] the dialogs stack right up, one after the other, in a seemingly never-ending display of stupidity [...]
It seems like Paul is a Microsoft fan -- and fans can be the harshest critics because they pay close attention to what MS actually does.
2006.Apr.18 TueTest driven development (TDD) is a design tool. So is a piece of paper on which you sketch out code. Or a white board on which you sketch out UML diagrams. On a somebody's blog, which I lost the link to, some commenter complained that TDD doesn't force you to do the right thing. Neither do the other tools I just mentioned. These are thinking/design tools.
The design process I go through with white-board, or pencil and paper, or TDD, or just writing code and exercising it in the debugger, is not terribly different for each of these tools. They do have their individual weaknesses: paper or whiteboard means I have to exercise the code in my brain, testing code in the debugger can be slow, TDD tests are not always easy to write.
The key difference with TDD, which is a thinking process that I emphasize isn't too different from designing using the other tools I mentioned, is that after going through this designing-and-coding process, we have unit tests that we can save and run again later. No additional effort required.
Doing design + coding with the other methods does not automatically result in unit tests, so if we want them, we have to make extra effort to write unit tests. And extra effort means things don't get done, or get done grudgingly. This psychological factor is not to be understated: you think you're "done" (coding), so you don't want a failing test to show you that you're not "done". So you don't test very aggressively. In TDD we want the test to fail before we make the test pass, in order to test the test; this is a very different motivation and can actually be fun!
Testing after the fact is more difficult because the code often isn't designed to be testable. "Testable" is a symptom of good design, but rather than seeing that some difficulty in testing is a flaw in our design (which impacts our ego when we think we're good designers), we tend to see the difficulty of writing unit tests after-the-fact as being a flaw in unit testing itself. As a result, many people avoid unit testing, and see experience reports and studies that show unit testing reduces defects and reduces the costs of development as being irrelevant to their situation, or just plain flam-flim and lies.
(By the way, if you design code and unit tests at the same time, that's pretty close to TDD and gets you many if not most of the same benefits. But remember that you and the programmers who do TDD are uncommon programmers.)
2006.Apr.16 SunTurning Off Potential Passionate Users.
Good user-experience is good marketing.
No one is passionate about something they suck at. If they can get beyond the beginner stage, and begin to "kick butt", they can become passionate about whatever tool is helping them do that. This is why so many Mac users are passionate about the Mac. They get beyond the beginner stage rapidly.
On the other hand, Microsoft Windows will give you terrible error messages that make a beginning user feel stupid. Here's some examples:
- In some versions of Windows, trying to save an empty file in Notepad will get you this error message: "You have not entered any text to be saved. Type some text, and then try again." This frustrates people who like to name the file by saving it before writing the text.
- Another error message is just useless: "Copy Profile Error: The operation completed successfully".
- And this one is fatally stupid: "Cannot delete filename. There is not enough free disk space. Delete one or more files to free disk space, and then try again."
And those error messages will still be there no matter how experienced you are at using Windows. Some people get beyond those messages, learning to ignore those condescending error messages, and become passionate Windows users. But the percentage of users becoming passionate is much greater for Mac users than for Windows users.
Steve Yegge writes about Python not being marketed properly. He seems to like Python; he wants it to popular. His example is purely about this "creating passionate users" experience. Quote:
[...]consider what happens when you type "python" at a command prompt. It fires up a little interactive interpreter. At the prompt, if you type "quit", it responds with 'Use Ctrl-D (i.e. EOF) to exit.'
Well that's not very nice, is it? It *knows* you want to quit, even going so far as to call you an EOF, whatever that means. (Yes, you and I both know, but is it really the right thing to show to a beginner? Hardly.) Why didn't it just quit, then?
If you were to bring this issue up on a Python newsgroup at any time in the past 10 years, someone would tersely have instructed you to go look at the FAQ. Or they'd have explained that having 'quit' quit would be a strict violation of the semantics of the REPL [...] blah Blah BLAH [....]
Never mind that it's patently obvious that "quit" should just quit the frigging shell, semantics be damned. They don't care a whit, because they're focused on the "right thing" at the expense of the user experience. There's an old adage for this; it's called "missing the forest for the trees."
Of course it's just as difficult to figure out how to exit the Perl shell, if not more so. But if you were to bring it up on a mailing list or newsgroup, some nice Perl person would come along, eager to show you how to add one more snippet of job security to your lineup of Perl folklore, and would spend an hour explaining how cool it is that you can quit the shell with a single keystroke, one that works in other Unix commands as well, and then maybe show you how to hack the Perl binary so that "quit" also exits the shell for you. The difference is huge: both shells have that crappy misfeature, but Python folks will bore you with justifications while the Perl folks excite you with marketing.
[...] 'cuz Ruby is soooo cool. Did I mention that "quit" exits the shell in Ruby? It does, and so does Ctrl-D. [...] Ruby's WAY fun [...]
By the way, one of things that rocks on the Mac these days, is that just about anywhere I'm typing a piece of text, even in a the web-browser or in the File-Save dialog, I can control-click on a word to spell-check it. That popup menu also includes the option "Search In Google", which is great if the word isn't found in the built-in dictionary. I used that feature just now when saving this file before publishing it to my blog. I didn't have to add software or otherwise "turn on" this feature - it's just there.
Kathy Sierra talks about passionate users in this MP3 recorded at a conference.
2006.Apr.15 SatThe Iceberg Class and Predicting the Future
Someone on a mailing list asked why would he take his private methods that unpack a legacy code data-structure out of his class that uses them, "just to test them." Here's my answer:
LegacyDataStructureUnpacker sounds like a great stand-alone class. If its code is embedded as private methods in some other class, that makes the program much harder to understand, and that other class has too many responsibilities (unpacking as well as processing, or whatever). Extracting LegacyDataStructureUnpacker into a stand-alone class not only makes it testable, but also more understandable. Understandable code is less likely to be abused or misused.
And if some other 'user' besides the tests and the processing class, needs to re-use LegacyDataStructureUnpacker, there it is, ready for re-use. Made re-useable because of test-driven-development and the extract-class-refactoring. The processing class is a user, the tests are another user. By having two users, the class LegacyDataStructureUnpacker is more likely to be reusable.
We can't predict the future, but making classes have a single responsibility makes us agile enough to handle the future. Glomming loosly-related sets of functionality into the private methods of big classes keeps us from being agile. That creates Iceberg Classes. It's attempting to predict the future : "no one else will need this set of private methods, ever".
"But I don't want anyone else using this" is the objection that comes up. You don't want this "public" now, but who knows what the future will bring? However, you do want this correct (and thus tested) now, and understandable now, so those are good enough reasons to make the functionality into a stand-alone class now.
The keyword "private" is small protection, when anyone can rewrite "private" into "public" in your source code, or if you're using Java or some other environments, effectively changing private into public by hacking your .class file without changing the source. If you don't want the extracted class used by others, then explain WHY in comments, documentation, and face-to-face.
2006.Apr.11 TueTell, don't ask. Keep this slogan in mind when doing object-oriented programming. Doing this assiduously results in good OO designs. Quick example:
// BAD: asking another object for its values // and then computing something with those values. float w = rect.width(); // ASK for width float h = rect.height(); // ASK for height float area = w * h; // compute area println( "you will need " + area + " square meters of fabric" ); // GOOD: tell the other object what you want, and // it figures out how to do it. float area = rect.area(); // TELL me the area println( "you will need " + area + " square meters of fabric" );Buddhism says that "ego" is something we make up, and thus have to constantly work to maintain it. This blog entry by Adrian Savage applies that understanding to leadership:
"Ego and egotism are fatal to good leadership. They cause over-optimism, over-confidence and arrogance. They inflate people into domineering monsters focused on petty personal victories, wreck relationships and encourage leaders to take on too much [...]"Johanna Rothman points us to 100 rules for project management. I quote a couple of rules that relate to Ego:
Rule #11: Never try to get even for some slight by anyone on the project. It is not good form and it puts you on the same level as the other person and, besides, probably ends up hurting the project getting done.
Rule #12: Don't get too egotistical so that you can't change your position, especially if your personnel tell you that you are wrong. You should cultivate an attitude on the project where your personnel know they can tell you of wrong decisions.
Rule #28: People who monitor work and don't help get it done never seem to know exactly what is going on (being involved is the key to excellence).
Rule #63: Software has now taken on all the parameters of hardware (i.e., requirement creep, high percentage of flight mission cost, need for quality control, need for validation procedures, etc.). It has the added feature that it is hard as blazes to determine it is not flawed. [...]
I tossed in rule 63 to remind us in the software business that feature-creep was around in the days of "mostly hardware". And it needed testing and validation then just as software needs testing and validation now.
2006.Apr.06 ThuAddictions come from feedback loops that contain "mixed messages". You take a drink or a pill and feel better... for a while. Then you feel worse. So you take another drink or a pill. And so on. Short-term: feels good, long-term: feels bad. Because whatever felt good in the short-term was actually bad for you.
Here's an interesting blog by J. Timothy King about some of the addictions developers pick up: Check it out.
2006.Apr.04 TueI've been reading his blog for a while and it's time to point out some blog entries that I found interesting.
2006.Apr.03 Mon
- Coding Without A Net describing his reading "Agile Web Development With Rails" and practicing Rails programming. See also his other post on this book.
Gaming Testing Metrics. How collecting and tracking estimated fix times failed to make a change in a "death march" project.- Better Feedback Loops with One on Ones. Ed is a manager and has been doing one-on-one meetings with his direct reports every week. He was inspired to do this from podcasts and books like Behind Closed Doors and likes the results.
- Story Points Estimating in which he talked about Scrum and estimating in Ideal Days or Story Points.
- Not Playing With Scrum which describes how he (and his agile coach) will deal with a developer who hasn't bought into Scrum yet. (Not necessarily in the way you would expect.)
- Our Agile Methodology: 6 Month Review. He summarizes the good and bad impacts of having adopted an agile method based on Scrum. This is a real-world experience report, so read it!
Agile is Based on Communication and Feedback
Talking with my wife about "solving problems for people" versus "solving technical problems". In the early days of computer engineering / computer science, just getting the software and hardware to do something useful was a problem, and developers could easily spend all their time on technical problems and avoiding people.
Today, (at least in theory) developers don't have to worry so much about technical problems (other than those problems we've created for ourselves).
The majority of problems today are those needs and wants that *people* have and to solve them requires that the developers understand other people -- which requires interaction and two-way communication, preferably face-to-face. Two-way communication is necessary to get proper feedback on whether information is being understood on both sides.
Esther Derby and Diana Larsen provide a workshop on agile skills that starts with communication. They call it Secrets of Agile Teamwork (PDF), See also her website. It seems that their next workshop is June 6-8 in Portland, Oregon.
2006.Mar.29 Wedminimsft. What's interesting is not so much the blog post, but all the comments; many, if not most, of them made anonymously by Microsoft employees. There's so much interesting stuff there that it's hard to pick a few best bits to quote. But here are a few quotes:
Yep, it's time for a shareholder revolt. Vista is the biggest software development failure of all time, outside of the federal government. IBM's office vision was the previous record holder, with $900M spent.
You know, I've pondered for years what MS would do in this situation, when it became clear that the OS was a complete train wreck.
Apple was able to buy NeXT, but MS has killed off all of their viable replacements. OS/2, BeOS, PenPoint? All strangled by MS's anti-competitive (and illegal) tactics.
So, here's the way out: MS should swallow real hard, ante up half of what they blew on Longwind, and buy an OS X license from Apple. That would be about $10B up-front, and a hefty royalty. MS would have to assume the burden of making it run on all the crapbox PCs out there, which have had all the quality squeezed out of them, due to MS's having sucked up the lion's share of the profit from all PCs for the last 20 years or so.
The benefit is that MS could finally ship a securable OS, and the users wouldn't have to lose countless hours trying to work around the malware. Meanwhile, the only semi-competent part of the company, the Mac Business Unit, would take the lead in Apps development.
Microsoft is depriving some village of its idiot. Send him [Steve Ballmer] home.
Is this what Windows has become? An upgrade no one wants, forced upon them because the new hardware they're buying doesn't support anything less?
Compare this to OS X, where people fall all over themselves trying to get the newest version running on their old hardware because there's actual value in the new features.So Vista has its guts ripped out, slips, and we wait another 5 years for a potentially insipring version of Windows, meanwhile Apple ships another 3 updates to OS X.
Vista - I wouldn't buy it with someone else's money. Then again What do I know, I've only been testing the dog for the last 2-3 yrs...
It almost feels like the cement of bureaucracy has set in at all levels. Its unbelievable how much of a pain in the butt it is to get anything done. If you need a bug fixed and its in you team you usually have a small fight to put up, but God help you if that bug is in another team and you are dependant on the fix. Forget about usability, or doing whats best for the user experience, its all about doing less work and managing perception to the upper brass. Thank you for letting me vent.
I took part in a computer trade show early this month in Germany, and Microsoft was showing Vista, and the Microsoft fans were saying it looks like OS X (Apple wasn't there). Apple is on a roll, and we've just given them enough time to get the next version of OS X out the door (whatever animal name it is going to be). And we can guess right now what their marketing push will be: Stop waiting for those guys who can't even copy our old stuff in time. Get the original from us -- we ship on time, we're shipping right now.
Quality has gone down, not up, since the emphasis was changed from individual developer accountability to heavy-handed process initatives that allow little time for real work on improving quality.
I wonder how many employees at PC hardware companies are wishing they had some way to call up Apple and license OS X for Intel. They could have 10.4 "Tiger" on PC hardware in a matter of weeks.
It's just not ready - just as my stuff isn't ready, shell isn't ready, the drivers, the perf.. The screw up did not occur now, not one year ago but way before that.
Why, you ask, wasn't my stuff ready on time? Because everybody works the same way, only intensifying their efforts around milestones. Tests weren't run, bugs were laying dormant, people were allocated to side, pet projects and vendors only pay attention to pri 0 bugs older than 2 weeks (if not longer). It all shows up now, and it's all important. Yes, it's my fault for not screaming earlier, but there must be at least two of us, 'cause I didn't write Vista by me onesy. It's also the manager's fault, for he didn't take steps to streamline my work. It's his manager's fault, too, 'cause he didn't infer from the greater picture that things are not moving progressively. See where I'm going? It's all of us, and the higher the rung we're clinging to, the greater the responsibility.
Basically we do not believe Vista will make January 2007 or even March 2007. Anyone with any access knows what a frankenstein's monster NT is on the inside. At some point there is a law of diminishing returns trying to do anything to it at all, it seems like that limit is being reached today. The release is pushed back because of bugs but fixing those bugs will create more bugs. It is just godawful to be honest. And the process gets in the way at every step. [...] I need to feel like i'm creating something good, not fighting 10 years old cruft every step of the way.
In fact, people often played schedule chicken. It didn't matter if you were running late by the metric of the day as long as another group was running later.
What's the difference between OS X and Vista? Microsoft employees are excited about OS X.
I'm only one-fifth of the way through reading that blog's comments. That reference is "schedule chicken" is very important. Google for it. Read what Johanna Rothman and Gerald M. Weinberg (among others) have written about it. It's a symptom of a very dysfunctional organization.
2006.Mar.27 MonWhen I heard that a Chronicle reporter was looking for Mac "Fans" for a newspaper article about Apple's up-coming 30th anniversary, I expected that would get printed would be the usual condensed history of Apple and several examples of those "crazy" Mac fans who shave or tattoo the Apple logo into their hair or body. I sent an email to the reporter in hopes that maybe they would do a little more than that.
But that's pretty much what happened. They did at least interview Woz and other people associated with Apple's history, though Steve Jobs declined to be interviewed. And they quoted me. The irony of course, is that despite quoting me, they pretty much did what I hoped they would not do.
Here's what they printed, in an article entitled "Faithful, sometimes fanatical", in a section that starts with "Well, you might be an Apple fanatic if:"
You get annoyed with how the media portrays Mac fanatics.
"One thing I don't like is newspapers and TV newcasters reporting how 'fanatic' Mac fans are, without even trying to explore the reasons for their preference," said Keith Ray in an e-mail. "Please don't you continue that lazy tradition. We're not crazy. Some Mac fans are rocket scientists at NASA and JPL (Jet Propulsion Laboratory), others are very non-technical like my mom."
What got edited out of that paragraph was "Probably all of us have used other platforms -- I doubt any Mac user has been fortunate enough to have only used the Mac."
My full email follows:
2006.Mar.21 TueResponding to "I’m writing a story about Apple’s 30th birthday and am looking to talk to Mac fans in the San Francisco Bay Area, the more devoted, the better. I want to hear about how they fell in love with their Mac, what they do to show their commitment, and, of course, what they think about Apple turning 30 next week. [...]
I wanted a Mac 'way back in 1984 when Byte magazine showed us what it was going to be: What You See Is What You Get. MacWrite and MacPaint and the Mac operating system were revolutionary in ease-of-use, and they were quite powerful given how little RAM and disk space the Mac started with.
I've worked on Mac and Windows and Unix and other operating systems since then and have preferred the Mac for many sane and logical reasons. I don't have time to list them all here, but suffice it to say that the alternatives have so many little irritating deficiencies that whole books have been written on their problems.
[books: "The UNIX Hater's Handbook", "Windows XP Annoyances", "Windows XP Annoyances for Geeks", "Fixing Windows XP Annoyances", 'Windows ME Annoyances", "Word Annoyances" -- all these titles are available at Amazon.]
People who haven't used the Mac don't understand that life with computers could and SHOULD be easier than what they experience every day.
One thing I don't like is newspapers and tv newcasters reporting how "fanatic" Mac fans are, without even trying to explore the reasons for their preference. Please don't you continue that lazy tradition.
We're not crazy. Some Mac fans are rocket scientists at NASA and JPL, others are very non-technical like my mom. Probably all of us have used other platforms -- I doubt any Mac user has been fortunate enough to have _only_ used the Mac.
"Stanford lecture: Stress and Coping: What Baboons Can Teach Us" by professor Robert Sapolsky.
Sapolsky seems to have spent decades studying the biological effects of psychological stress. Baboons live in complex social groups. They only spend about 3.5 hours a day obtaining food, so they have plenty of time to compete, cooperate, work their way up the "corporate ladder" (of the pack hierarchy), and just be mean to each other. His initial hypothesis was that individuals higher in pack rank had less stress, but it turns out to be less simple than that.
High rank and social stability, are two factors, but certain personality traits are also important:
- Being able to tell the difference between winning and losing.
- Being able to tell the difference between threat and non-threat behavior of a rival or enemy.
- Having a outlet for aggression after losing a battle or stare-down (for male baboons, that means beating up female baboons, unfortunately)
- And last but not least, having friends.
Interesting stuff. More stuff by Robert Sapolsky is here and Why Zebra Don't Get Ulcers, free iTunes Link: Zebras.
2006.Mar.20 MonSo far, I haven't heard many podcasts I would want to subscribe to. The main repellent of most podcasts that I've listened to is that they waste my time. Lengthy musical introductions. "Station-break" sound-bits. the speaker is rambling along with no point in mind. If I'm listening to a podcast via my ipod while I'm driving to work, I don't want to try to skip ahead in that podcast - it's easier to skip to the next song or podcast.
The best podcasts I've heard have been interviews. The Agile Toolkit Podcasts are focused interviews that don't waste my time.
Often it has been said that "agile doesn't scale". This (corrected link) is a good, short, article by Dean Leffingwell on seven agile practices that do scale. They are:
2006.Mar.19 Sun
- Iteration. "create tested working code in a small time box"
- Unified Define/Build/Test teams. "eliminating the functional barriers [...] there is no limit to the number of such teams that can be formed!"
- Smaller and more frequent releases. (see Creating Passionate Users on Ultra-fast release cycles/myspace)
- Two-level planning. the long and short view
- Concurrent testing. Unit testing, acceptance testing, [etc. ...] occur inside each iteration [....] that persists in the future as regression tests [...] in future iterations
- Continuous integration. "I witnessed one team of approximately 300 people building a large-scale systems infrastructure application reduce the integrate-build-regression test cycle time from a month to less than a day."
- Regular reflection and adaptation. "empowered, agile teams naturally address and eliminate the roadblocks [....] Since this process is not prescriptive, it can give some discomfort to project management [...] who tend towards documented, prescriptive and mandated processes.
Roy Osherove writes (with examples) about how design improvements can make software testable. I know some readers will complain that these design changes somehow "weaken" the design, but they need to look at the bigger picture. In each example that Roy gives, the refactored (more easily testable) code has now separated concerns that were tightly-coupled. Tightly-coupled is bad.
Repeat after me: "Tightly-coupled code is bad."
Quote:
2006.Mar.16 ThuHere’s my current definition of a testable system:
"For each logical part of the system, a unit test can be written relatively easily and quickly that satisfies all the following PC-COF rules at the same time:
Partial runs are possible
Consistent results on every test run
Configuration is unneeded before run
Order of tests does not matter
Fast run time"You Can't Change Just One Thing
A paper [pdf] by Glenn Vandenburg entitled "A Simple Model of Agile Software Processes - or - Extreme Programming Annealed" works out why the "interdependencies" of the various XP practices are not necessarily a bad thing. I like this bit:
"...Some process redundancy is invaluable in the face of such flawed workers.
"Many processes try to deal with these problems by ... adding enforcement steps or inspectors, or by adding practices solely for the purpose of redundancy. But such measures are costly in terms of time and effort... One strength of the XP approach is that the practices play multiple roles. In most cases ... the redundant compensation is merely a secondary role of the practice... and has the added benefit of using core team members in enforcement roles without making them seem like 'enforcers.'"
Of course, a drawback of this overloading of a practice, is that when a team "adapts" XP but drops a practice, they're really dropping multiple aspects of that practice. To fully replace that practice could require adding in several other practices.
If the team drops Test Driven Development, they are not only dropping unit tests, with its fast feedback on the quality of your code, but also the designing/refactoring aspect of TDD. They could add back in after-the-fact-unit tests (which provide slower, more difficult-to-implement feedback), but they would also need to add back in more design and refactoring practices as well.
If the team drops Pair Programming, they are not only dropping the fast feedback from pair programming's 'continuous code review', but also they also drop the idea-generation that helps get coding done quicker, and lose opportunities for the 'enforcement' for coding standards, testing and integration processes. Adding in after-the-fact reviews will slow down the feedback and enforcement; and you would also need more brainstorming/design sessions.
2006.Mar.14 TueWritten up here. I am one of those "veteran" Mac developers. :-)
2006.Mar.13 MonReading Short Methods and Small Classes
How to read short methods and small classes.
When reading a Long Method, we figure out the algorithm from the implementation. For example, we see an expression, and we figure out that the line of code "i = (lowIndex + highIndex) / 2;" is computing the middle index. This long details-only style can get tiring, and tends to lead to a lot of duplicated code.
When reading a Short Method, the algorithm is obvious (if intention-revealing names are used), but we have to delve into other methods to see the implementation. For example, the line of code "i = middleIndex()" states the intent, but you would have to look at its implementation to see how it does it. This can also get tiring, if we can't trust that anything is implemented correctly and therefore feel that we must investigate every method.
One way to get the feeling that you can trust the class and method names, so you don't have to investigate each and every method, is to have extensive automated tests that verify that methods and classes are behaving correctly individually and in integrated groups. These tests can also serve as documentation -- examples of usage.
Another way to help understand short methods and small classes is to have some high-level documentation explaining their relationships. Design patterns are one way to do this. JUnit originally came with a Cook's Tour that presented the classes of JUnit in terms of design patterns: Adapter, Command, Composite, Collecting Parameter, etc.
JUnit also has very short Cookbook that tells you how to use the framework in the simplest ways. There are more sophisticated ways to use JUnit, but those are documented elsewhere. The Cookbook just has enough to get you started with the common cases.
The success of JUnit probably stems from its easy-to-read documentation as much as its simple, (mostly) decoupled, easy-to-read code.
How do you know if a function is too long? Jerry Weinberg's test for that is:
2006.Mar.11 Sat
- Pick a function.
- Understand it, what it's supposed to do, how that is accomplished.
- Memorize the function.
- Then, without looking at the original code, write the function from memory.
- If you've made a mistake, the function was too long, or otherwise too complex.
The first step of any engineering process is to define the problem. Testing is making measurements so that someone can determine if the problem has been solved.
(Paraphrasing Alan Jorgensen in the software-testing mailing list.)
2006.Mar.09 ThuDr. Joseph M. Newcomer has a nice rant about premature optimization. Quotes:
So what to optimize is easy: those parts of the program that are consuming the time. But local optimizations that ignore global performance issues are meaningless. And first-order effects (total time spent in the allocator, for example), may not be the dominant effects. Measure, measure, measure.
(He has examples of second-order and third-order effects.)
Why doesn't efficiency matter when you're updating menus or controls? Look at the human factors. A mouse is held approximately 2 feet from the ear. Sound travels at approximately 1100 ft/sec. This means that it takes approximately 2ms for the sound of the mouse click or keystroke to reach the ear. The neural path from the fingertip to the brain of an adult is approximately 3 feet. Propagation of nerve impulses is approximately 300 ft/sec, meaning the sensation of the mouse click or keystroke takes approximately 10ms to reach the brain. Perceptual delay in the brain can add between 50 and 250ms more.
Now, how many Pentium instructions can you execute in 2ms, 10ms, or 100ms? In 2ms, on a 500MHz machine that's 1,000,000 clock cycles, so you can execute a lot of instructions in that time. Even on a now-clunky 120MHz Pentium there is no noticeable delay in handling the controls.
And yet, UI on some Microsoft [and Apple :-( ] products is sometimes slower than my (not terribly fast) typing-speed... but seems to come from poor algorithms and data structures (or n-th order effects like cache-misses), not some skanky c/c++ "function-call efficiency" issues.
2006.Mar.05 Sun[benchmark optimization based on cache archituture]... That's a factor of 20 performance improvement based solely on fourth-order effects, cache line hits. If you're doing image processing, particularly of large images, an awareness of cache issues (even relatively machine-independent) can buy you an order of magnitude performance.
...
Perhaps the best example of pure programmer stupidity in "optimizing" code occurred when I was porting a large library we used for our research project. Think of it as a 16-bit-to-32-bit port [...] after 12 solid hours of debugging [...] Another 5 hours [...] instead of seeing something like something->p1 = NULL; something->p2= NULL;, I found the moral equivalent of (*(DWORD*)&something.p1) = 0! When I confronted the programmer, he justified it by explaining that he was now able to zero out two pointers with only a single doubleword store instruction [...] Of course, when the pointers became 32-bit pointers, this optimization only zeroed one of the two pointers, leaving the other either NULL (most of the time), or, occasionally, pointing into the heap in an eventually destructive manner. I pointed out that this optimization happened once, at object creation time; the average application that used our library created perhaps six of these objects, and that according to the CPU data of the day before I'd spent not only 17 hours of my time but 6 hours of CPU time, and that if we fixed the bug and ran the formerly-failing program continuously, starting it up instantly after it finished, for fourteen years, the time saved by his clever hack would just about break even with the CPU time required to find and fix this piece of gratuitous nonsense. Several years later he was still doing tricks like this; some people never learn.
Way back around June 14, 2005, I criticized Apple's suggested solution to for reading and byte-swapping PPob resources: a single very long function named _FlipPPob with switch statement for all the 'known' types of PowerPlant view/control classes. They have subsequently changed their suggested solution to be very much in line with what I was recommending. It's a little different, because they're taking advantage of the fact that PPob resources are always going to be big-endian.
//my code LStream & LStream::operator >> (UInt32 & outNum) { ReadBlock(& outNum, sizeof(outNum)); if ( mByteSwapping ) { outNum = ByteSwapUInt32( outNum ); } return (*this); } //apple's recommended code is now something like: LStream & LStream::operator >> (UInt32 & outNum) { UInt32 value; ReadBlock(& value, sizeof(value)); outBool = CFSwapInt32BigToHost(value); } // where CFSwapInt32BigToHost is defined to do something like: #if defined( HOST_IS_BIG_ENDIAN ) return val; #else return ByteSwapUInt32(val); #endifLooking back at my previous code, I can see where "if" statements could be converted to polymorphism. Most of the read and write methods of my modified LStream class in my original solution would have had an "if" statement (directly, or indirectly like Apple did). Using the Replace Conditional With Polymorphism refactoring, we would create two stream classes: one that does byte-swapping unconditionally, and one that doesn't. A factory method would choose between them - reducing the if statements from one-per-read/write-method (about 8 methods) to just the one in the factory method.
LStream* LStreamFactory( FileRefence& aFile, bool fileIsBigEndian ) { if ( fileIsBigEndian == HOST_IS_BIG_ENDIAN ) return new LStream( aFile ); // non-byte-swapping stream. //else return new LByteSwappingStream( aFile ); } // don't take the above too literally. I didn't look up what // parameters LStream actually needs in its constructor. LByteSwappingStream & LByteSwappingStream::operator >> (UInt32 & outNum) { ReadBlock(& outNum, sizeof(outNum)); outNum = ByteSwapUInt32( outNum ); return (*this); } //etc.Note that the base class of LByteSwappingStream (which could be LStream or some common base class) would need to declare methods like "operator >>" as virtual to enable polymorphism.
2006.Mar.04 SatQuote from Creating Passionate Users/How to be an Expert:
Most of us want to practice the things we're already good at, and avoid the things we suck at. We stay average or intermediate amateurs forever.
Yet the research says that if we were willing to put in more hours, and to use those hours to practice the things that aren't so fun, we could become good. Great. Potentially brilliant.
Check it out
Satir Change Model and Competency
"Unconsciously Incompetent" = Late Status Quo
(missing) = Foreign Element
(missing) = Resistance
"Consciously Incompetent" = Chaos
(missing) = Transforming Idea
"Consciously Competent" = Integration
"Unconsciously Competent" = New Status Quo
See Steven M. Smith's paper on the Satir Change Model. The "Unconsciously Incompetent, etc." meme is all over the web, but I'm not aware of a definitive source.
Using the above with my experience with test-driven-development. Before I encountered XP, I was in the Late Status Quo. I had never done TDD - hadn't even heard of it. I had had the experience of writing unit tests after-the-fact, and it wasn't pleasant. The fact that many studies and experience reports indicated that thorough unit testing decreases bug-rates (and bug reoccurrences) by 60% or more wasn't impacting my consciousness very much. Since I wasn't even doing TDD, I was "Unconsciously Incompetent".
Then I heard about XP and TDD and read some pretty amazing stuff by or about Kent Beck, Ward Cunningham, Robert Martin, Ron Jeffries, et alia. I had already encountered these people ("virtually") in newsgroups on object-oriented-programming, books on design patterns, and so on, so they had earned my respect already. This was the Foreign Element that triggered the change process that would get me out of the Late Status Quo. If these smart, impressive, guys were finding TDD to be so productive that they never wanted to go back to traditional programming techniques again, maybe it could help me, as well.
So I tried TDD. It was hard to break my habit of writing the code first. (Sometimes it still is). I sometimes wrote too-big tests, or too-big blocks of code to make the tests pass. I learned later that TDD is easier in a pair programming situation (you have someone to help come up with tests or ideas on what code to write to pass the tests), but my initial TDD experiences were solo. This was the Chaos stage, and I was "Consciously Incompetent".
Various books and examples on the web provided me with techniques on TDD. How to write really small tests. How to write really small pieces of code to pass the tests. How to use Mock Objects to enable staying focused on one class. (The "mock" class is an object that the class-under-test collaborates with, and may not even exist in a 'real' form yet. If you want to get picky, there are "Fake" and "Stub" and other flavors of test-assisting-objects similar to but different than "Mock" objects.) These and other resources on refactoring and code smells were Transforming Ideas that helped me Integrate TDD into my repertoire of programming techniques. I became "Consciously Competent". I began to experience the benefits of TDD: cleaner designs, regression tests (that were not painful to develop), no longer spending so much time manually testing through the GUI or stepping through code with the debugger. I could make a change, run the tests, and get a quick failure or success indication.
I no longer have to invent/learn "from scratch" when doing test-driven development, so I could be in the New Status Quo. I'm comfortable with the process, including modifying the test framework to support new testing needs as they come up. However, I don't like to think that I'm in the "Unconsciously Competent" stage. TDD doesn't work if you don't think about what and how you're doing what you're doing. If you don't think when you're doing TDD, you can end up with lots of trivial tests and duplicated code to pass the tests. After you've cleaned up a few messes that came from not thinking, you learn to keep your brain turned on. A pair programming partner helps in that they can challenge your assumptions. And there are always new challenges, areas I haven't yet attempted in TDD: Cocoa UI, for example.
Consciousness is not just a good idea, it's a way of life. :-)
2006.Feb.25 Sat
Design (Domain-, Test-, and Client-Driven)
Domain Driven Design in action: Time and Money library Java, open-source. Looks to have pretty complete unit tests, since it's written by Eric Evans, Martin Folwer and others, I expect most -- if not all -- of it was developed test-first.
An essay on Test Driven Development [aka Test Driven Design aka Client Driven Design] motivated by a cartoon that got it all wrong. Quotes:
...So let's just go over some of that conversation again, starting with a few quotes to warm up:
“The act of writing a unit test is more an act of design than of verification” - Robert Martin
“Test-driven development seeks specification, not validation, letting you think through your design before you write your functional code” - Scott Ambler
“Test-Driven Development is a powerful way to produce well designed code with fewer defects” - Martin Fowler
“Fewer defects, less debugging, more confidence, better design, and higher productivity in my programming practice” - Kent Beck
There's obviously a common theme here :)
... Test-Driven Development is about design. It might as well be called Test-Driven Design. It might as well be called Client-Driven Design. Let's try that...
In Client-Driven Design, you design your classes from the perspective of the client. ... This is fine-grained or detailed design. This is what Jack Reeves is talking about in his 1992 article in C++ Journal called What is Software Design (by way of Martin Fowler's June 200 article, The New Methodology).
... In Client-Driven Design you refactor the code when you realize a better way to do something. Once you've refactored, a good way to know if you've broken anything is to simply execute all the client code [tests] you've written to help you design your API. If anything is broken, you'll know immediately.
... We don’t like big steps. We've just spent a career learning that coding non-stop for hours without validating our design leads to wonky design that will make the rest of the system that much more difficult to build and understand.
Scott Bellware also responds to the question "I'm having a heck of a time understanding how to thoroughly exercise my class ... a class with one public method, which performs the majority of its work ... in private methods ... A class that provides functionality that is highly dependent on the state of a larger process, in which my class only performs one logical unit of work ..." etc. These questions mostly stem from trying to work with bad designs (one public and lot of private methods is the "Iceberg Class" (or "God Class" design smell), and from failures of imagination, or lack of experience with design patterns (for reducing coupling). If it's hard to test, it's a bad design. TDD forces you to make code that is easy to test, and thus helps you create a good design.
See also Martin Fowler's Is Design Dead?". Quotes:
2006.Feb.23 Thu...In its common usage, evolutionary design is a disaster. The design ends up being the aggregation of a bunch of ad-hoc tactical decisions, each of which makes the code harder to alter. In many ways you might argue this is no design, certainly it usually leads to a poor design.... Planned Design is a counter to this....
Now the planned design approach has been around since the 70s, and lots of people have used it. It is better in many ways than code and fix evolutionary design. But it has some faults. The first fault is that it's impossible to think through all the issues that you need to deal with when you are programming... Now some of these requirements problems are due to not understanding requirements clearly enough. ... Many unforeseen requirements changes occur due to changes in the business. Those can't be prevented, however careful your requirements engineering process.... So all this makes planned design sound impossible...
... People who refactor their code in the disciplined manner suggested by XP find a significant difference in their effectiveness compared to doing looser, more ad-hoc restructuring. That was certainly my experience once Kent had taught me to refactor properly. After all, only such a strong change would have motivated me to write a whole book about it.
... At a conference dinner, Dave and I talked with a vocal opponent of XP. As we discussed what we did, the similarities in our approach were quite marked. We all liked adaptive, iterative development. Testing was important. So we were puzzled at the vehemence of his opposition. Then came his statement, along the lines of "the last thing I want is my programmers refactoring and monkeying around with the design". Now all was clear. The conceptual gulf was further explicated by Dave saying to me afterwards "if he doesn't trust his programmers why does he hire them?". In XP the most important thing the experienced developer can do is pass on as many skills as he can to the more junior developers. Instead of an architect who makes all the important decisions, you have a coach that teaches developers to make important decisions. As Ward Cunningham pointed out, by that he amplifies his skills, and adds more to a project than any lone hero can.
... In order to work, evolutionary design needs a force that drives it to converge. This force can only come from people - somebody on the team has to have the determination to ensure that the design quality stays high. ... This will does not have to come from everyone...
... If you're in the development team, then you sense whether design is happening by the quality of the code base. If the code base is getting more complex and difficult to work with, there isn't enough design getting done. ... A project that does healthy refactoring will be steadily deleting bad code. If nothing's getting deleted then it's almost certainly a sign that there isn't enough refactoring going on - which will lead to design degradation.
Speaking assembly lines, Toyota stops its line when a problem is detected (my emphasis added):
A [one-minute] production line warning can be given as often as 400 times during a shift if members are uncertain about the quality of the part they have received, the performance of a piece of equipment, poor fitting or the member cannot keep up with the cycle time of the job. [...]
During this one-minute period the team leader or engineer who goes to the side of the member who has sounded the alarm must resolve the issue at hand or that part of the line will halt its operations.
This emphasis on quality is considered a Lean Principle. Even though software development is far away from an assembly line, people ask what the equivalent is on Lean/Agile methods like Extreme Programming. Here's the answer: it's the unit tests and acceptance tests. Specifically, unit tests must be passing at all times, and a story is not considered "complete" until it passes all its tests.
If the unit tests are failing, an XP team will stop new development in order to find out why the tests are failing and fix them. This is not just the "stop the line" equivalent, it's also necessary because XP's test-driven-development requires a cycle of write-a-failing-test, write-just-enough-code, see-all-the-tests-pass. If you start with multiple failing tests (not just the one you just wrote), that messes up that cycle. If you can't get all the tests passing with a small amount of coding, that also disrupts the cycle. Those unit tests are there to help you detect bugs -- ignoring them when they fail is letting the bugs breed.
In the case of acceptance tests, the counting of a story as "done" or "not done" when all the tests are passing (or not) has a regulating effect when the "yesterday's weather" rule is enforced. If your team tried to implement 20 stories in one iteration (velocity = 20), and only got 15 stories "done" (passing all their tests, velocity = 15), then the team can only sign up for 15 stories in the next iteration. This gives them time to finish the unfinished tests, and time to make sure the next batch of stories are fully tested as well. This ends up "stopping the line" entirely if zero stories were implemented, otherwise it just slows down the process. The team is responsible for requesting more stories to do if time becomes available, so it's possible for increasing the velocity as decreasing it. (For the sake of simplifying the wording of this paragraph, let's assume that all the stories in this example are the same 'size' -- 1 story-point each.)
2006.Feb.22 WedWilliam Pietri writes, in the XP mailing list:
Like many others here, the metaphor of software development shop as factory has always raised my hackles. There are a lot of reasons it's misleading, but I finally put my finger on what I think is the biggest one.
Imagine a traditional factory with your standard assembly line. Let's just think of a simple one: you've got your receiving department on one end, five stations on the line, and some final quality control just before the shipping department. Let's further assume that your people all get things right 99 times out of 100. Sounds pretty good, right? Of course, when you multiply it out, that means either quality control or your customers will toss out circa 5% of your parts, but as long as you account for that in your pricing, you're good to go.
Now let's imagine that's software they're making. If the typical bug database is any guide, software quality control is not as effective as factory quality control. Let's suppose that half the bugs make it through and onto the truck. Then the truck pulls away and pulls right back up at your receiving department. In other words, the main input to most software development is what the team produced the day before.
This difference entirely breaks the factory model, If you repeat the same quality-reducing operations over and over on the same material, your quality inevitably declines and your costs for producing something worthwhile will increase. In the end, you have to throw out everything as unsalvageable. Which sounds exactly like a lot of software projects I've seen.
To survive with a circular assembly line, aiming for perfection isn't good enough. Instead, every time we touch the code, we need to leave it better than we found it.
Quoted with permission.
Encapsulation Wasn't Meant To Mean Data Hiding
Martin Fowler: "the point of encapsulation isn't really about hiding the data, but in hiding design decisions, particularly in areas where those decisions may have to change. ... When you are thinking about encapsulation I think it's better to ask yourself "what peice of variability are you hiding and why" rather than "am I exposing data. (Craig Larman wrote a nice column for me on this.)" Larman quotes Parnas: We propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then design to hide such a decision from the others.
In Cocoa programming, we often have to declare a bunch of getter methods in certain objects using "key-value-coding/observing/binding" conventions so that user-interface views can synchronize their data with controllers and model objects. It's explained rather well here.
Outside of special purpose getters required by a framework, it is appropriate to look for opportunities to move methods closer to the data they work with. If a method in one class is calling several getters of another class, perhaps that method belongs to that other class. Fowler writes: "...a simple rule of thumb ... that I first heard from Kent Beck, ... always beware of cases where some code invokes more than one method on the same object. This occurs with accessors and more reasonable commands. If you ask an object for two bits of data, can you replace this with a single request for the bit of data you're calculating? If you tell an object to do two things, can you replace them with a single command? Of course there are plenty of cases where you can't, but it's always worth asking yourself the question."
Allan Shalloway wrote (starting with a quote from Design Patterns: "Consider what should be variable in your design. Focus on encapsulating the concept that varies. This mandate was a bit confusing to me at first. To me, encapsulation meant "data-hiding". But to the Gang of Four, it means something else: Encapsulating the concept that varies is encapsulation of type."
2006.Feb.21 TueZen master Shunryu Suzuki: "Everything is perfect just as it is, and there’s lots of room for improvement." (from Crossroads Dispatches)
2006.Feb.19 SunOne of the premises of writing user-stories for XP and other agile methods is the that story -- as written -- should not get into implementation details. However, the conversations between Customer/Product Owner and Developer and Tester can and should concern themselves with implementation details.
An example of a user story (from an article(pdf) by Mike Cohn is "A user can remove her resume from this site". That seems to be a "what" because it doesn't get into the details how the user performs this action. But this is a "how" when considering a higher level "what" -- "the user has wants to stop the job-hunting process". From that perspective, maybe an appropriate lower-level story, that we haven't considered yet, is "the user can close their account with this site."
At yet another higher level, the "what" story is "this company can make money by hosting a job-hunting web-site." Or maybe this is a non-profit? What stories would be different if this a for-profit venture rather than a non-profit venture, or vice-versa? If non-profit, is this backed by a church, a government, a co-op, or another source of funding, and what requirements do these funders have? If this is for-profit, is the money coming from the job-seekers, the job-posters, or both? Do the venture capitalists, who are providing the initial funding for this venture, have requirements that the current set of stories don't address?
Ron Jeffries and others the XP mailing list, have often recommended breaking down stories into very small ones that can be easily estimated. While the original XP project and early books recommended breaking stories down into tasks, with programmers signing up for tasks rather than stories, most agile gurus now recommend smaller stories, with no formal breaking down into tasks (though doing so informally can help with estimating the story.)
There is an example of breaking a large story down into smaller stories in an article (pdf) by Mitchel, Auernheimer, and Noble, which I quote:
The owner of a business process uses their web browser to look up a list of processes on the server, select a process they would like to invoke, fill out a form and submit it to start the process. A participant assigned a step in the process uses their web browser to retrieve the task description, which includes the data values entered by the process owner.
...It's too big to estimate... smaller stories...
1 The owner of processes can use their web browser to see a list of processes they own.
2 The owner can select a process they would like to invoke and do so.
3 The owner can fill out a form for the process before submitting it.
4 The process participant can log into the system.
5 The process participant can see a list of tasks to which they've been assigned.
6 The process participant can see a specific task.
7 The process participant can see the data entered by the process initiator at process invocation time.
In the ensuing conversation, story 4 is rewritten to use the user's Windows NT name and password, so they don't have to log into the web app after logging into their computer, and prioritized lower than the other stories, so in the first iteration no log-in is required at all.
By the way, there are a bunch of good papers on user stories at the Agile Alliance web-site. And Mike Cohn has two books on user-stories, both good: User Stories Applied and Agile Estimating and Planning.
The topic of "what == how" reminds me of the Theory of Constraints thinking tools known as "cause-effect-cause" diagrams. An item in such a diagram is usually both an 'effect' from some other 'cause' and a 'cause' to another 'effect'. An example here (see Figure 6) has 'Furnace walls need cooling' is an effect of two causes, 'it is necessary to heat the furnace to a high temperature (to melt the ore)', and 'the furnace wall cannot operate at a high temperature', and itself causes the effect 'cooling is achieved by circulating water'.
Similarly, the 'what' of the user is to 'kick butt', to achieve success in some way, (see the Creating Passionate Users blog) and the 'what' of the developer is to 'make money', or achieve success in some way. The 'how' is the product that the developer makes and sells, and that the user users, and that is also a 'what'.
2006.Feb.16 Thu"Testing is like tasting your food while you are cooking - it's part of the fun of cooking - and if you don't taste it as you go along, it will probably taste like crap at the end." -- Kevin Lawrence, in a comment to a blog entry by Ashish Kumar.
One of the other comments said that "junit is great for testing extremely well designed code or extremely simple code, most [real world] code is neither". Of course, Test Driven Development isn't "testing". (See the article TDD Isn't Testing by Jeff Patton.) TDD is a way of developing code driven by one test at a time, and it (usually) turns out to be easier than the normal code-and-debug AND results in well-designed and simple code.
(If you do test-driven coding without refactoring then you will get poorly designed code, but refactoring is a required step in TDD. Not doing refactoring means you're not doing TDD.) Back to my main point.
Sometimes TDD is hard because the existing APIs you're trying to use are poorly-documented, or just hard to test. I had problems doing TDD with AppleEvent programming (on MacOS 9 in C++) for both reasons. This is worse than doing TDD on the "edge" of legacy code because legacy code can be (carefully) refactored, but system calls like the AppleEvent API are basically take-em-or-leave-em.
TDD is a new habit, and new habits can be hard to acquire. The rewards are worth it, in my opinion. One reward in particular is the nearly-complete elimination of stepping through the debugger, which can be so time-consuming. I can write a quick 3-line test asserting what I'm interested in, run ALL the programmer-tests, and get a pass or fail (with helpful output) in less time then it takes to run the software manually and break at the right place with the debugger. TDD doesn't prevent "missing-features" but it does help in finding and fixing bugs almost as soon as you write the buggy code. If a test fails unexpectedly during TDD, it's almost always in the last few lines of code you wrote -- you don't need to spend a lot of time in the debugging tracking down the defect.
Notice that I said "3-line test". A TDD test consists of (1) setting up the context, (2) exercising the method(s) of interest, and (3) asserting the expected outcome. If any of the steps is hard, this is an opportunity to improve the tests and the code (via refactoring) until it is easy. Of course, if a TDD test takes 5 lines, or 10, it's not necessarily an urgent task to refactor.
As for legacy code. See (google search).
2006.Feb.15 WedPascal Van Cauwenberghe has written a series of "Things I didn't learn" blog entries that relate experiences he had pre-XP with XP practices and principles. He has a slyly self-deprecating style that exposes a problem/solution with some element of surprise to it. There is conflict, mystery, adversity, and challenge in the situations he describes.
Check them out.
Part 1: The incredible shrinking program [Continuous Refactoring]
Part 2: How many points for a soldier? [Agile estimating, planning, and volunteering]
Part 3: Backup soldiers [Pair programming/soldiering]
Part 4: I'll take this task. I'll take that task. [self-organization, volunteering]
Part 5: One for the road [working just on the features that fit in the time-frame]
Part 6: Into the heart of darkness. And back again. [refactoring supported by automated testing
Part 7: The one without a bug [pair-programming/code-reviewing reducing bugs]
Part 8: Running on empty [sustainable pace]
2006.Feb.13 Mon(corrected URL) I wrote this article on Adopting XP (PDF) when Extreme Programming was mostly unknown by the mainstream, and widely feared to be more like bungie-jumping rather than a serious and effective software development method. It was originally published in the July/August 2002 issue of STQE magazine (now Better Software magazine).
2006.Feb.04 SatThere is a rather long conversation about Agile QA here: http://www.theserverside.com/news/thread.tss?thread_id=38785, which started in response to to a speech by Scott Ambler. There are various digressions and arguments, but a few snippets I want to repeat here:
Ranjiva Prasad wrote:
The software industry should aim for zero defects, through improved tooling and training. This is an ideal but as the Japanese philosophy of Kaizen (continuous improvement) says, if you don't start to take small steps towards an ideal, you'll never improve.Cameron Purdy wrote:
You can't have real quality without a healthy lust for kaizen, and you can't have it outside of an environment in which all participants feel individually and corporately responsible for it. Thank you for putting it so succinctly.Dave Rooney wrote:
I personally believe that there are things that automated tests can't test well - aesthetics, ease of use, proper correlation to the business work flow, and, yes, exploratory testing. After all, there are things that testers do that normal humans would never dream of! :)My response to Dave Rooney -- without the automated tests keeping the developers on their toes (and other practices like Pair Programming or Code Reviews), the testers will have to waste their time finding bugs that are NOT related to aesthetics, ease of use, etc. They will instead spend all their time finding bugs that the programmers and automated testing could have caught.
In response to Ambler's contention that agile techniques can reduce "QA" staffing down to 10% of their original numbers, I think many companies are already understaffed in their testing departments. The benefits of agile methods will let the testers become a more strategic and proactive part of the process, and less of a reactive part. This does involve different skills for testers -- just like agile development requires programmers to master skills like refactoring, TDD, incremental design and automated testing.
2006.Jan.27 FriOO - the Solution to the Problem of Switch Statements
Dan Ingalls, principle designer of Smalltalk, with Alan Kay at Xerox Parc. This video was sponsored by Apple, recorded in 1989. In introducing OO (and Smalltalk in particular), he describes the main failing of non-object oriented languages in writing complex software: the switch statement. The switch statement has all sorts of problems with cohesion and coupling. The irony today is that almost all OO languages today, other than Python and Smalltalk (in which it can easily be implemented), still have switch statements.
In most cases, switch statements can be eliminated by either using the Replace Conditional with Polymorphism refactoring, or by using a Map or Dictionary to hold keys and objects (the objects could be blocks of code, "pointers to methods", or regular objects).
On another subject: a recent (still in Beta) full-fledged Smalltalk implementation for MacOS X, which allows Cocoa programming: Ambrai Smalltalk. Screenshots. Cocoa Tutorial 2006.Jan.23 MonI recently came across a blog entry celebrating the 220th anniversary of religious freedom in Virginia, passed into law on January 16, 1786. I can't find that blog entry again, but here's some of the related information:
http://usinfo.state.gov/usa/infousa/facts/democrac/42.htm :
In Virginia, the American Revolution led to the disestablishment of the Anglican Church, which had been tied closely to the royal government. Then the question arose as to whether the new state should continue to impose taxes to be used for the support of all recognized churches. [many] still believed that religion should be supported by the public purse.For some Virginians, however, imposing religion on people smacked of tyranny. Thomas Jefferson and James Madison, ... argued that religious beliefs should be solely matters of individual conscience and completely immune from any interference by the state. Moreover, religious activity of any sort should be wholly voluntary. Not only did they oppose taxing people to support an established church, but they also objected to forcing people to pay taxes even for their own church. To Jefferson, a high wall of separation should always keep church and state apart.
Jefferson drafted the ... measure, but it was Madison who skillfully secured its adoption by the Virginia legislature in 1786. It is still part of modern Virginia's constitution, and it has not only been copied by other states but was also the basis for the Religion Clauses in the Constitution's Bill of Rights. ...
http://religiousfreedom.lib.virginia.edu/sacred/vaact.html: ... Be it therefore enacted by the General Assembly, That no man shall be compelled to frequent or support any religious worship, place, or ministry whatsoever, nor shall be enforced, restrained, molested, or burdened in his body or goods, nor shall otherwise suffer on account of his religious opinions or belief; but that all men shall be free to profess, and by argument to maintain, their opinions in matters of religion, and that the same shall in nowise diminish, enlarge, or affect their civil capacities
http://religiousfreedom.lib.virginia.edu/sacred/madison_m&r_1785.html
2006.Jan.22 SunWe the ... citizens ... having taken into serious consideration, a Bill ... entitled "A Bill establishing a provision for Teachers of the Christian Religion," and conceiving that the same ... will be a dangerous abuse of power, are bound as faithful members of a free State to remonstrate against it, and to declare the reasons by which we are determined...
... The preservation of a free Government requires not merely, that the metes and bounds which separate each department of power be invariably maintained; but more especially that neither of them be suffered to overleap the great Barrier which defends the rights of the people. The Rulers who are guilty of such an encroachment, exceed the commission from which they derive their authority, and are Tyrants.
... Because it will destroy that moderation and harmony which the forbearance of our laws to intermeddle with Religion has produced among its several sects. Torrents of blood have been split in the old world, by vain attempts of the secular arm, to extinguish Religious disscord, by proscribing all difference in Religious opinion. Time has at length revealed the true remedy. ... The very appearance of the Bill has transformed "that Christian forbearance, love and chairty," which of late mutually prevailed, into animosities and jeolousies, which may not soon be appeased. What mischiefs may not be dreaded, should this enemy to the public quiet be armed with the force of a law?
Pascal Van Cauwenberghe compares the Gerald M. Weinberg "It's always a people problem" approach to the Toyata Way "It's always a process problem" approach to the problem of "Mike the micromanager."
I believe this resolves the conflict: every problem is a people problem that must be solved now AND an underlying process problem that must be solved, so that this problem doesn't reoccur in the future. That means we have to clarify our goal: "To have an effective organisation, now and in the future".Video Neal Stephenson speech and Q and A at the Library of Congress website.
Video Ben Bova speech and Q and A at the Library of Congress website.
Refactoring as Decluttering
Refactoring as Gemba
2006.Jan.06 FriI think Martin Fowler's "Implicit Interface Implementation" is just what Java, C#, and C++ need -- a way to get polymorphism without inheritance, when there isn't already an explicitly-declared capital-I Interface. Ironically, there is an explicitly declared small-i interface declared - the class interface. Quote:
... One thing that neither Java nor C# allow you to do is to implement an implicit interface - that is you cannot write class ValuedCustomer implements Customer
What would it mean to implement an implicit interface? Essentially it would tell the type system that the ValuedCustomer class implements all the methods declared in the public interface of Customer but does not take any of its implementation, that is its public method bodies, and non public methods or data. In other words we have interface-inheritance but not implementation-inheritance.
When would this be useful? One case I remember ... We wanted to replace the Vector class with an implementation of our own, but couldn't because Vector was a class and we could only subclass it.
... This particularly comes up these days with testing. There are lots of times when you want to stub out stuff, but it's difficult or impossible unless you have an interface. It also leads to defining pure interface types when the only reason to do it is to support substitution for testing...
I predict that if there is enough discussion of this in the blogosphere, Microsoft will add it into C#; Sun would add it to Java only if Microsoft adds it to C#; C++ will never have it.
Somewhat related to this is one of Robert Martin's Principles of Object-Oriented Design: "The Interface Segregation Principle", which says "Clients should not be forced to depend on methods that they do not use. Interfaces belong to client, not to hierarchies."
In a dynamic system like Smalltalk, Ruby, or Python, we don't need explicit interfaces, so "The Interface Segregation Principle" doesn't seem to apply. In static systems like Java and C++, one could declare interfaces specific to each client, but it seems that few people do.
The principle seems to more about dependency-breaking than anything else. I'd be interested in hearing from people who have actually done this extensively.
By the way, Objective-C, the language used for Cocoa programming on MacOS X, has three ways of dealing with interfaces. One is the 'duck typing' approach - no inheritance, no explicitly defined interfaces, no compile-time checking. Example:
@interface Duck : NSObject -(id) init; -(void) dealloc; // called indirectly through release. -(void) quack; // the class method 'alloc' is inherited from NSObject. @end @implementation Duck //etc... @end //class DuckCall declared similarly. id duck = [[Duck alloc] init]; [duck quack]; [duck release]; duck = [[DuckCall alloc] init]; [duck quack]; [duck release];The next way is with an explicit interface, called a protocol. A class declared to conform to a protocol must implement all the methods in that protocol. (And additional syntax to allow distributed objects can optionally specify that method parameters are 'in', 'out', 'inout', 'bycopy', or 'byref'.) You compile-time checking with protocols.
@protocol Quackable -(void) quack; @end @interface Duck : NSObject < Quackable > -(id) init; -(void) dealloc; // called indirectly through release. -(void) quack; // the class method 'alloc' is inherited from NSObject. @end @implementation Duck //etc... @end // class duck-call declared similarly id < Quackable > duck = [[Duck alloc] init]; // The compiler warns us [duck quack]; // if Duck isn't Quackable. // you can also confirm at run-time that an object implements a protocol: assert([duck conformsToProtocol:@protocol(Quackable)]); [duck release]; duck = [[DuckCall alloc] init]; [duck quack]; [duck release];The third way to declare an interface, where implementing its methods is optional, is via a 'category'. This is often used in Cocoa where a user-interface object has an optional delegate. The delegate may implement one or more methods that the user-interface object will call (if the method exists) at particular times. This avoid having to subclass user-interface objects. It's similar to implementing a "Listener" class for the Java Swing framework.
@interface NSObject ( Quackable ) -(void) quack; @end // Duck and DuckCall classes are declared like the first example. NSObject* duck = [[Duck alloc] init]; if ( [duck respondsToSelector:@selector(quack)] ) { [duck quack]; } [duck release]; duck = [[DuckCall alloc] init]; if ( [duck respondsToSelector:@selector(quack)] ) { [duck quack]; } [duck release];The value of informal protocols in Objective-C is that the method-argument-types and returns are type-checked by the compiler. So if a "fly:to:" method took arguments of double and NSPoint, the compiler could detect if you used the wrong types in making that call -- and since neither double nor NSPoint are object types, it's essential that the types be correct to avoid a crash from passing parameters of the wrong size.
While it would be nice to have Objective-C be able to declare that a class has the same interface as an existing class without having to make a separate protocol declaration, the "duck-typing" built into the language lets us use implicit interfaces and informal interfaces fairly easily, with some compile-time checking (for argument and return types) and the same run-time checking that Ruby and Smalltalk have.
2006.Jan.03 TueIn his blog, Sam Gentile wrote "To quote Keith Ray, 'The best protection against change is making change easy.'"
It's a great quote, but I don't remember writing that!
Actually Google found that Greg Akins wrote that in a comment here. Maybe I said something like that in a mailing list. "Google books" didn't find that exact phrase either.
