Checklists – You build me up just to knock me down


it down.  Do this for a whole week.

You'll discover a few things during this experience.  First of all, it's infuriating.  You make a lot of mistakes and they're tiresome to chronicle.  Second, you can't bear to do this for a whole week.  It's maybe a little too much introspection.  The ego can take only so much self-examination.
Third, and most importantly, you'll find a pattern.  You make the same kind of mistakes over and over - it's not random.  In my case, for example, I always misspell the word "definitely," and I apparently use it all the time.  And I close all of QuickBooks when I mean to close just one window - the non-standard design of the close-box was confusing me subconsciously.

Of course the next step is to correct the common problems.  Many of them you'll correct
just by virtue of being conscious of them.  In the case of QuickBooks, I freeze just as I am about to close everything - just long enough to remember the right way and avoid the 90-second restart cycle.  Others errors you can correct with automated methods.  With my
misspellings I just use the auto-correct feature in Outlook and now "defenatly" gets transformed automatically.

Software checklists should be built in exactly the same way.  Take a look at the last one or two hundred defects found by code review and you'll find a pattern.  Or look at the code fixes for defects found in QA and find the pattern in the root causes.  This isn't as hard as it sounds - you don't need detailed analysis or 5 columns of data on each defect - just summarize the code fixes in one line or less and the patterns will jump out at you.

A checklist derived from these patterns will be tuned specifically to the errors you make most often.  So the fifth rule is: Let empirical evidence determine the checklist .

This technique provides the initial checklist, but it turns out the patterns necessarily change over time.  With a short checklist, everyone gets used to keeping an eye out for those certain things.  Just as with the personal introspection project, everyone will get used to finding and fixing the same problems.  Soon code authors will think about these same things while the code is being written - the problem will be fixed before it starts.

At this point, the number of defects associated with that checklist item will diminish, eventually becoming so infrequent as to make that item useless.  This is a good thing - the developers have developed habits and techniques to avoid this type of error.  Now you can replace that item with the next common pattern.

This implies you're actually tracking how many defects are attributed to each item.  You're doing that, right?  Don't forget to have a "none of the above" category as well - when you find an item is stale, the list of defects in that category will reveal the next pattern that needs fixing.

The sixth rule is: Track defects by item so you can replace stale items .
Checklist for Checklists

So to summarize, here's a checklist you can use when
building your own checklist:

  • No more than 10 items in the list
  • No obvious stuff
  • Nothing that can be automated
  • Things that are easy to forget
  • Use empirical evidence to build the list
  • Replace stale items

As if a checklist for checklists wasn't recursive enough already, I'd like to point out that this list is short, contains items apparently not obvious to most checklist designers, and it was built empirically from patterns I've found in the field.

Now go roll your own!

About the author

Jason Cohen's picture Jason Cohen

Jason Cohen is the author of Best Kept Secrets of Peer Code Review, the definitive work on the new process of Lightweight peer review with 7,000 copies in circulation. Jason conducted and published the largest case study of peer code review at Cisco Systems®—2,500 reviews over 10 months with a real, delivered product. Jason is also the founder of Smart Bear Inc., makers of Code Collaborator that assists developers and managers with lightweight peer code review processes. Jason has spoken at many conferences, trade shows, and ACM/IEEE society meetings across the country.

AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!