Doing Inventory

[article]
Exposing rogue features and reducing bloat

During testing we use requirements to guide our activities. We check to see that the program is doing what it is supposed to be doing. But how often do we stop and wonder how much more it is doing? Why does the install process take longer and longer every week? Why is build 210 three times larger than build 209? Why does issue 3197 reference a component of the product I've never even heard of?

Checking requirements doesn't answer these questions. Instead we need a way to work backwards from what we have, to what we thought we should have. This is the realm of asset management, and our part of this is asset reconciliation. Here, I provide a simple version that helps remove some of the unknown variables, or the wildcards of product quality, and also prevents unnecessary product bloat.

There is no magic in this technique--it is a simple four-step process: 1) locate all the assets, or parts, of the application package; 2) identify what each part could be used for, or what its immediate purpose is; 3) determine if this part is actually needed by the application; and finally 4) ensure these parts are being tested.

Remember, the goal is not merely to catalog assets. The goal is to track down and eliminate unneeded bloat and make sure all remaining assets have been properly tested. The accounting need only be as rigorous as your project necessitates.

Step 1: Locate the assets
Your first step is to locate all of the parts of the application. Note that you are interested primarily in the final version--what comes out of a ZIP file or an installation package. You want to identify initially all of the directories and files that the product comprises. This is a simple starting point that consists of a few "dir" commands at a shell prompt.

But don't stop there. Often, additional assets are hiding inside compressed, or archive files (some of the usual suspects end with ZIP, JAR, TAR, and CAB). You should also understand the manner in which UI resources are stored. Something as simple as a single DLL file can contain numerous dialogs, hundreds of images, strings, and other nondescript data items. Be sure to track down all the shared libraries the install program put in or the directories. And never forget about the final tidbits stored in the registry or placed in a file in the directory.

Now that you have a big list of assets you are ready to move on, but don't be afraid to come back to this list and add more items or further detail the contents of a file you later learn is an archive, such as ZIP. Also don't be afraid to move on before exploring every compressed file or configuration item. This can be treated as an iterative process: with each pass, you can further explore one area of the assets, or go one detail level deeper in the analysis.

Step 2: Identify their purpose
For each item you have in your list you need to determine what it is. Is this an image file? Is this a library component? Is this a help reference?

Focus on identifying the general purpose of the item, rather than obtaining a detailed description of its use. You are interested, at this stage, in understanding what each of the items could be used for, where it could be used, or how it could be referenced. Don't yet concern yourself with whether the item is actually necessary for the program to operate--that is the next step.

If there are some items on your list that you really can't identify, be sure to ask other members of the development team. Unidentified items will prove troublesome to further analyze. If nobody is able to identify what the item is, then you know you are dealing with a suspect asset.

Step 3: Determine if they are needed
Once you've identified each of the items, you then need to determine if they are actually needed by the product. A large part of this step is already completed simply by knowing the identified purpose of the items. It is nonetheless an important and longer step, with a distinct purpose from the previous one. This is when you determine the concrete use for each item on your list, determining when, where, and how each is used.

You want to know where each dialog appears in the program. Images may be used in the default interface, or perhaps they are selectable by means of a user-selectable skin. Check each shared library to see whether it is needed, or whether it is a remnant from a previous version. Review each configuration setting in the registry, or configuration files, to ensure that it is applicable to the product (configuration bloat can be just as much of a problem as file bloat). Be watchful of items that look like they might be used but simply aren't (for example an extra JPG in the directory which is actually never referenced).

The information you collect needs to be enough to determine when, where, and how the item is used. "When" ensures that you've considered all the variables that may impact the use of the item. It is quite possible that certain items, such as language resources or device drivers, are used only in certain configurations. "Where" and "how" will give you the insight you need to test the item. They can also further identify those cases where an item is used, but only in another component that is not used (perhaps an icon that is displayed only in a dialog that is no longer referenced).

Beyond merely determining that the item is used by the program, you need to determine if it is supposed to be used by the program. You should be able to state which requirement describes, or requires, each of the items on your list. Often, requirements are quite strict in what they state, other times they are open. Sometimes a directory filled with selectable images will have no specifics, but other times the requirement may specifically state only three skins that can be chosen. Your work here may be of interest to the requirements management team as they strive to improve their system: don't leave them out of the loop!

At this point, there are three situations that will result in change requests for the product. First, if an item is both not used by the product and not described by the requirements, you will likely want to create a deletion request for this item. Second, if an item is used by the program but not included in the requirements, you will need to speak with the team to determine whether the item should be added to the requirements, or removed from the program. Third, if an item is described by the requirements but not used by the program, in this case you need to determine whether the program is to be adjusted to use the item, or whether the item and the requirement for the item both need to be dropped.

Step 4: Ensure they are tested
This last step seems obvious at first, but it plays an important role when you have fairly open requirements. When a requirement allows many items to be added to a product, normal testing of the requirement may only result in one of these items being tested. This step is where you formalize the inclusion of all of these items into the testing process.

A common area in this regard is that of examples, or samples. Typically, the requirements don't specify every example that will be shipped with the program, and the examples end up being a collection of those created for testing requirements, debugging the application, or showing off the GUI. Here it is important that you really want to make sure all the examples demonstrate correctly what they are supposed to, and that each example meets the quality standards of your product.

Conclusion
It is clear that the first few iterations of this process will likely be the most difficult; it is a daunting task to go from tracking zero assets to the thousands, or even tens of thousands of assets in a reasonably sized project. Don't be aggressive in your first attempts; instead, concentrate on tailoring the technique to your needs and creating the tools you will need to track this work. With each iteration, you can improve your tools and the level of detail you track. It should be clear that a simple editor with one EXE, a few DLLs, and some examples is not going to require the same effort or tools as a modern real-time strategy game with some 40,000 image files. In the latter case, you are advised to try to integrate this technique into your standard asset management process (late reconciliation may be too costly for your project).

The simplest form of this process does not require any other changes in development or testing. You can implement this over several weeks, or months, and not worry that it might interfere with your existing process. Once you've started, the confidence you have in your product should improve. You can feel more secure that there aren't any hidden aspects of the program left untested, or that some spurious library isn't going to cause the program to crash. Ultimately, this gives one more little boost to product quality.

(Note: This process does not replace asset management, rather it is best seen as the quality control process for asset management. Use as much information as possible from your asset management system–it may reduce the workload of this activity.)

About the author

AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.