Larry is a writer just finishing his first book. In a moment of inspiration, he decides to switch the order of two middle chapters. Using his word processor, his tool of choice, Larry grabs one chapter and drags it beyond the next. Suddenly his computer freezes, and Larry can't get it to thaw. He has to reboot it, and then re-open his work. To his horror, he discovers that his precious book has been mangled beyond recognition.
What happened to Larry can be traced back to when his word processor was designed. The designers didn't consider the Larry's of the world-that is, book writers-to be important users of the word processor. They put some functionality in it to support moving text, but not chapters, and the code which handled those transactions was very fragile. Unfortunately for Larry, the designers' concept of who the end users would be leaves his word processor prone to file corruption when manipulating large chunks of text.
We often fail to properly consider all of the users during design. Is this something you might be doing? How can you tell? Why should you care?
You may be designing the wrong product! Our different views about users are a major source of design risk.
Managing Design Risk
Before looking at user modeling, take a look at design risk itself. While risk comes in many forms, two important design risks are:
When managed properly, these two risks are effectively balanced against one another. Too much effort on product definition increases the risk of late product delivery, while too much emphasis on schedules increases the risk of delivering the wrong product.
Two important user constituencies in any design activity are the designerand the client.
Designer transform the design requirements (problem statement or context) into the final product (problem solution or form). The designer's efforts are devoted primarily to resolving the "how" issues.
Client Clients control the design process by allocating resources. The client hires and fires the designers and has final say in time and budget constraints. The client has ultimate responsibility, as exercised by funding control, over the "what" issues.
Figure 1 depicts a new set of design risks as related to client expectations. This is a visualization of what Perry and Rice in Surviving the Challenges of Software Testing refer to as the "expectation gap." The client bases her expectations on what she knows-the top row of the diagram. The designer bases the design on what he knows-the left-hand column. Each of the four quadrants leads to a qualitatively different form of risk.
Nature's Last Laugh This is the area in which neither of the key players in the design process has access to or knowledge of critical information. Nature is hiding her cards from us. State-of-the-art projects are laden with this risk, as much of the relevant information has yet to be developed and may, in fact, not reveal itself until the final product is put into service.
Lost Opportunity The opportunity is lost because the designer did not obtain relevant information from the client. This is a major source of disappointment to the client in the delivered product, as it forms part of the client's expectations.
Surprise The client is surprised by features and functions that are introduced in the product based on information known to the designer but not to the client. Some of these surprises are pleasant, while others are not so pleasant. This area leads to unwanted system complexity. In The Mythical Man-Month, Brooks talks about this in his chapter "The Second-System Effect."
Bias We wouldn't expect to experience risk in this, the safest of all four areas. But, we should be aware that we, as human beings, are notorious for our misuse of information that we do know.
If we were to draw Figure 1 in a way that more accurately reflects the relative risk that we can expect to experience at the beginning of a design project, it is more likely to look like Figure 2. We can see that our predominant source of risk comes from "Nature's Last Laugh." We could expertly manage the design process, incorporating only information obtained from the client and designer, and very likely deliver a "client-successful" product.
History has shown us, however, that while a satisfied client is necessary, it's not a sufficient condition to produce a satisfactory design. The Pruitt-Igoe housing project in St. Louis is an architectural example. It was a high-rise, low-income housing project designed to replace aging row houses. It won design awards, but the buildings were literally dynamited out of existence just seventeen years later. They looked great in theory, but turned out to be unlivable in practice. Where did the designers go wrong? Information known only to the future tenants, firefighters, police officers walking the beat, delivery and service people was not factored into the design.
Had the clients and designers of Pruitt-Igoe modeled users, and then included those users in the design process, that development might well still exist today. Including the proper set of users in the design process is crucial to effectively managing design risk.
Construct a description of the important users of your product, then use that model to drive how you design and test it. People often think of the user as the direct end user of the product-the actual hands-on user. For example, a defect tracking system typically has users such as Testers, QA Managers, Developers, and Development Managers who use the system to help prioritize their work. But we have found that a broader definition is very useful:
User: any individual who is affected by or who may affect your product.
Our purpose in including such a broad range of potential constituents is to avoid the risk of omitting a critical constituency and thus depriving us of important information. Users are described by roles that people play as they interact with our system. The roles include not only hands-on users, but also those who are indirectly affected by the product. For example, senior executives may indirectly use a defect tracking system to decide when to ship a product, based in part on overall defect status. Customers are affected by the defect tracking system because they speak to support analysts who use it to help identify known problems.
What happens if we overlook lawyers as potential users when we design this system? We don't ordinarily think too much about showing our defect data to people who wouldn't access the system. But if we get into legal troubles, the data will almost certainly be seen by unfriendly lawyers. How would that affect our system security model? How about accessibility? What about the content? If we didn't consider lawyers as a critical user when we built our system, there's a good chance we may have to start over. They will likely use our own system as a weapon against us.
Here is a set of guidelines we recommend to model users. First, identify the client. If we don't clearly understand our relationship with our client, nothing else matters.
GUIDELINE ONE Have a clear and well-documented understanding of just exactly who the client is at the beginning of the design project. Don't start the project without it.
An often-overlooked source of design risk originates from ill-defined designer and client responsibilities. As we have stated, we hold the designer accountable for the creation of the end product. The client, in turn, is responsible for economic matters, which include the ultimate financial success of the product. Table 1 indicates the types of risk and potential results that we have observed in the development of software products.
Although the risks of Table 1 pervade the lifetime of the design project, they are the result of decisions made at the beginning of the project. It must be clearly understood that the client will be accountable for the general direction of the design project. As such, he will be expected to participate in all formal design reviews and follow well-defined sign-off procedures for critical design decisions and changes. This encourages greater diligence on the part of the client, as the client knows that he will be answerable to the user population served. In our experience, many commercial software development projects proceed along, some to their demise, without a clear and well-understood agreement defining the client, or-even more commonly-without an assignment of a specific individual with clear client responsibilities. You need to know who the client is to know the client's expectations.
GUIDELINE TWO Spend a concerted effort before beginning the design process to define, to the best of your ability, the full set of user roles, i.e., people who will be affected in some way by the system about to be designed.
We have found brainstorming to be very effective in reaching for "all" of the possibilities. We form a small group of initial users, including the client, and run a one- to two-hour brainstorming session for the purpose of identifying all of the kinds of individuals who we believe will be affected. This is done in the best spirit of brainstorming, such as Adams describes in Conceptual Blockbusting, in which all judgment is suspended. We don't end our search for user roles until we have identified at least 200 users.
Now, you might be thinking, "That seems like a lot of users. Do we really need to go that far?" We remember a workshop where a participant objected to coming up with all those users during the brainstorming session. She complained that it was boring, and clearly thought it impossible that any more users could make any difference. Before we could even open our mouths to respond, another workshop participant interrupted to say, "Oh, I wish we had done this exercise on my last project! It would have saved us most of the trouble we're having today. We forgot the system administrator as a user, and we've had to buy a whole suite of tools to support the activities we should have designed in easily in the first place. What a headache." We apply guideline three as an additional test for omissions.
GUIDELINE THREE Once at least 200 user roles have been identified, make sure that at least three can be associated with each of the following categories: 1. clients, 2. designers, 3. direct users, 4. secondary users, 5. tertiary users, 6. frivolous users.
We have defined clients and designers. By "direct users," we mean the people who are making direct contact with the system to be designed. "Secondary users" are the people who come into contact with the products or by-products of the system to be designed. They may provide information to the system or receive information from the system. "Tertiary users" are individuals who will never see or use the system but who have relevant information pertaining to its design. "Frivolous users" are, as the term implies, those individuals who have been mentioned as part of the brainstorming process but in a humorous or frivolous vein.
When we get to this point, we are bound to have, almost immediately, discovered many new features and functions suggested by this disparate set of users. We have also identified a number of serious but previously invisible users. This information opens up many opportunities but is not without risk.
The following guidelines are put into practice to mitigate the risks identified in Table 2. Once we have a super set of users, we need to pare them down to a smaller set which we will actively consider when building and testing our product.
GUIDELINE FOUR Designate each of the user roles that remain, after removing the frivolous roles, to one of three statuses: favored, disfavored, and ignored.
Favored Users We will specifically build features to benefit these users.
Disfavored Users We will deliberately build features to cause the disfavored user to experience inconvenience and "difficulty of use." This may seem strange, at first glance, but there are perfectly good reasons to want to make a system difficult to use for almost any constituency. The reasons may be associated with security, legal, competitive, or safety issues. For example, most security precautions explicitly disfavor unauthorized people regarding system access.
Ignored Users We will not overtly build any features to specifically benefit these users.
All resources will be directed at developing functionality for the favored and disfavored users, while the ignored users may or may not find useful functionality. This is not to say that the ignored users will not benefit in some way by the new system; but the benefits will accrue as a result of meeting the needs identified by the favored and disfavored users.
GUIDELINE FIVE Pare Down the List of Critical Users.
The users whom we decide not to ignore are our critical users. We have found that it is useful to design for no more than fifteen to twenty-five critical user roles. As the number grows beyond this point, design thinking becomes too diffuse.
Eliminating the majority of users causes conflict early in the design process, but we contend that this is where we want the conflict. If we don't resolve this conflict at this point, it will either be resolved at a more expensive point, when we have to reverse design decisions, or the conflict will be designed into the product in the form of inconsistencies of form or function.
We choose the critical users by deciding, as a design team, which ones we're going to keep and which ones we're going to eliminate. There are any number of ways to choose the user set, none of which is especially easy. We will essentially be deciding based on what we care about and believe is important to our product. One common method of choosing users is for the design team to vote for the set of critical users. For example, each team member may vote for seven users. After seeing if there's some convergence on an initial set of critical users, a negotiation begins to determine the status of the marginal ones.
Once this assignment has been made and approved by all interested parties, the assignment will, by default, direct design thinking for the remainder of the design process. The assignment can be changed, but not without reason and not without approval of all interested parties.
Applying the User Model
Include Users in the Design Process Now that we have decided which users will receive our top attention, we must put teeth into our decisions. We do this by committing ourselves to an agreed-upon User Inclusion Plan. This plan states the "who, how, and when" for each of the critical users. Here are some possible choices for including users in the design process.
Surrogate The user is represented by an individual or group of individuals who will look out for that particular constituency's best interests. This representative may not have had direct experience as a user, but is expected to know as much as possible about the needs of the user. The surrogate user may be a member of the design team. This person is held personally accountable for the success of the design with respect to the user she represents.
Sample This is a representative sample of actual users.
Population In the case of a relatively small population, we can choose to elicit all of the users.
Judgment We ask for opinion or judgment of the user.
Survey User information is obtained in the form of a survey.
Simulation User information is obtained by noting user response to a simulation (computer or physical) of parts or all of the target system.
Prototype We observe the users as they experience parts or all of the early system prototype.
One Time This constituency is polled one time only.
Critical Times This constituency is polled at designated times. These times, agreed upon in advance as part of the project schedule, would minimally occur whenever formal design reviews take place, whenever critical design decisions are about to take place, or whenever requirements or design responsibility are transferred from one group to another.
Periodic This constituency is elicited at regularly scheduled calendar intervals.
Continuous Some constituencies may be considered to be so critical that they will be represented continuously throughout the design process.
Once we have a User Inclusion Plan, we need to apply it. There's no substitute for including the users in the design process.
Overlooking an important user is a very expensive mistake. It can cause us to spend much of our time later trying to force our design to accommodate that user. If our product's architecture wasn't built with that user in mind, it is unlikely that we will serve that user well. Ever.
We've presented simple, easy-to-use heuristics to lower the probability that you may overlook important users, thereby lowering your design risk. Remember that it doesn't matter when you deliver, if you build the wrong product.