When you think about a software quality Assurance (SQA) function in your organization, what's the first thing that comes to mind? Organizational saviors? Bane of our existence? Pain in the patootie? Ignorant nitpickers with no business sense? Source of all knowledge as we know it today? You're not alone. Because SQA performs reviews and audits, it may be seen as a direct threat to Engineering. Rather than operating as partners, SQA and Engineering can become adversaries. SQA should function more like Jiminy Cricket. Remember the story of Pinocchio, the little puppet who wanted to become a boy? He had a creature who sat on his shoulder and reminded him when, in his heart of hearts, he wasn't doing what he knew he should be doing. Jiminy Cricket was the physical embodiment of Pinocchio’s conscience. And that's what SQA should be to an organization—its conscience.
Who Is Responsible for Quality?
Who is responsible for quality, for assuring that products meet established standards and that the process is followed to create those products? Is quality everyone's responsibility? This typical assertion actually creates an interesting paradox of human behavior in organizations. When quality is declared to be everyone's responsibility, no one is truly designated to be responsible for it, and quality issues fade into the chaos of the crisis du jour.
Is quality SQA's responsibility? Not necessarily. A conscience is not responsible for proper behavior. It's merely a fair reminder. (Let your conscience be your guide.) Let me illustrate with a personal story.
Years ago, I was brought in as the new SQA "hired gun" to help straighten out a contract that had been operating for several years and was in big trouble. I started by poking around the declared project standards and artifacts. I noticed that this organization had a wonderful standard for doing Software Development Notebooks (SDNs). The SDN was a loose-leaf binder, one for each software unit, created by the original software developer, then transitioned to the software maintainer to serve as an ongoing record of the life of the applicable software unit.
This project had been running for years, and there were hundreds of these SDNs in the project's library, each with the applicable software unit's name recorded on the spine. I decided that a good way for me to get an idea of the software status would be to audit the SDN library. I was shocked to discover that over 90% of the SDNs were empty! I wrote a memo that described the findings of my audit.
Remarkably soon, I was called into one of those luxurious mahogany conference rooms with the huge rectangular table and plush chairs. There sat the Software Development Manager, along with most of the pertinent managers on the contract. The Software Development Manager had my memo in front of him. He slid it across the table to me and declared, "You can’t write this!"
I replied, "Well, the information is true, and if the client's representatives should perform the same audit, it could be yet another embarrassment for us."
The manager folded his arms over his chest, rocked back in his chair and said, "We have a deal with the client regarding our library: They can only look at quantity, not at quality."
That was why the empty SDNs were on the shelves. They could only be counted, not opened!
Having an SQA function doesn't assure anything. The workforce will do what management recognizes and rewards. Management didn’t really care whether the SDNs were done, so few programmers actually did them. Having me there didn't really make a difference.
So what is SQA's role? The Software Capability Maturity Model (SW CMM) version 1.1 states: "The purpose of Software Quality Assurance is to provide management with appropriate visibility into the process being used by the software project and of the products being built."
Notice that this purpose definition does not include "test" or "enforce" or "establish quality standards." Yet in many organizations, SQA is viewed as the dreaded "Compliance Police"—those jackbooted thugs that march through the project, put their thumbs on the backs of the developers' necks, and make them do things they hate to do.
When I see this type of adversarial relationship, I usually find that the organization hasn’t internalized the need to do disciplined process stuff (we don't need no stinking process!), and they fool themselves into believing that by implementing an SQA function, somehow the process stuff will happen.
Let's look back at the purpose paragraph quoted from the SW CMM. Notice that it provides "management with ...visibility." So the presumption for an effectively operating SQA function is that management is interested in how the work is being performed. Because management is interested, and they don't have the time to look themselves, they rely on an SQA function to fill them in on how the work is performed. This is much the same way that management relies on an accounting function for cost data.
So who is responsible for quality? Management! And when management truly commits to a quality culture, all of a sudden everyone will, indeed, be responsible for quality. Then SQA can function as it was intended to do.
What Can Noncompliance Teach Us?
Even when management does dance to the tune they are whistling, things don't always go as expected. Otherwise, it would make no sense to have an SQA function monitor it. Why might the expected standards, procedures, or processes not be used? The least encountered reason is that some practitioner simply refuses to follow the standard or procedure. If management is truly interested in the attainment of procedural or product standards, it would be career suicide to deliberately conflict with management desires. Most of the time when noncompliance occurs, there is some process reason for it:
To illustrate, imagine that part of your development process involves unit testing by the programmer—white-box testing before the code is delivered to integration testing. The SQA function, in examining a portion of a project, finds that some folks have skipped unit testing. The noncompliance is documented and the manager or supervisor closest to the problem is apprised of the situation.
There are three ways to resolve (close out) the noncompliance:
COMPLY. Have the developers go back and perform the missing unit tests.
ADJUST. What if the process was initially documented for large projects of 20 or more developers, and it is discovered that on a smaller (1 or 2 developer) project, later stages of testing cover the elements of unit testing? Ah! Learning has occurred! The organization should augment its process guidance to allow for this context.
WAIVE. Sometimes the customer wants the product really, really bad—so they get it really, really bad! What if the developer skipped unit testing because the customer demanded that the product be shipped without (adequate) testing? You are allowed to waive your process, as long as you document the waiver. But if you waive it more than you do it, I would question whether it’s really your process.
At the end of the project, when you perform a post-project review, you should look at the waivers and see if they made any difference. I would hope that if you waived all testing, you (or your customer) would find that it wasn’t such a good idea after all. But you may find that waiving a process, or process step, didn't seem to make much difference in the end. Time to re-examine the process to see if a contextual or universal modification is in order. To quote Frank Zappa, "Without deviation, progress is not possible."
Should SQA Be Independent?
A conscience must be free to speak the truth. Does this mean that it has to be independent? There is a subpractice in the SQA Key Process Area of the SW CMM that says that SQA should have "a reporting channel to senior management that is independent of the project manager, the project's software engineering group, and the other software-related groups." Does SQA have to be a completely independent function to satisfy the SW CMM? No. The issue at the SQA goal level is one of objectivity. SQA needs to be able to safely tell the truth—to tell the emperor he has on no clothes!
One of the easiest ways to demonstrate objectivity is to have an SQA function that is not beholden to the project for his/her performance review, i.e., to make the function independent. But this creates what some organizations feel is unnecessary overhead. SQA is one of the most flexibly interpreted concepts in the SW CMM. I have seen it implemented successfully a number of different ways that don't require a substantial investment in a functionally independent segment of the organization.
For instance, one organization I know of implements the "buddy system" for all tasks. Each assigned task has at least two individuals involved—one who has the primary responsibility for completing the task, and someone else who has the responsibility for ensuring that the applicable standards and procedures are followed. A part-time management function spot-checks to make sure that collusion ("Esther, I know you think this standard is a bunch of crap and I agree. So you tell them I did it and I'll tell them you did it. Okay?") doesn't occur. Actually, subversive collusion is quite rare, but time pressures can create collusive shortcuts.
every project manager’s time (4 hours per week) to serve as the SQA function on another project. In this case, a middle manager spot-checks to ensure that no collusion is occurring. (Remember, management is supposed to be interested in how the work is being done.)
Several organizations use Peer Reviews to implement almost all of their SQA function. A staff position at or above the project management level serves as the subject-matter expert for Peer Reviews and monitors them for fidelity.
I have seen a couple of presentations declaring a "person-less" SQA function in which the process itself has built-in checks and balances to ensure that processes and standards are followed as the product moves through the development life cycle. While those who utilize this technique vociferously defend it, I would caution time/schedule-stressed organizations against trying this as a first implementation of SQA. I have visited organizations that try to perform a selfchecking process in "pressure cooker" environments, and have found that the time pressures were inevitably forcing the smart practitioners to work together in implementing shortcuts in order to meet aggressive deadlines. Many of the shortcuts involved skipping steps that later increased project and product risk. The end result is that the process the organization says it is following is not actually what is happening. But without a human-based conscience, it is so easy to either fool oneself into believing or lull oneself into complacency.
What’s the Bottom Line?
Procedures and standards cannot always be religiously followed. Life is just not that perfect. Watts Humphrey is one of the prime movers behind establishing process rigor. But he reminds us that "rigor" is in the dictionary between "rigmarole" and "rigor mortis." Rigor, despite its connotations, walks the boundary between these states. However, you should be consciously aware of how the work is being performed—when it's done to expectations and when it's not. An improving organization is a learning organization. SQA can serve as a vehicle for that learning, but only if it is free to operate as the organizational conscience.