of undesirable attributes. This categorization prevents any single measure from having a distorting influence on development excellence.
Quality metrics provide an indication of defects that escape development as well as an indication of functional compliance to specification. This underscores a particular strength of Agile processes: iteratively executed story-based QA provides feedback on technical and functional compliance with greater frequency than other methodologies. As a result, quality problems are exposed much earlier in a project, reducing the probability of substantial re-work or compliance blowback as the target delivery date draws near.
Business responsiveness measures make business and IT alignment highly visible. Requirements in agile projects are captured as stories, and stories are an expression of business value. Thus, the team can index the degree of business value targeted and business value delivered. Variance in this measure suggests an alignment problem between business and IT.
It is also important to rate project management . In waterfall or ad-hoc projects, project management assessment is Boolean (e.g., was the release date met or not?) as opposed to being a continuous assurance of delivery.While variance of delivery phases (including development, integration and certification) can be analyzed on an agile project, story-based requirements and iterative planning and reporting uniquely allow agile teams to provide accurate forecasting. Successfully executed, this creates a high degree of transparency throughout the delivery process. On agile projects, the question changes from "was the date made or not?" to "what was the accuracy of goal-setting?" This exposes potential deficiencies in requirements gathering, estimating, and planning.
The metrics scorecard consolidates performance data and shows an overall rating of delivery, a rating by category and details within each.Because each metric will be reported in a different unit of measure (time, money, scores, percentages, etc.) the data must be normalized into a common scale (such as -2 to +2) before it can be consolidated into a scorecard. To form the scale and individual rating, it is also necessary to define scoring thresholds. For example, unit test coverage above 80 percent might be considered an excellent result. Therefore, the threshold for a score of "2" would be a score at or above 80 percent coverage, a "1" would be awarded for code coverage between 60 percent and 80 percent, "0" would be for 40 percent to 60 percent code coverage, and so forth. A project reporting 75 percent code coverage would score "1" on this scale. In the same way, simple heuristics can be defined for each individual measure, and aggregate scores calculated.
The resulting scorecard will paint a complete picture of a project, program or department performance relative to organizational or industry baselines.
In the early stages of a metrics campaign not all of this data will be available, either because there is no history or because other development methodologies don’t allow for this type of analysis. The absence of the baseline does not negate the value of collection: over time a baseline will emerge by virtue of collecting this data.
The resulting scorecard provides a thorough analysis of the internal business processes of a software development organization. From this analysis, hotspots as well as an overall picture of business/IT alignment become highly visible.
About the Author
Ross J. Pettit has over fifteen years experience delivering complex development projects and managing multi-national operations as a developer, manager, and consultant. He holds a BS in Management Information Systems and an MBA. He is currently consulting to global clients implementing agile practices as a Client Principal with ThoughtWorks.