people metrics

Conference Presentations

The Dangers of the Requirements Coverage Metric

When testing a system, one question that always arises is, “How much of the system have we tested?” Coverage is defined as the ratio of “what has been tested” to “what there is to test.” One of the basic coverage metrics is requirements coverage-measuring the percentage of the requirements that have been tested. Unfortunately, the requirements coverage metric comes with some serious difficulties: Requirements are difficult to count; they are ideas, not physical things, and come in different formats, sizes, and quality levels. In addition, making a complete count of “what there is to test” is impossible in today’s hyper-complex systems. The imprecision of this metric makes it unreliable or even undefined and unusable. What is a test manager to do?

Lee Copeland, Software Quality Engineering
The Metrics Minefield

In many organizations, management demands measurements to help assess the quality of software products and projects. Are those measurements backed by solid metrics? How do we make sure that our metrics are reliably measuring what they're supposed to? What skills do we need to do this job well? Measurement is the art and science of making reliable and significant observations. Michael Bolton describes some common problems and risks with software measurement, and what we can do to address them. Learn to think critically about numbers, what they appear to measure and how they can be distorted. Improve the quality of the information that we're gathering to understand the relationship between observation, measurement, and metrics. Evaluate your measurements by asking probing questions about their validity.

Michael Bolton, DevelopSense, Inc.
Performance Appraisals for Agile Teams

Traditional performance evaluations, which focus solely on individual performance, create a “chasm of disconnect” for agile team members. Because agile is all about team performance and trust, the typical HR performance evaluation system is not congruent with agile development. Based on his practical experience leading agile teams, Michael Hall explores how measurements drive behavior, why team measurement is important, what to measure, and what not to measure. Michael introduces tangible techniques for measuring agile team performance-end of sprint retrospectives, sprint and project report cards, peer reviews, and annual team performance reviews. To demonstrate what he’s describing, Michael uses role plays to contrast traditional, dysfunctional annual reviews with agile-focused performance reviews.

Michael Hall, WorldLink, Inc.
Quality Metrics for Testers: Evaluating Our Products, Evaluting Ourselves

As testers, we focus our efforts on measuring the quality of our organization's products. We count defects and list them by severity; we compute defect density; we examine the changes in those metrics over time for trends, and we chart customer satisfaction. While these are important, Lee Copeland suggests that to reach a higher level of testing maturity, we must apply similar measurements to ourselves. He suggests you count the number of defects in your own test cases and the length of time needed to find and fix them; compute test coverage-the measure of how much of the software you have actually exercised under test conditions-and determine Defect Removal Effectiveness-the ratio of the number of defects you actually found divided by the total number you should have found. These and other metrics will help you evaluate and then improve the effectiveness and efficiency of your testing process.

Lee Copeland, Software Quality Engineering
A New Paradigm for Collecting and Interpreting Bug Metrics

Many software test organizations count bugs; however, most do not derive much value from the practice, and other metrics can actually harm the quality of their software or their organization. Although valuable insights can be gained from examining find and fix rates or by graphing open bugs over time, you can be more easily fooled than informed by such metrics. Metrics used for control instead of inquiry tend to promote dysfunctional behavior whenever people know they are being measured. In this session, James Bach examines the subtleties of bug metrics analysis and shows examples of both helpful and misleading metrics from actual projects. Instead of the well-known Goal/Question/Metric paradigm, James presents a less intrusive approach to measurement that he describes as the Observe/Inquire/Model. Learn about the dynamics and dangers of measurement and a new approach to improve your metrics and the software you produce.

James Bach, Satisfice Inc
Using Defect Data to Make Real Quality Improvements

A large development organization was challenged to decrease production defects by at least 70 percent. Without extra money or time to install major process changes, what should be done? For a baseline, there was a production defect database that had been running at a steady state for over a year, but no way to size the many different projects and no appetite for either function points or measuring lines of code. In this interesting case study, Betsy Radley reports how they used approximations and sometimes crude assumptions to develop measurements from the defect data. These measurements identified applications that had the fewest product defects. Find out how they used that information to look for processes and tools used in these "good" applications and then applied them to the "bad" applications.

Betsy Radley, Nationwide Insurance Company
Metrics to Inspire your Software Project Team

Typically, organizations pick metrics they feel will accurately measure the results their projects are achieving. We know, too, that metrics can affect behavior, and, therefore, we try hard to pick unbiased metrics. For example, you might not use "lines of code" as a metric to track productivity because you know metrics might cause developers to write more inefficient code. However, you can do the opposite! Jan Scott shows how to choose metrics designed to improve the progress of your projects by inspiring people to improve. As your project progresses and problems develop, appropriately designed and biased metrics will help you solve problems before they get out of control. See examples of biased metrics that have had the desired effect of improving results without alienating those involved. You will even learn about certain metrics that can inspire management to perform tasks assigned to them.

Jan Scott, QB Software
Implementing and Sustaining a Measurement Program

Are you looking to install new measurements at the department or enterprise level? Are parts of your existing measurement program shaky? Starting a measurement program or revitalizing an existing one requires a good road map and checkpoints along the way. Janet Russac offers the fundamentals for establishing an organization-wide measurement program based on defined objectives. Find out about the principles of when to use metrics and when not to use them. Get a proven measurement program implementation strategy from this industry veteran, and take away an understanding of the key steps and attributes of a successful program. Make your measurements even more valuable by incorporating a benchmarking component into your program.

  • Key steps to a successful measurement program
  • Identification of key indicators of readiness and factors for success
Janet Russac, The David Consulting Group
Customer Focused Business Metrics throughout the SDLC

Focusing on the customer throughout the software development lifecycle (SDLC) is difficult to do. Teams often can become mired in technical problems, internal resource limitations, or other issues. Following the customer mantra of "Faster! Better! Cheaper!" Steve Wrenn offers measurement and process techniques that he has used to deliver projects on time, on budget, and, most importantly, meeting customers needs. By focusing on the development cycle from the outside in, his organization provides business-based metrics dashboards to monitor and adjust the project plan throughout the development project. Find out how their performance dashboard helps the team and the customer stay on course and drive directly to the targeted results. Discover an approach to determine what customers really want and match product development to customer expectations.

Steve Wrenn, Liberty Mutual Insurance Information Systems
Software Metrics State of The Practice

This presentation reviews the results of KLCI's Fourth Annual "best practices" study, including: Metrics "Best Practices"; Spending benchmarks for software metrics; Benefits of software metrics; Software measurements used; and Tools for software metrics.

Peter Kulik, KLCI Research Group


AgileConnection is one of the growing communities of the TechWell network.

Featuring fresh, insightful stories, is the place to go for what is happening in software development and delivery.  Join the conversation now!