Agile discussions often focus on stories, backlogs, development, and testing. At Experian they also brought product strategy management and strategy into the agile fold to ensure their teams were in lock-step with customer requirements and priorities. That resulted in the delivery of...
As semi-scientific software professionals, we like the idea of measuring our work. In some cases, our bosses like the idea much more than we do. Yet, meaningful software development metrics are notoriously challenging to define, and many people have given up trying because metrics often...
When testing a system, one question that always arises is, “How much of the system have we tested?” Coverage is defined as the ratio of “what has been tested” to “what there is to test.” One of the basic coverage metrics is requirements coverage-measuring the percentage of the requirements that have been tested. Unfortunately, the requirements coverage metric comes with some serious difficulties: Requirements are difficult to count; they are ideas, not physical things, and come in different formats, sizes, and quality levels. In addition, making a complete count of “what there is to test” is impossible in today’s hyper-complex systems. The imprecision of this metric makes it unreliable or even undefined and unusable. What is a test manager to do?
In many organizations, management demands measurements to help assess the quality of software products and projects. Are those measurements backed by solid metrics? How do we make sure that our metrics are reliably measuring what they're supposed to? What skills do we need to do this job well? Measurement is the art and science of making reliable and significant observations. Michael Bolton describes some common problems and risks with software measurement, and what we can do to address them. Learn to think critically about numbers, what they appear to measure and how they can be distorted. Improve the quality of the information that we're gathering to understand the relationship between observation, measurement, and metrics. Evaluate your measurements by asking probing questions about their validity.
Measurements and metrics are a hot topic again. Theorists often rail against them as meaningless and potentially harmful. Practitioners fear them because they don’t want to be metric-ed out of a job. Managers want them so they can better understand what is happening in the software development lifecycle and try to make their processes more efficient. David Gilbert shares the simple set of measurements and metrics Raymond James has implemented and describes the practical benefits they have gained. By first asking stakeholders what they really wanted to understand and then developing metrics to support their goals, David and his team have made measurement work at their company. They educated the stakeholders on key metrics concepts-first order measures and subjective relativity-and established a common model everyone agreed to follow.
Are your testing and quality assurance activities adding significant value in the eyes of your stakeholders? Do you have difficulty convincing decision-makers that they need to invest more in improving quality? Selecting metrics that stakeholders understand will get your improvement project or program past the pilot phase and reduce the risk of having it stopped in its tracks. Todd Brasel and Kent McDonald show you how to avoid getting bogged down in the minute details of test results reporting and approach quality measurement from a systems perspective. They offer practical lessons about using metrics to promote quality improvement initiatives to upper managers and executives-the people who make the real decisions about quality. Learn about common traps and problems of measurement programs and how to avoid them.
Many software test organizations count bugs; however, most do not derive much value from the practice, and other metrics can actually harm the quality of their software or their organization. Although valuable insights can be gained from examining find and fix rates or by graphing open bugs over time, you can be more easily fooled than informed by such metrics. Metrics used for control instead of inquiry tend to promote dysfunctional behavior whenever people know they are being measured. In this session, James Bach examines the subtleties of bug metrics analysis and shows examples of both helpful and misleading metrics from actual projects. Instead of the well-known Goal/Question/Metric paradigm, James presents a less intrusive approach to measurement that he describes as the Observe/Inquire/Model. Learn about the dynamics and dangers of measurement and a new approach to improve your metrics and the software you produce.
In this case study of an award winning project, Andy Redwood describes how his team used "best shoring" of testing services to reduce costs, reuse assets, and get the best from their test automation tools. In an enterprise-wide transformation process at a large investment bank, his team used available infrastructure, technology, tools, and process to reduce business risk from software changes with a new automated regression test suite. With some facts and figures and a little hindsight, you will learn how to provide global, automated testing services on a twenty-four hours a day/seven days a week, on-demand basis. Find out what metrics you need to accurately measure the costs and benefits of a "follow the sun" test automation strategy.
A successful outsource project that measurably improved business resilience
Are you looking to install new measurements at the department or enterprise level? Are parts of your existing measurement program shaky? Starting a measurement program or revitalizing an existing one requires a good road map and checkpoints along the way. Janet Russac offers the fundamentals for establishing an organization-wide measurement program based on defined objectives. Find out about the principles of when to use metrics and when not to use them. Get a proven measurement program implementation strategy from this industry veteran, and take away an understanding of the key steps and attributes of a successful program. Make your measurements even more valuable by incorporating a benchmarking component into your program.
Key steps to a successful measurement program
Identification of key indicators of readiness and factors for success
Good data feedback of software measurements is critical when analyzing measurement data, for drawing conclusions, and as the basis for taking action. Feedback to those involved in the activities being measured helps validate the data as well. In this presentation Ben Linders shows examples of how Ericsson Telecommunications delivers feedback at two levels: projects and the total development center. Although the basics are similar, the application differs, and the key success factors depend on the level and the audience. At the project level, you will see how the team reviews defect data, including defect classifications and test matrices. For development center feedback, you will see how line management and technical engineers review data and analyze information based on a balanced score card approach with measurable goals.