Conference Presentations

STAREAST 2013 Keynote: The Mismeasure of Software: The Last Talk on Measurement You’ll Ever Need to Hear

Lee Copeland maintains that most organizations have some kind of metrics program—and almost all are ineffective. After explaining the concept of measurement, Lee describes two key reasons for these almost universal metrics program failures.

Lee Copeland, Software Quality Engineering
Keynote: The Mismeasure of Software: The Last Talk on Measurement You’ll Ever Need to Hear

Lee Copeland maintains that most organizations have some kind of metrics program—and almost all are ineffective. After explaining the concept of measurement, Lee describes two key reasons for these almost universal metrics program failures. The first major mistake people make is...

Lee Copeland, Software Quality Engineering
Lessons Learned from Forty-five Years of Software Measurement

Counting is easy. However, what makes measurement really valuable-and really hard to get right-is knowing what to count and what to do with the results. If your organization is mostly tracking resource usage, costs, and schedule data, it is making a big mistake. What about the users? The customers? The overall business strategy? Sharing the lessons he has learned from fighting-and surviving-many software measurement battles, Ed Weller offers a step-by-step approach for implementing a practical and valuable metrics program. After understanding what measures are most important to the business strategy and all stakeholders, the next step is to decide what data supports those measures and how to capture it. With data in hand, you can create simple and informative ways to make the resulting metrics visible and easy to digest. The biggest challenges-avoidance, disbelief, and rationalization-come next.

Edward Weller, Integrated Productivity Solutions, LLC
Data Collection and Analysis for Better Requirements [Better Software Conference East 2012]

According to the Standish group, 64 percent of features in systems are rarely-or never-used. How does this happen? Today, the work of eliciting the customers' true needs, which remains elusive, can be enhanced using data-driven requirements techniques. Brandon Carlson introduces data collection approaches and analysis techniques you can employ on your projects right away. Find out how to instrument existing applications and develop new requirements based on operational profiles of the current system. Learn to use A/B testing-a technique for trying out and analyzing alternative implementations-on your current system to determine which new features will deliver the most business value. With these tools at hand, you can help users and business stakeholders decide the best approaches and best new features to meet their real needs. Now is the time to take the guesswork out of requirements and get "Just the facts, Ma'am."

Brandon Carlson, Lean TECHniques, Inc.
The Dangers of the Requirements Coverage Metric

When testing a system, one question that always arises is, “How much of the system have we tested?” Coverage is defined as the ratio of “what has been tested” to “what there is to test.” One of the basic coverage metrics is requirements coverage-measuring the percentage of the requirements that have been tested. Unfortunately, the requirements coverage metric comes with some serious difficulties: Requirements are difficult to count; they are ideas, not physical things, and come in different formats, sizes, and quality levels. In addition, making a complete count of “what there is to test” is impossible in today’s hyper-complex systems. The imprecision of this metric makes it unreliable or even undefined and unusable. What is a test manager to do?

Lee Copeland, Software Quality Engineering
The Metrics Minefield

In many organizations, management demands measurements to help assess the quality of software products and projects. Are those measurements backed by solid metrics? How do we make sure that our metrics are reliably measuring what they're supposed to? What skills do we need to do this job well? Measurement is the art and science of making reliable and significant observations. Michael Bolton describes some common problems and risks with software measurement, and what we can do to address them. Learn to think critically about numbers, what they appear to measure and how they can be distorted. Improve the quality of the information that we're gathering to understand the relationship between observation, measurement, and metrics. Evaluate your measurements by asking probing questions about their validity.

Michael Bolton, DevelopSense, Inc.
Lessons from a DevOps Journey

In large financial institutions, treasury departments-specialized teams of traders and experts in liquidity, risk, accounting, financial forecasting, and quantitative analysis-manage the organization’s wealth and financial risk. These departments require large, complex, third-party software products that must change often to support the treasury’s complicated business processes. Matt Callanan describes how a team of developers and operations staff-the DevOps team-applied agile principles to the “last mile” and reduced software deployment from one week to one day. He discusses how their DevOps team collaborated to develop automation solutions to support ongoing deployment activities and solve many issues in the operational environment.

Matt Callanan, Independent
Agile Testing: Challenges Beyond the Easy Contexts (Agile Development & Better Software West 2012)

Don’t let anyone tell you differently: agile testing is hard! First, we have to get over the misconception that you don’t need testers within agile teams. Then, we have to integrate testers with the developers and engender a holistic quality approach. And those are only the challenges when the going is easy! In more difficult contexts, testing in agile environments is-well, even more difficult. Bob Galen explores how to handle testing in difficult contexts-lack of test automation capabilities, agile in highly regulated environments, testing when your team is spread globally and real-time interactions are nearly impossible, and more. He describes contexts and approaches for blending existing, traditional testing techniques with their agile counterparts. With real-world examples, Bob describes how teams have achieved a good working balance between the two-for example, in test planning and quality metrics reporting.

Bob Galen, Deutsche Bank
Optimal Project Performance: Factors that Influence Project Duration

Speedy delivery is almost always a primary project goal or a significant project constraint. To shorten project duration without sacrificing quality or budget, you need to know where to focus the team’s efforts. Mining the QSM database containing many quantitative metrics and numerous qualitative attributes, Paul Below shares the factors that have the greatest influence on project duration. While he’s at it, Paul debunks a couple of myths. For example, many managers consider team skill to be important in determining duration of software projects-not so. The most important factors are certain types of tooling, architecture, testing efficiency, and management/leadership skills, which Paul explores in depth. Learn a technique for normalizing your projects for size by computing the standardized residual of duration.

Paul Below, QSM, Inc.
Simple Metrics for Starters

Measurements and metrics are a hot topic again. Theorists often rail against them as meaningless and potentially harmful. Practitioners fear them because they don’t want to be metric-ed out of a job. Managers want them so they can better understand what is happening in the software development lifecycle and try to make their processes more efficient. David Gilbert shares the simple set of measurements and metrics Raymond James has implemented and describes the practical benefits they have gained. By first asking stakeholders what they really wanted to understand and then developing metrics to support their goals, David and his team have made measurement work at their company. They educated the stakeholders on key metrics concepts-first order measures and subjective relativity-and established a common model everyone agreed to follow.

David Gilbert, Raymond James


AgileConnection is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.