Measure Outcomes Rather than Output
An alternative approach I'd suggest, which is similar to one suggested by Laura Brandenburg in her post “ How Do You Measure the Success of a Business Analyst,” is to measure the effectiveness of business analysis by looking at whether an organization's projects are meeting their objectives. The first difference between this approach and the one mentioned above is that it measures the effectiveness of analysis work as opposed to the analysts who are doing it. This implies the belief that while a person with the title of analyst may be the primary driver of analysis activities, several members in the team have a part to play.
The other main difference is that the measurement focuses on outcomes rather than outputs. In other words, you are more concerned about whether business objectives are being met than whether intermediate outputs (requirements documents) are being produced. This approach assumes that you are more interested in using measurements to help your delivery team members improve their processes and the resulting outcome.
Some measures that can be used to determine the effectiveness of business analysts with this mindset include: the percentage of efforts where there is genuine stakeholder consensus about the objectives of the project; the percentage of those projects where the objectives were met; and the percentage of users who adopted the solution. You can find more ideas for possible measures of this sort in Laura Brandenburg's post mentioned above as well as Joy Beatty's post “ Measuring the Success of a Requirements (or Business Analysis) Center of Excellence .”
How to Make Meaningful Metrics
Last April I had the opportunity to collaborate with Todd Brasel, a QA engineer and project manager at Pitney Bowes, to share his approach to working with metrics in a business setting. During that collaboration, Todd introduced me to six key steps to creating a metric for process improvement, which I think is helpful when trying to establish measures of business analysis.
The first step is to identify the problem you are trying to solve or a question you are trying to answer with the metric. For example, we may be interested in knowing whether our business analysis efforts are being successful in driving agreement among a project’s stakeholders on the objectives of that project.
The second step is to identify quantitative ways to represent progress toward the goal. Regarding reaching an agreement on business objectives, we want to know the percentage of projects where the stakeholders agree on the objectives of the project.
The third step is to determine criteria for measurement and how to get the values. We want to see if the stakeholders for a given project all have the same objective(s) in mind for that project.
The fourth step is to identify data to collect and how to collect it. You don’t want data collection to be onerous, so think about what data do you already have that can tell you that you need to know. If nothing exists right off the bat, think about what is the simplest way to gather that data. In the case of our example, we would individually poll the stakeholders of a project to see if they give the same answer for the business objectives.
The fifth step is to identify the baseline, which gives you a comparison point against which you can identify improvements that happen after you put your plan into action.
The final step is to put your plan into action and review the results. This step assumes that you have a specific reason for measuring the performance of business analysis and that your main purpose is to check progress.